Prompt Testing Assistant
JHC Consulting
Remote
The Role
In this position, you will be responsible for the mechanical and intellectual testing of prompts. The day-to-day work is focused on the precision of language and the observation of how small changes in syntax can lead to significant changes in reasoning. Your work will inform the broader advisory mandates handled by our principals.
- Designing and executing test cases to evaluate how language models respond to specific instructions.
- Identifying logical errors, contradictions, and factual inaccuracies in model outputs.
- Drafting short memoranda that summarize the results of your testing cycles.
- Isolating specific linguistic triggers that cause models to stray from the intended logical path.
- Documenting the limits of the models we use, specifically regarding their ability to process complex governance or strategy questions.
- Comparing the outputs of different model versions to ensure consistency across our internal tools.
- Reviewing the clarity of the instructions themselves to ensure they are free of ambiguity.
We are looking for individuals who view writing as a form of thinking. This is an entry level role, making it suitable for students, recent graduates, or those early in their professional life who possess a high degree of intellectual maturity. You do not need a background in computer science, but you must have a sharp eye for the written word.
- A background in a field that requires close reading and precise writing, such as philosophy, law, history, or literature.
- The ability to follow complex instructions without direct supervision.
- A basic understanding of how large language models function and their common failure points.
- Discipline in documenting your findings clearly and without the use of filler language.
- Comfort working in a fully remote environment where clear written communication replaces most verbal meetings.
- A skeptical mind that naturally seeks out the gaps in an argument.
- Strong organizational skills to manage multiple testing threads simultaneously.
JHC Consulting is an independent advisory practice. We do not participate in the standard ceremonies of corporate consulting, such as the production of slide decks or lengthy meetings. Instead, we focus on the discipline of the written word. We believe that a signed memorandum is the ultimate record of a thought process. In this firm, the person who conducts the reading and the analysis is the person who authors the output. This model of individual accountability ensures that every claim can be traced back to a clear assumption. As a Prompt Testing Assistant, you will follow this same model, producing clear records of your testing that can be relied upon by our principals for years to come.
Working at JHC Consulting
Our firm is small by design and operates with a focus on individual autonomy. This role is strictly remote, allowing you to manage your hours from wherever you are, provided you meet the deadlines of the mandate. We value the courage to name uncomfortable truths and the intellectual rigor required to challenge the internal consensus of a project. While this is a part-time contract role, the position is paid and the compensation is competitive for the entry-level market. We treat every contributor as a professional, expecting a high degree of discretion and attention to detail in return for a flexible, remote work arrangement.
At JHC Consulting, every page of work concludes with the author's name. This practice reflects our commitment to individual responsibility and the high standard we set for our counsel. When you test a prompt or draft a notes memorandum, you are creating a record that stands as a testament to your analytical ability. We take these contributions seriously, as they form the foundation of the advice we provide to board committees and chief executives. We look for colleagues who understand that the quality of their signed work is their most important professional asset.