'Produce a good summary' sounds reasonable until reviewers disagree about whether a missing qualification matters. One values brevity. The other needs every exception that could change the decision.
Photo by Jakub Żerdzicki on Unsplash.
AI output acceptance criteria state what a usable result must contain, what it must not claim and when the task should return to a person. Write them before tuning the prompt so quality has a stable meaning.
Start with the downstream decision
Ask what somebody will do with the output. A reading aid and an instruction to update a customer record need different standards.
In an illustrative supplier-proposal summary, the reader must compare scope, exclusions and proposed dates. A polished paragraph omitting an exclusion fails the task even if every included sentence is accurate.
Separate essential checks from preferences
Use three groups:
Required: information necessary for the intended decision.
Prohibited: unsupported facts, commitments or actions outside scope.
Preferred: presentation choices such as ordering and length.
Do not let a strong style score compensate for an unsupported price or wrong client identity. Define failures that stop use regardless of how readable the rest is.
Make each criterion observable
Replace 'accurate and professional' with checks a reviewer can apply. For the proposal example, require supplier name, source version, deliverables, exclusions and unresolved conditions. Require unknown dates to remain unknown.
Include a rule for conflicting information. The right output may be 'the proposal contains two start dates' rather than a confident choice between them.
Use an example of a pass, a correct escalation and a failure. These show distinctions more clearly than adjectives alone. Explain what the reviewer should do after each result.
Check whether people can apply the rules
Ask two reviewers to assess outputs independently, then discuss disagreements. If a criterion cannot be applied consistently, clarify its terms or the business rule behind it.
Keep criteria under version control. When the purpose changes, revisit them before deciding whether the tool is good enough. Output accepted for internal preparation is not automatically approved for client distribution.
Related guidance: guide to mapping the work before changing it.
If your team is debating whether AI results are 'good', Hili Consulting can help turn that judgement into acceptance checks tied to the work.
