Define evaluation criteria
Write criteria before comparing prompts.
Lesson 1 of 4 · Learn it by trying itWatch: Evaluating and improving results
An original animated overview. No sound—follow the on-screen explanation, or read the transcript.
Read the video transcript
Define evaluation criteria. Write criteria before comparing prompts.
Build a test set. Use varied cases including edge cases and adversarial inputs.
Compare reliably. Keep relevant conditions consistent and record model, prompt version and date.
Act on evidence. Group failures, revise one factor and rerun relevant checks.
Created for AI For Numpties · Original course material · Hosted with this website
Getting your practice ready…
Explore the idea with an example
Write criteria before comparing prompts. Use representative, permitted inputs and a baseline. Include accuracy, safety and usefulness.
Write an evaluation rubric before comparing outputs. Separate factual accuracy, completeness, usefulness, tone and safety so a fluent answer cannot hide a serious failure. Define what counts as unacceptable. A customer message that invents a refund should fail even if it sounds friendly. Use examples of strong and weak outputs to help reviewers apply the criteria consistently.