05 · Evaluation

How to know whether your AI system works, keeps working after changes, and holds up against attacks. Evals are the "tests" of AI engineering.

You'll be able to: build a golden dataset, write code-based and LLM-judge graders, measure RAG retrieval and answers separately, and run automated red-team scans.

#	Note	Level	Time
1	Why evals are the tests of AI	Beginner	10 min
2	Building a golden dataset	Beginner	10 min
3	LLM-as-a-judge	Intermediate	12 min
4	Measuring RAG quality	Intermediate	12 min
5	Red-teaming LLM apps	Intermediate	12 min

Tip

Design a 10-case eval for any chatbot you use today: 4 common questions, 2 hard ones, 2 out of scope, 2 adversarial. Write what a good answer must and must not include. You've just done the most important part of evaluation.

Projects: P3 and P4 both include eval suites that run in CI with minimum scores.

Next: 06 · Agent memory