01 / DECIDE
Architecture decision
Use it for prompt, model, tool, retrieval, and orchestration changes where anecdotal demos cannot prove stability.
Define representative inputs, human-readable criteria, automated graders where appropriate, and a baseline before changing the system. Map input, output, state, and side effects as one observable path before deciding which layer owns make evaluation the release gate.
02 / BUILD
Three-step implementation
- 01
Frame the contract
Write down the caller, data classification, success condition, timeout, cancellation, and ownership. Use it for prompt, model, tool, retrieval, and orchestration changes where anecdotal demos cannot prove stability.
- 02
Build one narrow path
Implement one end-to-end path with request correlation, typed state, and reversible failure handling. Define representative inputs, human-readable criteria, automated graders where appropriate, and a baseline before changing the system.
- 03
Prove the outcome
Turn acceptance into a repeatable fixture, contract test, or browser test. Release automation blocks when required slices regress beyond an agreed threshold and records the comparison artifact.
03 / BOUND
Production boundary
Do not optimize only for one score; include safety, refusal, latency, cost, and regression slices relevant to users.
04 / PROVE
Acceptance evidence
Release automation blocks when required slices regress beyond an agreed threshold and records the comparison artifact.
SOURCE / HTTP
Reproducible source probe
curl -fsSI 'https://developers.openai.com/api/docs/guides/evals' | sed -n '1,5p'