Project evals & analysis
Error analysis
Annotate runs or uploads, then generate project failure modes.
Error Analysis (/error-analysis) helps you label conversations from Assurance Lab runs or uploaded transcripts, then generate project-level failure modes.
How to annotate
- Open Error Analysis.
- Choose Lab runs or upload conversations (partial uploads are fine).
- Annotate each conversation: Pass, Fail, or Unsure, and add observations on Fail / Unsure.
- Unannotated conversations are skipped when you submit.
Generate failure modes
Submit needs at least three Fail or Unsure annotations with observations. QAVE Studio proposes failure modes you can promote into Project Evals.
Tips
- Prefer short, concrete observations (“hallucinated policy id”) over vague notes.
- Re-run after a model change to see whether the same modes still appear.