Assurance SDK
SDK overview
Packages, responsibilities, and the honest boundary between eval-only and SUT drive.
Packages (code names)
| Package | Role |
|---|---|
qurator client | Qurator.evaluate → sovereign/pilot evaluate endpoints |
qurator-sdk-agent | Polling agent, JSONL loader, Langfuse tracer, local agent API |
QAVE Studio qurator_client | Studio backend calls into Gateway for eval-only workflows |
What works today
- Eval-only: caller supplies
execution_results(messages / traces); QAVE Studio or Gateway scores them - Live suite fetch: project SDK API key →
GET /sdk/projects/{id}/test-suitesand.../cases(see QAVE Studio suite fetch API) - Langfuse → Gateway: fetch + normalize +
POST /v1/assure/evaluate - JSONL agent loop: polls inputs and calls MCP
evaluate_conversation(reference SUT MCP is not closed — use mock for demos)
QAVE Studio workflow (eval-only)
POST /workflows/execute-and-evaluateRequires suite + cases APPROVED and a non-empty execution_results payload. This path does not load target_config or drive the SUT by itself.
A single execute_and_evaluate call that drives your SUT and scores in one step is coming soon. Use fetch + evaluate today.