QAVE Studio Docs
QAVE Studio Docs
Sign inGet started — the Studio loop
Getting StartedFirst assurance runQuick assure
ConceptsLifecycleGlossaryHow QAVE Studio worksComing soon
Account & workspaceSignup and loginPeopleDemo creditsRolesSettings
Generate simulation suitesContext templatePII and field enhanceDimensionsAgents and plannerDraft review and Save
ExportExport formatsExport formats (detail)
Assurance SDKSDK overviewQAVE Studio suite fetch APIAgent and JSONL queueLangfuse bridge
Execute and tracesRecipe chatbot (reference SUT)TracingFetch tracing
AssureEvaluate APIRubrics and evidenceReading scores
History
WorkbenchGolden data
CoverageCoverage dashboardCoverage strategiesCoverage advisorTest case quality
RAG AssuranceRAG runsRAG datasetsRAG analytics
Project evalsError analysisUnderstanding scores
Studio guidesDimensionsTuplesConversation modesMulti-lingual testingWorkflowsHuman reviewBest practices
ReferenceAPI and packagesTroubleshooting
Export

Export formats (detail)

Export formats (detail)

QAVE Studio supports multiple export formats to suit different use cases. All formats include tuple metadata for full traceability.

Supported Formats

FormatUse Case
JSONLPhoenix evaluation, fine-tuning pipelines, general ML datasets
Simulator JSONLVoice simulators, Qurator integration, A2A workflows
PytestCI/CD pipelines, automated regression testing

JSONL Format

Use case: Phoenix evaluation, fine-tuning pipelines, general ML datasets

Structure:

{
  "id": "test-case-uuid",
  "tuple_id": "tuple-uuid",
  "tuple_label": "Intent=Complaint | Tone=Frustrated | UserType=New User",
  "dimension_values": [
    { "dimension_name": "Intent Type", "value": "Complaint" },
    { "dimension_name": "Emotional Tone", "value": "Frustrated" },
    { "dimension_name": "User Type", "value": "New User" }
  ],
  "conversation_initiator": "user",
  "language": "en",
  "conversation_history": [
    { "role": "user", "content": "I've been waiting 3 days for a response!" },
    { "role": "assistant", "content": "I'm sorry to hear that. Let me look into your case." }
  ],
  "expected_behavior": "Bot acknowledges urgency and escalates to human agent",
  "criteria": [
    "Acknowledges the delay without dismissing the user",
    "Offers immediate escalation path",
    "Does not provide false resolution timelines"
  ],
  "source": "qave_synthetic"
}

Field Reference:

FieldTypeDescription
idstring (UUID)Test case ID
tuple_idstring (UUID)Source tuple
tuple_labelstringHuman-readable tuple name
dimension_valuesarray[\{dimension_name, value\}]
conversation_initiatorstring"user" or "bot"
languagestringISO 639-1 output language
conversation_historyarrayOpenAI message format
expected_behaviorstringExpected system behavior
criteriaarrayEvaluation criteria
sourcestring"qave_synthetic"

Simulator JSONL Format

Use case: Voice simulators, Qurator integration, A2A workflows

Structure:

{
  "id": "test-case-uuid",
  "tuple_id": "tuple-uuid",
  "tuple_label": "Intent=Complaint | Tone=Frustrated | UserType=New User",
  "dimension_values": [
    { "dimension_name": "Intent Type", "value": "Complaint" },
    { "dimension_name": "Emotional Tone", "value": "Frustrated" }
  ],
  "conversation_initiator": "user",
  "language": "en",
  "messages": [
    {
      "role": "user",
      "content": "I've been waiting 3 days for a response!"
    },
    {
      "role": "assistant",
      "content": "I'm sorry to hear that. Let me look into your case."
    }
  ],
  "expected_behavior": "Bot acknowledges urgency and escalates to human agent",
  "criteria": ["Acknowledges the delay", "Offers escalation path"],
  "artifacts": {},
  "metrics": {}
}

Key differences from JSONL:

  • Uses messages array format (Qurator-compatible)
  • No persona object (replaced by dimension_values)
  • Includes artifacts and metrics placeholders (filled by simulator)

Pytest Format

Use case: CI/CD pipelines, automated regression testing

Structure:

def test_case_uuid_intent_complaint_tone_frustrated():
    """
    Tuple: Intent=Complaint | Tone=Frustrated | UserType=New User
    Language: en
    Initiator: user
    Expected: Bot acknowledges urgency and escalates to human agent
    """
    input_text = "I've been waiting 3 days for a response!"
    expected_behavior = "Bot acknowledges urgency and escalates to human agent"
    criteria = [
        "Acknowledges the delay without dismissing the user",
        "Offers immediate escalation path",
    ]
    # Test implementation
    assert result_meets_criteria(expected_behavior, criteria)

Tuple label is included in both the function name and docstring.


Config Matrix

All formats support config matrix expansion for audio variation testing:

{
  "configuration": {
    "audio_speed": [0.9, 1.0, 1.1],
    "accent": ["neutral", "us", "uk"],
    "snr": [20, 30, 40]
  }
}

This generates all combinations: 3 speeds × 3 accents × 3 SNR = 27 variations per test case.


Smart Sampling

When Qurator is integrated, use the Smart Sampler to optimize config selection:

  • Reduces export size while maintaining coverage
  • Prioritizes configs likely to reveal issues
  • Learns from evaluation history

Requirements: Qurator API integration active + evaluation history available.


Export Process

  1. Navigate to Exports → click Trigger Export
  2. Select suite(s): Latest Suite, All Approved Suites, or a specific suite
  3. Choose format: JSONL, Simulator JSONL, or Pytest
  4. Export runs asynchronously (progress tracked via WebSocket)
  5. Download using the provided link when complete (presigned URL, valid 1 hour)

Format Comparison

FeatureJSONLSimulator JSONLPytest
Tuple traceability✅✅✅ (in docstring)
Dimension values✅✅✅ (in docstring)
Message formatOpenAIOpenAI (strict)N/A
Config matrix✅✅✅
Smart sampling✅✅❌
Artifacts placeholders❌✅❌

Qurator Integration

Tuple metadata is passed to Qurator via the artifacts field in the evaluation request:

EvaluationRequest(
    messages=conversation_history,
    criteria=test_case.criteria,
    source="qave_synthetic",
    artifacts={
        "tuple_id": str(test_case.tuple_id),
        "tuple_label": test_case.tuple_metadata.get("label"),
        "dimension_values": test_case.tuple_metadata.get("dimension_values"),
    }
)

This is additive — no breaking changes to the Qurator integration. Qurator stores the artifacts alongside the evaluation result.

Export formats

JSONL, Simulator JSONL, Pytest, Excel, and Editable JSON contracts.

Assurance SDK

The unifying runtime for execution, trace normalization, and assure calls.

On this page

Supported FormatsJSONL FormatSimulator JSONL FormatPytest FormatConfig MatrixSmart SamplingExport ProcessFormat ComparisonQurator Integration