Export formats (detail)
Export formats (detail)
QAVE Studio supports multiple export formats to suit different use cases. All formats include tuple metadata for full traceability.
Supported Formats
| Format | Use Case |
|---|---|
| JSONL | Phoenix evaluation, fine-tuning pipelines, general ML datasets |
| Simulator JSONL | Voice simulators, Qurator integration, A2A workflows |
| Pytest | CI/CD pipelines, automated regression testing |
JSONL Format
Use case: Phoenix evaluation, fine-tuning pipelines, general ML datasets
Structure:
{
"id": "test-case-uuid",
"tuple_id": "tuple-uuid",
"tuple_label": "Intent=Complaint | Tone=Frustrated | UserType=New User",
"dimension_values": [
{ "dimension_name": "Intent Type", "value": "Complaint" },
{ "dimension_name": "Emotional Tone", "value": "Frustrated" },
{ "dimension_name": "User Type", "value": "New User" }
],
"conversation_initiator": "user",
"language": "en",
"conversation_history": [
{ "role": "user", "content": "I've been waiting 3 days for a response!" },
{ "role": "assistant", "content": "I'm sorry to hear that. Let me look into your case." }
],
"expected_behavior": "Bot acknowledges urgency and escalates to human agent",
"criteria": [
"Acknowledges the delay without dismissing the user",
"Offers immediate escalation path",
"Does not provide false resolution timelines"
],
"source": "qave_synthetic"
}Field Reference:
| Field | Type | Description |
|---|---|---|
id | string (UUID) | Test case ID |
tuple_id | string (UUID) | Source tuple |
tuple_label | string | Human-readable tuple name |
dimension_values | array | [\{dimension_name, value\}] |
conversation_initiator | string | "user" or "bot" |
language | string | ISO 639-1 output language |
conversation_history | array | OpenAI message format |
expected_behavior | string | Expected system behavior |
criteria | array | Evaluation criteria |
source | string | "qave_synthetic" |
Simulator JSONL Format
Use case: Voice simulators, Qurator integration, A2A workflows
Structure:
{
"id": "test-case-uuid",
"tuple_id": "tuple-uuid",
"tuple_label": "Intent=Complaint | Tone=Frustrated | UserType=New User",
"dimension_values": [
{ "dimension_name": "Intent Type", "value": "Complaint" },
{ "dimension_name": "Emotional Tone", "value": "Frustrated" }
],
"conversation_initiator": "user",
"language": "en",
"messages": [
{
"role": "user",
"content": "I've been waiting 3 days for a response!"
},
{
"role": "assistant",
"content": "I'm sorry to hear that. Let me look into your case."
}
],
"expected_behavior": "Bot acknowledges urgency and escalates to human agent",
"criteria": ["Acknowledges the delay", "Offers escalation path"],
"artifacts": {},
"metrics": {}
}Key differences from JSONL:
- Uses
messagesarray format (Qurator-compatible) - No
personaobject (replaced bydimension_values) - Includes
artifactsandmetricsplaceholders (filled by simulator)
Pytest Format
Use case: CI/CD pipelines, automated regression testing
Structure:
def test_case_uuid_intent_complaint_tone_frustrated():
"""
Tuple: Intent=Complaint | Tone=Frustrated | UserType=New User
Language: en
Initiator: user
Expected: Bot acknowledges urgency and escalates to human agent
"""
input_text = "I've been waiting 3 days for a response!"
expected_behavior = "Bot acknowledges urgency and escalates to human agent"
criteria = [
"Acknowledges the delay without dismissing the user",
"Offers immediate escalation path",
]
# Test implementation
assert result_meets_criteria(expected_behavior, criteria)Tuple label is included in both the function name and docstring.
Config Matrix
All formats support config matrix expansion for audio variation testing:
{
"configuration": {
"audio_speed": [0.9, 1.0, 1.1],
"accent": ["neutral", "us", "uk"],
"snr": [20, 30, 40]
}
}This generates all combinations: 3 speeds × 3 accents × 3 SNR = 27 variations per test case.
Smart Sampling
When Qurator is integrated, use the Smart Sampler to optimize config selection:
- Reduces export size while maintaining coverage
- Prioritizes configs likely to reveal issues
- Learns from evaluation history
Requirements: Qurator API integration active + evaluation history available.
Export Process
- Navigate to Exports → click Trigger Export
- Select suite(s): Latest Suite, All Approved Suites, or a specific suite
- Choose format: JSONL, Simulator JSONL, or Pytest
- Export runs asynchronously (progress tracked via WebSocket)
- Download using the provided link when complete (presigned URL, valid 1 hour)
Format Comparison
| Feature | JSONL | Simulator JSONL | Pytest |
|---|---|---|---|
| Tuple traceability | ✅ | ✅ | ✅ (in docstring) |
| Dimension values | ✅ | ✅ | ✅ (in docstring) |
| Message format | OpenAI | OpenAI (strict) | N/A |
| Config matrix | ✅ | ✅ | ✅ |
| Smart sampling | ✅ | ✅ | ❌ |
| Artifacts placeholders | ❌ | ✅ | ❌ |
Qurator Integration
Tuple metadata is passed to Qurator via the artifacts field in the evaluation request:
EvaluationRequest(
messages=conversation_history,
criteria=test_case.criteria,
source="qave_synthetic",
artifacts={
"tuple_id": str(test_case.tuple_id),
"tuple_label": test_case.tuple_metadata.get("label"),
"dimension_values": test_case.tuple_metadata.get("dimension_values"),
}
)This is additive — no breaking changes to the Qurator integration. Qurator stores the artifacts alongside the evaluation result.