QAVE Studio Docs
QAVE Studio Docs
Sign inGet started — the Studio loop
Getting StartedFirst assurance runQuick assure
ConceptsLifecycleGlossaryHow QAVE Studio worksComing soon
Account & workspaceSignup and loginPeopleDemo creditsRolesSettings
Generate simulation suitesContext templatePII and field enhanceDimensionsAgents and plannerDraft review and Save
ExportExport formatsExport formats (detail)
Assurance SDKSDK overviewQAVE Studio suite fetch APIAgent and JSONL queueLangfuse bridge
Execute and tracesRecipe chatbot (reference SUT)TracingFetch tracing
AssureEvaluate APIRubrics and evidenceReading scores
History
WorkbenchGolden data
CoverageCoverage dashboardCoverage strategiesCoverage advisorTest case quality
RAG AssuranceRAG runsRAG datasetsRAG analytics
Project evalsError analysisUnderstanding scores
Studio guidesDimensionsTuplesConversation modesMulti-lingual testingWorkflowsHuman reviewBest practices
ReferenceAPI and packagesTroubleshooting
Studio guides

Multi-lingual testing

Multi-lingual testing

QAVE Studio supports generating test cases in multiple languages. This guide explains how to set up multi-lingual testing and interpret language coverage data.


Setting Primary Languages

In the Context Template, the Primary Languages field accepts a comma-separated list of ISO 639-1 language codes.

Example:

en, hi, ar, es, fr

These become the target languages for test generation and are tracked in the Coverage Dashboard's language coverage section.

Supported ISO 639-1 codes (examples):

CodeLanguage
enEnglish
hiHindi
arArabic
esSpanish
frFrench
ptPortuguese
deGerman
zhChinese (Mandarin)
jaJapanese
koKorean
taTamil
teTelugu
mrMarathi
bnBengali

Dimension Value Translations

For multi-lingual projects, you can add translated values to each dimension. These translations are used to constrain the generated conversations in the target language.

Adding a translation:

  1. Open a dimension in the Dimensions page
  2. Click on a value (e.g., "Frustrated")
  3. Click Add Translation
  4. Select the language code and enter the translated value (e.g., hi → "निराश")

When QAVE Studio generates a test case for the Hindi language, it uses "निराश" instead of "Frustrated" in the dimension constraint, helping the LLM generate more natural conversations.


Language Selector on the Generate Page

On the Generate page, use the Language dropdown to select the output language for this generation run.

  • If you select hi (Hindi), all generated conversations will be in Hindi
  • The dimension value translations for Hindi will be used in the generation prompt
  • The generated test cases will have language: "hi" in their metadata

Recommended workflow for multi-lingual coverage:

  1. Generate English test cases first (baseline coverage)
  2. Check the Coverage Dashboard → Language Coverage section
  3. Use "Generate in [language]" shortcuts for each uncovered language
  4. Repeat until target language coverage is achieved

How Language Affects Generated Conversations

When a non-English language is selected:

  1. The generation prompt includes a language instruction: "Generate this conversation entirely in [Language Name]."
  2. If dimension value translations are available for that language, they are included in the tuple constraint
  3. The LLM generates the full conversation (user and bot messages) in the target language
  4. The test case's language field is set to the ISO 639-1 code

Note: The expected_behavior and criteria fields are always generated in English for consistency in evaluation, regardless of the conversation language.


Coverage Across Languages

The Coverage Dashboard tracks language coverage:

  • A progress bar shows test case counts per configured language
  • Languages with 0 test cases are highlighted as gaps
  • Each language bar has a "Generate in [language]" shortcut that takes you to the Generate page with that language pre-selected

Target: All languages listed in the Context Template's primary_languages field should have test cases.


Language Coverage vs. Pairwise Coverage

Language coverage and pairwise coverage are tracked separately:

  • Pairwise coverage is calculated across all tuples regardless of language
  • Language coverage tracks which languages have test cases

To achieve both full pairwise coverage AND language coverage, you need to generate test cases for each language across your approved tuples. The Coverage Dashboard shows both metrics independently.


Troubleshooting Multi-Lingual Generation

Issue: Generated conversations mix languages (code-switching)

  • Ensure dimension value translations are added for the target language
  • Increase the specificity of the generation focus: "Generate this conversation entirely in [language]. Do not use any English."

Issue: Test cases for a language have poor quality

  • Check that the configured model in DIMENSION_GENERATOR_MODEL supports the target language
  • For Indic languages (Hindi, Tamil, etc.), prefer models with strong Indic support
  • For Arabic, prefer models with Arabic language support

Issue: Language not showing up in Coverage Dashboard

  • Verify the language code is listed in the Context Template's primary_languages field
  • Only languages explicitly listed in primary_languages appear in the language coverage section

Conversation modes

Conversation modes

Workflows

How Studio runs generation and scrubbing jobs in the background.

On this page

Setting Primary LanguagesDimension Value TranslationsLanguage Selector on the Generate PageHow Language Affects Generated ConversationsCoverage Across LanguagesLanguage Coverage vs. Pairwise CoverageTroubleshooting Multi-Lingual Generation