QAVE Studio Docs
QAVE Studio Docs
Sign inGet started — the Studio loop
Getting StartedFirst assurance runQuick assure
ConceptsLifecycleGlossaryHow QAVE Studio worksComing soon
Account & workspaceSignup and loginPeopleDemo creditsRolesSettings
Generate simulation suitesContext templatePII and field enhanceDimensionsAgents and plannerDraft review and Save
ExportExport formatsExport formats (detail)
Assurance SDKSDK overviewQAVE Studio suite fetch APIAgent and JSONL queueLangfuse bridge
Execute and tracesRecipe chatbot (reference SUT)TracingFetch tracing
AssureEvaluate APIRubrics and evidenceReading scores
History
WorkbenchGolden data
CoverageCoverage dashboardCoverage strategiesCoverage advisorTest case quality
RAG AssuranceRAG runsRAG datasetsRAG analytics
Project evalsError analysisUnderstanding scores
Studio guidesDimensionsTuplesConversation modesMulti-lingual testingWorkflowsHuman reviewBest practices
ReferenceAPI and packagesTroubleshooting
Coverage

Coverage advisor

Coverage advisor

This guide covers the Domain Coverage Advisor (Layer 2 taxonomy analysis — the card on the Coverage Dashboard). For the Ask Coverage conversational chat (the floating button, bottom-right), see the Coverage Guide.

What is the Coverage Advisor?

The Domain Coverage Advisor analyses your project's context template and tells you which AI-testing categories are covered by your current test space — and which are missing.

It applies an 8-category Conversational AI taxonomy (based on ISTQB, ISO 25010, and industry chatbot testing standards) to your test dimensions and tuples, giving you a domain-aware assessment rather than a generic checklist.


The 8 Coverage Categories

The advisor evaluates your test space against these categories:

#CategoryWhat it checks
1Core FunctionalityAre the primary declared use cases covered?
2Edge Cases & Boundary ConditionsInputs at limits, unexpected input types, empty/null values
3Error Handling & Failure ModesKnown failure scenarios declared in your context template
4User Intent VariationsMultiple ways to express the same intent (paraphrasing, languages)
5Persona & Context DiversityDifferent user segments, experience levels, and roles
6Multi-Turn & Conversation FlowConversation continuity, context retention, session management
7Security & CompliancePII handling, injection attempts, boundary enforcement
8Performance & ScalabilityConcurrent users, timeout handling, high-load scenarios

Categories that are not relevant for your system type (for example, a single-turn FAQ bot may have limited multi-turn requirements) are flagged as low-priority rather than critical gaps.


How to Read the Advisor Panel

The Coverage Advisor appears in the Scorecard tab of the Coverage Dashboard.

Status Indicators

  • Covered — Your test space has dimensions or tuples that adequately address this category
  • Partial — Some coverage exists but gaps remain (for example, only one persona covered out of three declared)
  • Missing — No dimensions or tuples address this category at all
  • Low Priority — Not relevant for your declared system type (based on your context template)

Gap Descriptions

Each gap includes:

  • What is missing — a plain-language description of the uncovered area
  • Why it matters — the risk if this category is not tested
  • Suggested action — a concrete next step (e.g. "Add a dimension for error scenarios")

How the Advisor Uses Your Context Template

The advisor reads these fields from your Context Template to personalise its recommendations:

  • ai_system_description — identifies the system type (FAQ bot, scheduling agent, etc.)
  • core_use_cases — determines which functional categories are required
  • target_users_personas — drives persona and context diversity requirements
  • failure_sensitivity — raises priority for error handling and edge case coverage
  • input_modalities — determines whether voice, image, or multi-modal testing is relevant
  • business_objectives — identifies compliance and performance requirements

If your Context Template is incomplete, the advisor will note which fields it needed but could not read.


Cache and Refresh

Advisor results are cached per project to avoid repeated LLM calls. The cache is invalidated automatically when:

  • Dimensions are added, modified, or deleted
  • Tuples are approved or rejected
  • The context template is updated
  • A dimension or tuple generation run is cancelled

To manually refresh: click Refresh Advisor in the Coverage Advisor panel.


Tips for Getting the Most from the Advisor

  1. Fill in your Context Template first — the more fields you complete, the more precise the advisor's recommendations
  2. Act on Missing categories before Partial ones — a completely uncovered category is a higher release risk than a partially covered one
  3. Use Suggested Actions — each gap links to the right page (Pipeline, Generate, Test Suites) to fix the issue
  4. Re-run after major changes — the cache auto-invalidates, but if you've made many changes, click Refresh to see updated recommendations immediately

Relationship to Quality Audit

The Coverage Advisor and the Quality Audit serve different purposes:

Coverage AdvisorQuality Audit
Asks"Are we testing the right things?""Are our tests good enough?"
ScopeEntire test space (dimensions + tuples)A specific test suite
MethodCategory taxonomy + context template analysis5-rubric scoring per test case
OutputGap list with action linksRubric scores + improvement suggestions

Run both to get a complete picture of your test programme's health.

Coverage strategies

Coverage strategies

Test case quality

Test case quality

On this page

What is the Coverage Advisor?The 8 Coverage CategoriesHow to Read the Advisor PanelStatus IndicatorsGap DescriptionsHow the Advisor Uses Your Context TemplateCache and RefreshTips for Getting the Most from the AdvisorRelationship to Quality Audit