Coverage advisor
Coverage advisor
This guide covers the Domain Coverage Advisor (Layer 2 taxonomy analysis — the card on the Coverage Dashboard). For the Ask Coverage conversational chat (the floating button, bottom-right), see the Coverage Guide.
What is the Coverage Advisor?
The Domain Coverage Advisor analyses your project's context template and tells you which AI-testing categories are covered by your current test space — and which are missing.
It applies an 8-category Conversational AI taxonomy (based on ISTQB, ISO 25010, and industry chatbot testing standards) to your test dimensions and tuples, giving you a domain-aware assessment rather than a generic checklist.
The 8 Coverage Categories
The advisor evaluates your test space against these categories:
| # | Category | What it checks |
|---|---|---|
| 1 | Core Functionality | Are the primary declared use cases covered? |
| 2 | Edge Cases & Boundary Conditions | Inputs at limits, unexpected input types, empty/null values |
| 3 | Error Handling & Failure Modes | Known failure scenarios declared in your context template |
| 4 | User Intent Variations | Multiple ways to express the same intent (paraphrasing, languages) |
| 5 | Persona & Context Diversity | Different user segments, experience levels, and roles |
| 6 | Multi-Turn & Conversation Flow | Conversation continuity, context retention, session management |
| 7 | Security & Compliance | PII handling, injection attempts, boundary enforcement |
| 8 | Performance & Scalability | Concurrent users, timeout handling, high-load scenarios |
Categories that are not relevant for your system type (for example, a single-turn FAQ bot may have limited multi-turn requirements) are flagged as low-priority rather than critical gaps.
How to Read the Advisor Panel
The Coverage Advisor appears in the Scorecard tab of the Coverage Dashboard.
Status Indicators
- Covered — Your test space has dimensions or tuples that adequately address this category
- Partial — Some coverage exists but gaps remain (for example, only one persona covered out of three declared)
- Missing — No dimensions or tuples address this category at all
- Low Priority — Not relevant for your declared system type (based on your context template)
Gap Descriptions
Each gap includes:
- What is missing — a plain-language description of the uncovered area
- Why it matters — the risk if this category is not tested
- Suggested action — a concrete next step (e.g. "Add a dimension for error scenarios")
How the Advisor Uses Your Context Template
The advisor reads these fields from your Context Template to personalise its recommendations:
ai_system_description— identifies the system type (FAQ bot, scheduling agent, etc.)core_use_cases— determines which functional categories are requiredtarget_users_personas— drives persona and context diversity requirementsfailure_sensitivity— raises priority for error handling and edge case coverageinput_modalities— determines whether voice, image, or multi-modal testing is relevantbusiness_objectives— identifies compliance and performance requirements
If your Context Template is incomplete, the advisor will note which fields it needed but could not read.
Cache and Refresh
Advisor results are cached per project to avoid repeated LLM calls. The cache is invalidated automatically when:
- Dimensions are added, modified, or deleted
- Tuples are approved or rejected
- The context template is updated
- A dimension or tuple generation run is cancelled
To manually refresh: click Refresh Advisor in the Coverage Advisor panel.
Tips for Getting the Most from the Advisor
- Fill in your Context Template first — the more fields you complete, the more precise the advisor's recommendations
- Act on Missing categories before Partial ones — a completely uncovered category is a higher release risk than a partially covered one
- Use Suggested Actions — each gap links to the right page (Pipeline, Generate, Test Suites) to fix the issue
- Re-run after major changes — the cache auto-invalidates, but if you've made many changes, click Refresh to see updated recommendations immediately
Relationship to Quality Audit
The Coverage Advisor and the Quality Audit serve different purposes:
| Coverage Advisor | Quality Audit | |
|---|---|---|
| Asks | "Are we testing the right things?" | "Are our tests good enough?" |
| Scope | Entire test space (dimensions + tuples) | A specific test suite |
| Method | Category taxonomy + context template analysis | 5-rubric scoring per test case |
| Output | Gap list with action links | Rubric scores + improvement suggestions |
Run both to get a complete picture of your test programme's health.