# Quality reports

**Summary:** Score the Customer Agent against a curated set of control questions. Track quality trends over time.

**In short:** Curated set of control questions and the answers you expect. Quality reports run the questions against the agent and score each answer.

**Last updated:** 2026-05-27

## Key concepts

### Control questions
Questions you would expect any customer to ask, with the answer you would expect the agent to give. Build the list once, run it many times.

### Control answers
The expected answers. Used as the benchmark when scoring.

### Report runs
Each time you generate a report, the agent runs every control question. Scores compare each answer to the control answer.

### Report score
Average of all question scores. Useful as a quick health indicator over time.

### Per-question score
The signal you actually act on. Low-scoring questions point straight at the Source that needs work.

## What you can do here

- Add control questions, manually or pulled from top questions in Analyze
- Set the expected answer for each question
- Run a report and view per-question scores
- Compare reports over time to track quality trends
- Inspect which Sources the agent used for a low-scoring answer

## When to use it

- After any change to Living Knowledge, to confirm scores held or improved
- On a weekly cadence, as a regular health check
- Before showing the agent's results to leadership or auditors

## How it works

Each control question runs against the live agent. Scores compare the generated answer to the control answer using semantic similarity, not exact match. Small differences in wording get high scores. Real disagreements get low scores.

## Frequently asked questions

**How do I add a control question?**
Open \

**How do I pull questions from real customer activity?**
Open \

**How do I run a report?**
Open \

**What does a low score mean?**
The agent's answer disagreed substantially with the control answer. Open the question to see the agent's answer, the Sources it used, and the control answer side by side.

**How do I improve a low score?**
Find the Source the agent used. Either fix that Source, add a new Source that has the right information, or update the control answer if the agent was actually correct.

**How often should I run reports?**
Once a week as a baseline. Plus on demand after any major change.