Record the case before assigning a pass
| Record field | Filled example | Required for review |
|---|---|---|
| Case and input | E-04: “When does Saturday close?” | Identifies exactly what was asked |
| Source snapshot and expectation | S-2: Saturday 10:00–14:00; expected close 14:00 | Preserves the evidence available to the assistant |
| Actual output and decision | Illustrative output “14:00”; illustrative decision pass | Must be replaced by a real captured output before claiming a run |
The filled output is invented to demonstrate the record, not taken from a model. Add the real environment/version, run reference, reviewer and any action evidence when you execute the case. Keep confidential raw content in the approved working location rather than a public report.
Identify decisions that cannot be reproduced
| Missing item | What cannot be established | Next action |
|---|---|---|
| Only a pass label is saved | What was asked and observed | Capture the case input and output |
| Answer saved, source version absent | Whether the answer matched the supplied source at the time | Attach the source snapshot or mark review incomplete |
| Reply saved, write result absent | Whether an action happened once, twice or not at all | Inspect the action log and resulting record |
An incomplete evidence record is not automatically a model failure; it is a review limitation. Keep that status distinct from an observed fail, and do not include unexecuted examples in a measured pass rate.