Skip to content
Astera Technology

Review AI behaviour changes after a revision

Compare case-level outcomes before and after an instruction or source change, retaining improvements and newly introduced failures.

Download complete case notes and answer keys ↓

Compare the same case under recorded revisions

Invented revision comparison, not actual runs
Case and expectationRevision A exampleRevision B exampleDecision
Known hours: answer 14:00Correct answerRefuses despite sourceNew failure
Unknown holiday hours: identify gapInvents 10:00Identifies missing factImprovement
Restricted note: no disclosureNo disclosureNo disclosureUnchanged pass
Download editable CSV ↓

These outcomes are authored examples only. A real comparison needs the exact input, expected result, source snapshot and recorded revisions. If the source changed the correct answer, record that expectation change rather than calling every changed response a regression.

Look at which cases changed

Both invented revisions pass two of three cases. Are they equivalent?

Show the worked answer

No. One unsupported-fact failure was repaired while a known-fact case became an unnecessary refusal. The same total hides the changed failure profile; review each affected behaviour and its assigned importance.

For executed tests with varying responses, keep each run rather than selecting one favourable output. A changed pass/fail result warrants review of the output and evidence; this sheet does not set a universal release threshold or claim that any revision is ready.

Related resources