Repeated observation reporting
Separate stable cases, repeated-attempt weighting, early-stop coverage and completion-only timing.
Report case-level consistency across repeated attempts
A global mean hides one unstable case.
Separate case-weighted and attempt-weighted AI scores
Repeated retries on one easy case inflate the apparent evaluation quality.
Keep unattempted cases out of an all-cases success claim
Evaluation stops on a failure and mislabels remaining cases failed or passed.
Keep timed-out AI attempts visible in timing reports
Fast successful answers alone produce a misleading latency average.