Define the local evaluation task
A synthetic candidate ingestion ledger accepts nonblank text case IDs compared exactly, with each occurrence retained until validation. Hold missing/nontext IDs; empty batches remain valid at this ingestion gate. Count all occurrences before building a map. Any duplicated ID rejects the batch even when its later answer is correct; a first-wins policy would be equally unapproved. Return duplicate IDs with their counts.
- Keep each candidate occurrence
- Count occurrences before map construction
- Reject repeated keys with explicit counts
Repeated keys require a supplied attempt identity or a repaired file.
Use the filled cases
| Case | Supplied input | Expected structured output | Decision and reason |
|---|---|---|---|
| Later correct answer would overwrite | {"rows":[{"id":"C1","answer":"wrong"},{"id":"C1","answer":"correct"}]} | {"duplicates":[{"id":"C1","count":2}],"accepted":false} | Both attempts remain evidence; neither replaces the other before grading. |
| Two unique cases | {"rows":[{"id":"C1","answer":"wrong"},{"id":"C2","answer":"correct"}]} | {"duplicates":[],"accepted":true} | Unique-key ingestion passes; the wrong C1 answer still needs content grading. |
| Candidate has no usable case key | {"rows":[{}]} | {"error":"case-identity"} | An absent key cannot be accepted as one distinct candidate identity. |
| Two duplicated case IDs | {"rows":[{"id":"B"},{"id":"A"},{"id":"B"},{"id":"A"},{"id":"A"}]} | {"duplicates":[{"id":"A","count":3},{"id":"B","count":2}],"accepted":false} | Counts three and two are retained independently rather than reduced to one duplicate total. |
Work the multiple-duplicates case. Which output is required, and why?
Show the worked answer
{"duplicates":[{"id":"A","count":3},{"id":"B","count":2}],"accepted":false} — Counts three and two are retained independently rather than reduced to one duplicate total.
Run the self-contained evaluator
import assert from 'node:assert/strict';
function evaluate(x){
if(x.rows.some(row=>typeof row.id!=='string'||!row.id.trim()))return {error:'case-identity'};
const counts=new Map();for(const row of x.rows)counts.set(row.id,(counts.get(row.id)||0)+1);
const duplicates=[...counts].filter(([,n])=>n>1).sort(([a],[b])=>a<b?-1:a>b?1:0).map(([id,count])=>({id,count}));return {duplicates,accepted:duplicates.length===0};
}
const cases = [
{
"id": "late-correct-overwrite",
"input": {
"rows": [
{
"id": "C1",
"answer": "wrong"
},
{
"id": "C1",
"answer": "correct"
}
]
},
"expected": {
"duplicates": [
{
"id": "C1",
"count": 2
}
],
"accepted": false
}
},
{
"id": "two-unique-cases",
"input": {
"rows": [
{
"id": "C1",
"answer": "wrong"
},
{
"id": "C2",
"answer": "correct"
}
]
},
"expected": {
"duplicates": [],
"accepted": true
}
},
{
"id": "case-key-absent",
"input": {
"rows": [
{}
]
},
"expected": {
"error": "case-identity"
}
},
{
"id": "multiple-duplicates",
"input": {
"rows": [
{
"id": "B"
},
{
"id": "A"
},
{
"id": "B"
},
{
"id": "A"
},
{
"id": "A"
}
]
},
"expected": {
"duplicates": [
{
"id": "A",
"count": 3
},
{
"id": "B",
"count": 2
}
],
"accepted": false
}
}
];
for (const c of cases) assert.deepEqual(evaluate(c.input), c.expected, c.id);
console.log(JSON.stringify({checked:cases.length,caseIds:cases.map(c=>c.id)}));
Save the untouched occurrence ledger for staff review. This check concerns ingestion identity only; it never selects a favourable candidate or reports actual model accuracy.