Skip to content
Astera Technology

Export AI evaluation outcomes without flattening nested failures

An array of two failures becomes one ambiguous comma-containing label.

Download complete case notes and answer keys ↓

Define the local evaluation task

A synthetic evaluation export uses one row per case/reason-code association. A case with no failures receives one explicit pass row with an empty reason. Failure codes are distinct per case; repeats invalidate the result before export. The output is a row matrix suitable for CSV serialization, not an opaque joined reason string.

  1. Validate distinct reasons within a case
  2. Expand one case/reason row per failure
  3. Retain an explicit row for passed cases

Repeated reason codes require diagnostic correction before export.

Use the filled cases

Original synthetic input and literal answer key
CaseSupplied inputExpected structured outputDecision and reason
Two distinct failure reasons{"cases":[{"id":"C1","failures":["fieldA","fieldB"]}]}{"rows":[["C1","fail","fieldA"],["C1","fail","fieldB"]]}Both reasons retain the same case identity in separate export rows.
Passed case remains visible{"cases":[{"id":"C2","failures":[]}]}{"rows":[["C2","pass",""]]}A case without failures is retained rather than disappearing from the export.
Repeated reason code{"cases":[{"id":"C1","failures":["fieldA","fieldA"]}]}{"error":"duplicate-reason"}Repeated fieldA is a malformed diagnostic set, not two independent failures.
Download editable CSV ↓

Work the duplicate-reason-code case. Which output is required, and why?

Show the worked answer

{"error":"duplicate-reason"} — Repeated fieldA is a malformed diagnostic set, not two independent failures.

Run the self-contained evaluator

Save as .mjs and run with Node.js; supplied fixtures only
import assert from 'node:assert/strict';

function evaluate(x){
 if(x.cases.some(c=>new Set(c.failures).size!==c.failures.length))return {error:'duplicate-reason'};
 return {rows:x.cases.flatMap(c=>c.failures.length?c.failures.map(reason=>[c.id,'fail',reason]):[[c.id,'pass','']])};
}

const cases = [
  {
    "id": "two-independent-reasons",
    "input": {
      "cases": [
        {
          "id": "C1",
          "failures": [
            "fieldA",
            "fieldB"
          ]
        }
      ]
    },
    "expected": {
      "rows": [
        [
          "C1",
          "fail",
          "fieldA"
        ],
        [
          "C1",
          "fail",
          "fieldB"
        ]
      ]
    }
  },
  {
    "id": "pass-still-visible",
    "input": {
      "cases": [
        {
          "id": "C2",
          "failures": []
        }
      ]
    },
    "expected": {
      "rows": [
        [
          "C2",
          "pass",
          ""
        ]
      ]
    }
  },
  {
    "id": "duplicate-reason-code",
    "input": {
      "cases": [
        {
          "id": "C1",
          "failures": [
            "fieldA",
            "fieldA"
          ]
        }
      ]
    },
    "expected": {
      "error": "duplicate-reason"
    }
  }
];
for (const c of cases) assert.deepEqual(evaluate(c.input), c.expected, c.id);
console.log(JSON.stringify({checked:cases.length,caseIds:cases.map(c=>c.id)}));
Download runnable example ↓

Add proper quoting when serialising these rows to CSV. This row-grain example does not itself claim to escape arbitrary spreadsheet formulas or execute an export to another system.

Related resources