測試擾動只可改指定因素

訂明本例驗收任務

假設穩健性測試對明訂哪些意思欄可改。核基線與變體全部鍵；未容許加或刪欄也算變。容許及意外變動分開列。依據保留也一起測：只改問法時，所需依據編號不能變。

列容許改動欄
核全部自身鍵及本例規範值
模型測試前列意外變動

意外改動令測試對無效，不當候選模型失敗。

使用已填個案

原創假設輸入及確切答案鍵
個案	已提供輸入	預期結構化結果	判斷及原因
問法改了，資料保留	{"base":{"wording":"Read code K1","code":"K1","quantity":3},"variant":{"wording":"Read code K1","code":"K1","quantity":3},"allowed":["wording"]}	{"allowedChanges":["wording"],"unexpected":[],"valid":true}	只改已訂問法因素。
同時改了數量	{"base":{"wording":"Read K1","code":"K1","quantity":3},"variant":{"wording":"Read K1","code":"K1","quantity":4},"allowed":["wording"]}	{"allowedChanges":["wording"],"unexpected":["quantity"],"valid":false}	數量變會改預期業務答案，此對已不受控。
刪掉所需依據	{"base":{"wording":"Read K1","evidence":["D1"]},"variant":{"wording":"Please read K1","evidence":[]},"allowed":["wording"]}	{"allowedChanges":["wording"],"unexpected":["evidence"],"valid":false}	測問法卻刪 D1，也改了來源可用性，不能當同條件配對。

下載可編輯 CSV ↓

逐步處理 required-evidence-removed 個案，須得到甚麼結果？原因是甚麼？

查看參考答案

{"allowedChanges":["wording"],"unexpected":["evidence"],"valid":false}。測問法卻刪 D1，也改了來源可用性，不能當同條件配對。

執行完整本機驗算

另存為 .mjs，以 Node.js 執行；只驗提供的個案

import assert from 'node:assert/strict';

function evaluate(x){
 const keys=[...new Set([...Object.keys(x.base),...Object.keys(x.variant)])].sort();const changed=keys.filter(k=>Object.hasOwn(x.base,k)!==Object.hasOwn(x.variant,k)||JSON.stringify(x.base[k])!==JSON.stringify(x.variant[k]));
 return {allowedChanges:changed.filter(k=>x.allowed.includes(k)),unexpected:changed.filter(k=>!x.allowed.includes(k)),valid:changed.every(k=>x.allowed.includes(k))};
}

const cases = [
  {
    "id": "spacing-only",
    "input": {
      "base": {
        "wording": "Read code K1",
        "code": "K1",
        "quantity": 3
      },
      "variant": {
        "wording": "Read  code K1",
        "code": "K1",
        "quantity": 3
      },
      "allowed": [
        "wording"
      ]
    },
    "expected": {
      "allowedChanges": [
        "wording"
      ],
      "unexpected": [],
      "valid": true
    }
  },
  {
    "id": "quantity-confound",
    "input": {
      "base": {
        "wording": "Read K1",
        "code": "K1",
        "quantity": 3
      },
      "variant": {
        "wording": "Read  K1",
        "code": "K1",
        "quantity": 4
      },
      "allowed": [
        "wording"
      ]
    },
    "expected": {
      "allowedChanges": [
        "wording"
      ],
      "unexpected": [
        "quantity"
      ],
      "valid": false
    }
  },
  {
    "id": "required-evidence-removed",
    "input": {
      "base": {
        "wording": "Read K1",
        "evidence": [
          "D1"
        ]
      },
      "variant": {
        "wording": "Please read K1",
        "evidence": []
      },
      "allowed": [
        "wording"
      ]
    },
    "expected": {
      "allowedChanges": [
        "wording"
      ],
      "unexpected": [
        "evidence"
      ],
      "valid": false
    }
  }
];
for (const c of cases) assert.deepEqual(evaluate(c.input), c.expected, c.id);
console.log(JSON.stringify({checked:cases.length,caseIds:cases.map(c=>c.id)}));

下載可執行例子 ↓

本例 JSON 值已按固定鍵及陣列次序提供。若容許等意但寫法不同，另訂比較器；只測問法時，依據可用性須保持。

訂明本例驗收任務

使用已填個案

執行完整本機驗算

相關資源