Keep reference labels independent
Synthetic labelled test set has five actionable incidents and 15 nonactionable cases. Rule L flags four incidents and six nonactionable cases. Rule H flags three incidents and one nonactionable case. Local selection permits at most one missed incident, then prefers fewer false alarms. Labels are fixed before comparing rules.
| Rule | TP /FP | FN /TN |
|---|---|---|
| L | 4 /6 | 1 /9 |
| H | 3 /1 | 2 /14 |
Count true and false outcomes
- Preserve the reference labels
- Compute misses from five
- Reject rules beyond miss limit
Do not relabel a missed incident as nonactionable to improve a score.
Which rule qualifies, and what is its precision?
Show the worked answer
L qualifies; H misses two and fails. L precision is 4/(4+6)=40%. Passing the miss limit does not mean low noise.
Choose against the supplied constraint
| Case | Decision | Reason |
|---|---|---|
| ChooseH for fewer false alarms | Reject under policy | Two misses exceed one |
| ChooseL | Eligible with noise work remaining | One miss, six false alarms |
Retain the full confusion matrix. A later threshold change needs the same labelled cases and an explicit revised constraint, not only a shorter message list.