Which AI-classified maintenance requests need a human dispatcher?

Give the classifier a limited routing permission

The decision is whether a classification may move a request into a team's work queue without a dispatcher reading it first. Keep that permission separate from sending a technician, promising an arrival time or closing a job. For this pilot, allow automatic routing only for approved routine categories with sufficient location and task information. Ambiguous messages and anything potentially urgent must reach a person, even when the model reports high confidence.

Compare the consequences before selecting a score cutoff. Sending a cupboard-hinge repair to the wrong trade creates reassignment work. Treating a report of water near electrical equipment as an ordinary plumbing request may delay an urgent assessment. Sending a harmless request for prompt human review consumes staff time, but the costs are not symmetric. Have the operations owner document which mistakes block automatic routing and which can be corrected through normal reassignment.

These examples define routing behaviour, not a complete hazard-detection checklist or technical repair advice.

Label the decision in both languages

Build a labelled sample from authorised historical requests, removing unnecessary personal details. Include English, Traditional Chinese and mixed messages, with local shorthand, incomplete locations and several faults in one request. Preserve enough surrounding conversation to judge the message correctly. Label the expected queue, trade if known, urgency-review requirement and missing information. Ask an experienced dispatcher to resolve disputed labels; if staff cannot agree, refine the category definition instead of asking the model to settle the disagreement.

Keep a development set for adjusting rules and thresholds, then reserve unseen messages for the final check. Separate near-duplicate reports about the same incident so they do not appear on both sides. Add deliberately difficult examples such as an urgent clue buried after a routine repair request, but report these challenge cases separately from the representative sample. A deliberately enriched urgent set can expose failures; it cannot establish how often urgent requests occur in daily work.

Test the whole routing rule, not just a confidence score

Microsoft's classification guidance explains that confidence thresholds need evaluation on representative production data, and that category coverage and language can affect performance. A confidence score is not a guarantee of correctness. For this pilot, compare three alternatives on the same labelled messages: dispatcher review of every request; simple rules for a tightly defined routine category; and AI routing with human-review gates. Record staff handling time as well as errors, so extra exception work is visible.

Try several candidate cutoffs for each routine category on the development set. Apply the same overrides every time: possible urgency, missing essential information, conflicting requests and unsupported categories go to human review. If the system only generates a self-reported confidence number, do not assume it behaves like a validated classifier score. Keep it in suggestion mode until its relationship to routing errors has been tested. Freeze the proposed rule before evaluating the reserved set.

• Wrong automatic routes: incorrect team assignments divided by all automatically routed requests, with raw counts by category and language.
• Missed urgency: labelled potentially urgent requests that failed to reach prompt human review, divided by all labelled potentially urgent requests.
• Review workload: number awaiting review, oldest waiting request and observed handling time; report untested groups as untested.

Inspect a confusion table of expected queue against actual destination, especially urgent requests sent into routine queues. Review the original text behind every missed urgent case. A higher routine threshold cannot fix an urgency rule that misses the relevant wording. If one language or category has too little evidence, retain human review for that segment. No observed misses in a small test is useful evidence to inspect, not proof that future requests will be safe.

Make human review an operating commitment

Name the duty dispatcher and backup for each operating period. Agree how quickly potentially urgent and ambiguous requests must be acknowledged under the company's existing procedures, who covers out-of-hours arrivals and what happens when the queue exceeds capacity. Show the reviewer the original message, proposed route, reason for review and receipt time. Model failure or missing scores should fall back to the owned manual queue. Confirm that a handoff was received rather than treating a successful software submission as a completed dispatch.

Start in shadow mode: the dispatcher keeps routing real requests while the proposed rule records what it would have done. Before enabling any automatic route, review missed urgency, wrong-team assignments and actual queue delays with the operations owner. Continue sampling automatically routed routine work after launch. Pause the affected route after a missed urgent case or an unmanageable backlog, investigate and retest. The immediate next action is a short working session with the dispatcher to agree queue definitions, ownership and the labelled sample before anyone chooses a cutoff.