Test an AI translator before using it for product copy

Decide what passing will allow

Suppose your importing business has chosen a translation workflow for supplier descriptions and Hong Kong product listings. The decision now is whether staff may use its drafts, for which product families, and with whose approval. Start with one ordinary product family and both English-to-Traditional-Chinese and Traditional-Chinese-to-English work if staff need both. Passing one direction does not establish quality in the other. Keep publication outside the pilot until a named bilingual reviewer has approved the output.

A modest budget still offers alternatives: reuse approved bilingual phrases for repeated specifications, translate manually where volume is low, or use AI drafts with human review where wording varies. Compare the complete effort, including checking and corrections. If nobody can assess both languages and the product facts, arrange a qualified reviewer or keep the existing manual route. A cheap subscription cannot replace that missing capability.

Build a test pack from product facts

Choose approved supplier material that staff are allowed to use in the selected tool. Remove customer details and unnecessary commercial information. Give each case an identifier, source revision, language direction, source text, required facts, allowed terminology and forbidden additions. The reference answer should describe what must survive translation, while allowing natural alternative wording. Set a Hong Kong Traditional Chinese glossary for recurring terms, keep model codes unchanged, and state whether units may be converted. Where conversion is unnecessary, require the original quantity and unit.

Include routine descriptions, short spreadsheet cells, longer paragraphs and missing or conflicting facts. Run them through the same copy-and-paste or batch route staff will actually use. The fictional cases below are a starter pack, not measured model accuracy or a complete catalogue of risks. Add cases from your own product family, including a Chinese source written independently. Reversing a previous translation alone is not a separate test of real Chinese inputs.

Use both directions where relevant. Numbers are fictional test inputs, not performance results.

Grade errors by their consequence

Have the bilingual reviewer compare each raw output directly with the approved source and product facts. Record the incorrect span, its correction and its consequence before polishing style. A second translation generated by the same workflow is not independent evidence. If the reviewer disputes a specification, ask the product owner to resolve it against supplier material; do not ask the translator to guess. Retain the raw output so a corrected final version cannot hide an initial failure.

• Critical: a changed safety restriction, quantity, unit or unsupported product claim. Stop the affected use and investigate.
• Material: wrong model identity, missing accessory exclusion or ambiguous specification. Reject the draft and resolve before release.
• Editorial: awkward phrasing with facts intact. Correct it and track whether the review effort remains worthwhile.

Agree a gate before running the pack. An illustrative rule is no critical or material errors in any retained acceptance run, and every deliberately unresolved input routed to a person. Repeat sensitive cases to check consistency and retain every attempt, rather than selecting the best answer. Record failures by severity and language direction with the number of cases tested. A good average must not cancel a changed warning. Passing this limited pack supports only the tested scope, not an accuracy claim for all products.

Set approval routes and a retest trigger

Keep new descriptions, changed specifications, warnings and claims under bilingual approval before publication. Previously approved fixed phrases can follow a simpler route only when their wording and underlying facts are unchanged. For the pilot, measure review time as well as errors and agree a workload ceiling with the reviewer. If checking consumes the expected saving, narrow the scope or return to approved templates and manual translation. If a serious error escapes, pause the affected workflow, identify related drafts and published copy, and have the owner arrange corrections.

Save the tool and available model identifier, settings, prompt, glossary revision, source revision, date, outputs and reviewer decisions with the pack. Rerun it when the model, prompt, glossary, input format or product family changes. NIST AI RMF 1.0 supports documented test sets, evaluation in conditions resembling deployment and defined human oversight. It is voluntary guidance; the source page notes that revision is in progress. The product examples and release gates here are editorial proposals, not NIST certification requirements.

Your next action is to book a working session with the product owner and bilingual reviewer. Bring one approved supplier sheet, fill in the case records above and agree the error gate before anyone runs the translator. End with a written decision: reject the pilot, revise and retest, or permit reviewed drafts for the named product family. For the wider design of data access and evaluation ownership, the existing AI application planning guide provides useful context.