Beyond uptime: measure whether customers can place an order

Start with the order the customer expects

Imagine an established distributor whose retailers order replenishment stock through an app before the daily dispatch cut-off. The server is reachable, yet a retailer sees a spinning button, calls sales and submits again. Another order reaches the warehouse with cartons converted into individual units. These are fictional examples, but they expose the decision: the support dashboard should show whether the ordering service delivers its business outcome. Keep server uptime for diagnosis; add measures that explain the customer’s unfinished work.

Write the boundary before choosing a percentage. For this proposed scorecard, an eligible journey starts when a signed-in customer submits a basket that passes the agreed business validation. Success means exactly one durable order exists and the app displays its matching order reference within a locally chosen 30-second observation limit. The receipt confirms recording, not stock allocation or delivery. Repeated taps and network retries share one journey identifier. A later successful recovery remains useful information but does not erase the original failed journey.

Use three measures with named owners

The following is a proposed starting design, not measured performance or a provider guarantee. Use a rolling 28-day window, refreshed each day, with all hours included and timestamps grouped in Hong Kong time. Also inspect the firm’s actual pre-cut-off peak separately so quiet hours cannot conceal a difficult ordering period. The operations manager approves the definitions; replace each role below with a named person and a deputy. Display the numerator, denominator, collection coverage and last refresh beside every percentage.

All thresholds, including 3 and 30 seconds, are local hypotheses for discussion. Correctness is grouped by order creation date; recent orders not yet due are shown as pending, and overdue unmatched orders do not count as correct.

For correctness, agree the actual fields: customer account, product identifier, quantity, unit of measure, agreed price and requested delivery date. Compare the accepted snapshot with the first warehouse record before staff corrections; otherwise manual repair makes the app look more reliable than it was. Exclude later customer amendments from this comparison and track them separately. If full reconciliation is unavailable, report a documented sample as a sample, with its size and selection method. Do not label it as whole-service correctness.

Make missing evidence visible

Join app submission events, displayed receipts and order records using the journey identifier. If the app cannot send an event during a connection failure, store it for later delivery where feasible; until reconciled, show the measurement gap. A missing receipt event is not proof of success. Document exclusions such as test accounts and baskets rejected for a genuine validation reason, and count those exclusions separately. An unavailable validation service is a service failure, not an invalid basket. Keep planned maintenance visible in the all-hours result.

Consider a fictional 28-day period with 1,000 eligible journeys, complete collection and 990 successful submissions. Submission success is 990 ÷ 1,000 × 100 = 99%, below the trial 99.5% target. If only 940 journeys display a valid receipt within three seconds, receipt speed is 94%, even if the successful requests alone look fast. These illustrative counts explain the calculation; they are not results from a Hong Kong business. With small volumes, show individual failures as well as rates. Never display a zero-denominator period as 100% success.

Choose a target before making a commitment

Google’s Service Level Objectives chapter distinguishes an indicator, which measures behaviour, from an objective, which sets a target, and an agreement, which includes consequences. It also recommends selecting a small set of measures relevant to users. Apply that distinction here: the trial percentages guide internal improvement priorities. A customer agreement needs its own agreed measurement scope, reporting and consequences. Do not copy an internal dashboard threshold into a sales promise or assume a hosting provider’s uptime commitment covers the complete ordering journey.

There are practical alternatives. Infrastructure-only monitoring is cheaper to start but leaves order completion unmeasured. Support tickets reveal context but miss customers who abandon silently. Scheduled test orders can expose breakage when traffic is low, but must be labelled separately from real customer outcomes. Start with existing event exports and a shared scorecard if buying a monitoring platform would consume the improvement budget. Collect a baseline covering busy and quiet trading periods, then adjust the trial targets against customer tolerance, correction workload and the team’s funded support hours.

Turn the scorecard into next week’s decisions

In the next working session, have the order operations lead, support lead and app owner trace one ordinary order, one timeout and one unit mismatch. Agree the event boundary and identify missing evidence before setting the dashboard live. Review the scorecard weekly; route current failed orders to the staffed support channel each day rather than waiting for a 28-day target to fail. Assign a specific action and verification date for each recurring problem. A confirmed wrong quantity deserves attention even when the overall correctness percentage is above target.