19. Metrics That Actually Matter

Measure prevented harm and legitimate-user burden across the whole decision system.

Reader outcome

Build a scorecard spanning harm, errors, friction, latency, recurrence, support, infrastructure, economics, and adaptation.

Candidate measures

Prevented harm, false-positive cost, appeal overturn, attacker replacement cost, legitimate-user friction, time to detection, time to intervention, recurrence, displacement, support burden, and defense cost.

Decision questions

• What counterfactual establishes “prevented” harm?
• Which metric becomes a target attackers or teams can game?
• How are uncertainty and delayed outcomes represented?
• Can aggregate success hide concentrated harm to a user group?

Evidence needed

• Causal evaluation and experimentation methods
• Metric gaming and feedback-loop cases
• Trust-and-safety or fraud scorecards
• Longitudinal displacement measurement