What do I say to the person this lands on

Thesis

Anyone who has shipped a system that makes decisions about people has had to answer one question before it went live: what do I say to the person this lands on? It isn’t on any checklist, and policy review doesn’t reproduce the reflex.

Reader

Assurance peers, CTOs choosing an advisor, and NfP leaders whose systems touch vulnerable people.

Opening

WorkCover NSW, 2000 to 2002: automated selection for audit, with impact assessment, explainability, a tiered response and investigators in the loop. Robodebt, fifteen years later, had none of these. The knowledge existed; it was left out.

Beats

1. Institutional failure, not technical failure.
2. The reflex: builders ask the question because they have had to answer it.
3. What it costs when it’s missing: assurance that checks controls are documented rather than that they work.
4. Monitoring, not approval, is where assurance succeeds or fails. Link to BigDocs piece 8 as the worked example (9.1, A.6.2.4).

Keep out (from the idea notes)

Not “hire practitioners”. Not “auditors don’t understand technology”. Not a told-you-so. Robodebt caused real harm; the point is institutional humility.

Decision

Whether to name the WorkCover work as your own.

What you can do on Monday

For each AI system that makes or shapes a decision about a person, write the explanation you would give them. If you can’t, that is your first finding.