Build the tests in: borrowing confidence from engineering

Software teams stopped trusting “it looks right” a long time ago. Code that matters ships with tests: small checks that run every time something changes and say plainly whether it still works. Nobody thinks this is a lack of faith in the programmer. It is how you get confidence in something too big to read every time.

Big-document work with AI deserves the same habit, and it is rarely given it. Most self-built tools are judged by someone reading an output and feeling reassured. That tells you about that run, on that day, and nothing about the next one.

Tests inside the work

The first set of tests live inside the process and run every time:

• Completeness. Every part of the source is captured, or excluded with a reason. Structure first
• Nothing lost in the edits. Rejecting every change gives back the original, word for word. Text outside the edits is unchanged. Never retype
• Every pointer resolves. Every position cited exists in the Wordhouse.
• Every clause has a decision. No silent gaps in the output. Output built for oversight

When a check fails, the work loops: the failing piece is rerun or flagged until the checks pass. That is where agents are genuinely good. Let it loop until you’ve got it, but let the tests, not the model, decide when that is.

Tests on the system

The second set of tests measure the system as a whole, and these are what the benchmark was built from. BigDocs: the benchmark

• A test set with known answers. Fifteen departures were planted in each test case, graded from direct conflicts with our position, through tension and ambiguity, to missing clauses and compliant clauses that should be left alone. A known answer turns “seems good” into a score.
• Repeat runs. Every case was run more than once, because a system that is right once and wrong the next time is not right. This is where the single prompt’s lottery showed up: 7 of 15 classifications stable across runs, against 14 of 15.
• Honest marking. The answer key was mine, and it turned out to be too strict in places. Corrections were only allowed when backed by text in the library, applied to both approaches, and checked by an independent reviewer who hadn’t seen the scores. A benchmark that only confirms what you hoped is not worth much.

From test set to monitoring

The test set doesn’t retire once the results are published. It becomes the regression suite: rerun on every model change, every library update and every change to the skill, with coverage, consistency and accuracy tracked over time. Models change underneath you without notice. This is how you find out before your customers do.

That is also the ongoing monitoring an AI management system needs. ISO 42001 asks for performance to be monitored and evaluated (Clause 9.1) and for AI systems to be verified and validated (Annex A.6.2.4). Australia’s Guidance for AI Adoption asks organisations to test and monitor, and to maintain human control. A regression suite with a history is evidence of all of that, produced as a side effect of doing the work properly.

For the CTO with self-serving teams

If your teams are building their own skills and agents, you don’t need to review every one. You need to know which ones exist, which ones matter, and that each of those has a small test set someone reruns when things change. That is a modest ask of a team, and it turns a pile of clever tools into something you can stand behind.

What your team can do on Monday

Take one document job your team runs with AI. Write down five cases where you already know the right answer. Run the tool on them three times each. If the answers change between runs, you have learned more than any demo will tell you, and you have the start of a test set.