BigDocs: rolling the dice on big documents

A lot of business still runs on big documents. A customer contract checked against your own terms and policy. An employment agreement checked against the award. A tender or security questionnaire answered from everything you have said before. A knowledge base kept in step with the product it describes. Each one is long, often several documents at once, and each needs an exact answer rather than a good impression.

More and more of this work now sits with the teams who own it. Legal and commercial, HR, sales and marketing, support and operations are being encouraged to self-serve with tools like Claude, and they can do a great deal. What they don’t have is an engineering pipeline sitting behind them. When the results are unreliable, the problem finds its way back to the technology team, who end up designing around each team’s document job one at a time.

The answer usually looks right. Then someone finds the clause that was never read, the condition that was quietly paraphrased, or the decision nobody can explain, and the whole job gets done again by hand.

Why one big prompt is a roll of the dice

The way most people approach this is reasonable: put all the documents into one prompt and ask for a complete review. Context windows are large enough to take it all in. But a single prompt holding our paper, their paper and our policy asks the model to hold every clause against every other clause at once. Everything is in competition with everything else.

It reminds me of the Stroop effect, where you are asked to name the colour of the ink and the word itself spells a different colour. The information you need is right there, and the information you don’t need keeps getting in the way. People get slower and make mistakes. Models aren’t people, but the remedy looks similar: compartmentalise. Give each task only what it needs, and nothing that competes with it.

Over the years I have adopted a handful of patterns, mostly borrowed from data architecture and engineering, that do exactly that. The next piece benchmarks them on a hard case, contract review, against the one big prompt.

Why I’m writing this

I’ve spent a long time on both sides of this, in the boardroom signing off on the results and in the codebase building what produces them, across legal, commercial, sales, HR and operational work. The patterns here came from having to make big-document work hold up in each of those places, usually without an engineering team on call.