Every system is wrong sometimes. The only question that matters is whether you can find where.
This series has argued one idea from seven directions. Here it is in a single sentence:
Use a model where the work needs judgement or language. Use ordinary code everywhere else.
That sounds modest. In practice it goes against how most enterprise AI is being built right now, where the default is to give the model the whole problem (read the question, plan the query, write the SQL, fetch the context, check its own work, write the answer) and hope the loop converges. It often does, in a demo. In production it is slow, expensive, hard to audit, and wrong in ways nobody can locate.
We built Equative Insights OS the other way round. This post explains why, and links the series together so it can be read from one page.
One question, asked of every stage
When we designed the pipeline, we put the same test to every step:
Does this step need judgement, or does it need to give the same result every time?
If it needs judgement, a model does it. If it needs to repeat exactly, code does it. Almost nothing needs both.
| Stage | The Job | Who Does It |
|---|---|---|
| Intent | Work out what the person meant: entity, period, comparison | Model |
| Resolution | Map that to a defined business measure | Semantic layer (code) |
| Entitlement | Limit the data to what this person is cleared to see | Enforced at read (code) |
| Retrieval | Find the relevant definitions and documents | Hybrid search: keyword and vector ranking |
| Computation | Produce the figure | Deterministic statement against defined measures |
| Narration | Explain what the number means | Model |
Two jobs out of six are the model’s: reading what was meant, and writing about what was found. Those are things models do better than any rule we could write. The middle of the pipeline is where a confident guess does the most damage, so that is where we don’t allow one.
Why this is the sustainable choice
“Sustainable AI” usually gets discussed as an energy problem. It is one, but the architecture decision comes first, and it affects four things at once.
- Fewer calls. An agentic loop that plans, writes a query, inspects the result, retries and then summarises can easily take six to ten model calls for one question. A pipeline that only uses the model at either end needs a small, fixed number. (Illustrative figures. Your loop will vary, which is the point.)
- Less compute, lower cost per answer. Fewer calls means fewer tokens, and fewer tokens is the most direct lever on both carbon and cost. It also makes cost predictable, which, as Post 07 argued, is what lets a business case survive a rollout to a thousand seats.
- Fewer failure modes. Every model call is a place where a plausible wrong output can enter. Code fails too, but it fails loudly, the same way each time, and once it’s fixed it stays fixed. If you have six places a guess can enter, you have six places to debug. If you have two, you have two.
- Wrong answers you can find. This one matters most. When a number is wrong in a model-everywhere system, the honest diagnosis is often “the model did something.” In ours, the trace takes you to a stage: intent was read as the wrong quarter; the measure resolved to gross instead of net; a period-end balance was summed across months. Each of those is a fix with an owner. “The model did something” is not.
This is not an anti-AI argument
It would be easy to read this as caution dressed up as engineering, so to be clear: the position is pro-model. Reading an ambiguous question from a relationship manager and working out which client, which period and which comparison they meant is genuinely hard, and models are remarkably good at it. So is turning a table of computed figures into three sentences a CFO will actually read.
What we refuse to do is spend that capability on jobs where being creative is a defect. Arithmetic shouldn’t be creative. Access control shouldn’t be creative. “Year-to-date” should mean the same thing every time anyone asks.
Put the model where its strengths are, and keep it away from everything else.
The series on one page
If you’re arriving here first, this is the series in order. The odd-numbered posts named a way enterprise AI goes wrong; the even-numbered posts showed what we built instead.
- 01 · Your dashboard can tell you what happened. It still can’t tell you why. Traditional BI describes the past accurately and then stops. Root cause, meaning and next action are still manual work, repeated every reporting cycle. The case for decision intelligence as its own category.
- 02 · Show your working: why every AI answer needs a visible trace The panel under the answer: intent, resolution, retrieval, computation, narration, each stage stating what it decided and why. A pilot becomes a rollout when someone is willing to sign off a number they didn’t compute.
- 03 · The pitfall nobody demos: when the model writes the SQL Text-to-SQL rarely fails with a broken query. It fails with a plausible one that joins at the wrong grain and returns a number that looks right. Ask twice and you get two statements and two figures.
- 04 · YTD is not the sum of MTD Flows add across months; balances, exposures and year-to-date figures don’t. The most common source of wrong management numbers, and why the rule belongs in a semantic layer rather than in whoever writes the next query.
- 05 · Your retrieval pipeline can’t spell RoRWA Pure vector search finds things that mean the same. Specialists type things that are the same: codes, ratios, acronyms. Hybrid retrieval combines keyword exactness with semantic recall in a single ranking.
- 06 · Permissions are not a feature you add later Filtering an answer after retrieval is not a control. Entitlement enforced where data is read means the right question for a reviewer isn’t “did it refuse?” but “could it have retrieved it at all?”
- 07 · Nobody can tell you what one answer costs Pilots rarely die on accuracy; they die on unit economics nobody measured. Cost read back from every call, attributed to the stage that spent it, with a model you can fill in for your own seat count.
- 08 · AI where it counts, deterministic everywhere else You’re reading it.
Read together, the seven posts describe one system from the outside. A trace exists because the stages are separate. The SQL is safe because the model doesn’t write it. YTD comes out right because a semantic layer owns the rule. Retrieval finds RoRWA because keyword search is allowed to do its job. Permissions hold because they’re enforced before any model sees the data. Cost is knowable because there are only a few calls to count.
None of these is a feature you can add at the end. They all follow from one early decision about what the model is for.
The standard we’d ask you to hold any vendor to
Whoever you evaluate, us included, ask four questions in the demo:
- Run the same question twice. Do you get the same number?
- Show me the statement that computed this figure. Who wrote it?
- Could this answer have drawn on data I’m not cleared to see?
- What did this answer cost, measured and not estimated?
A system built on the principle in this post answers all four in the time it takes to open a panel. A system that doesn’t will usually answer with a roadmap.
