When a Figure Is Questioned, "the System Shows It" Is Often the Only Answer. Here Is What the Public Record Shows.
We went through the public record on system figures that were relied on without anyone being able to check them. Here is what we found, four ways to answer a disputed figure, and the benchmark we would set.
An officer is going through a representation. It disputes the arrears shown in one account.
The section’s note is one line long: amount as per system. The officer asks how the system arrived at the amount. The section does not know. The IT cell says it will have to look into the code.
The amount stands, because the system shows it.
The paradox: The system’s figure is treated as settled, and the only reason is that nobody in the office is able to check it.
What We Found
For this article we went through the public record on systems whose figures were relied on: reporting in 2025 on a large UK public body’s system, reporting in 2026 on a large UK council’s finance system, and the US national auditor’s July 2025 report on the most critical federal legacy systems.
First, what we did not find: no Indian public body, and no body of our readers’ size, on the record with this problem, and nothing from the Middle East. We found no figure for how often an old system’s calculation is wrong, and we did not research any law, rule of evidence or service rule. Every body described below is far larger than those we write for.
A system’s output can be relied on over people. A large UK public body ran a supplier-built system, more than 25 years old, which, as the trade press reported in 2025, performed poorly while its figures were relied on. We add nothing to that account. We do not name the body or its supplier, or say what the fault was, and we pass no judgement on the body, its officials, its supplier or any court. It is a national body’s system, far outside our kind of work. The point we take is narrow: a system’s figures were relied on, and the system was wrong.
A function that produces figures can fail while the rest of the system carries on. A large UK council replaced its finance system with a packaged product. Custom changes included a reconciliation function that failed, and the council could not produce auditable accounts. This was a packaged finance product, which is not our kind of work. The point we take: when one function fails, an organisation may be unable to show that its figures are right.
The systems that apply public rules stay in service for decades. The US national auditor reported in July 2025 that the 11 most critical federal legacy systems are “about 23 to 60 years old”. Some use languages and machines that are not our kind of work. The point we take is about time: such a system has had its rules amended, one change at a time, for decades.
Our reading: a figure from an old system is believed for reasons that have little to do with being right. It is consistent, and consistency looks like accuracy. The rules are in the code, where the officers who stand behind the figure cannot read them. The people who wrote them have moved on. And doubt has no procedure, so looking at the same screen twice is taken as a check.
What our review does not show: The record we used on the national public body is one sentence. It does not say what the fault was, how it arose or how many people were affected, and we say none of these. The council case concerns a packaged product, and we did not examine why the function failed beyond what the source gives. The US figure describes federal systems far larger than any body we write for. We found nothing from India or the Middle East, and no figure for how often a system’s figures are wrong; we do not suggest it is common. The four reasons a figure is believed are our reasoning, not a finding of any source. We make no comment on any body, official, supplier or court.
What Waits While Nobody Can Show the Rule
A department that answers “the system shows it” is not careless; it answers with what it has. Our reading is that several things it needs to do cannot happen until it can show the rule behind a figure.
A disputed figure cannot be answered on its merits. The officer can confirm that the system produced the figure, not that its rule is the one the department intended.
A fault that touches many cases looks like many separate complaints. Each dispute is handled alone. Nobody sees that ten of them come from the same rule, because nobody can name the rule.
Audit and vigilance have nothing to check against. The only answer to how a reported figure was worked out is the report itself.
The gap we see: most bodies we write for could say what their systems produce. Few could say, for any figure, which rule produced it, where that rule sits in the code, and which section’s order it carries out. A body that can say so answers a dispute with the rule. A body that cannot repeats the figure.
This is our reading of the evidence. It is not a measured market figure.
Four Ways to Answer a Disputed Figure, Weighed
There are four real options. Each one is right for somebody.
| Option | When it fits | What it risks |
|---|---|---|
| Check the system again and confirm the figure | The figure is rarely questioned, and its rule is simple and known | The system agrees with itself; a wrong rule is confirmed as readily as a right one |
| Ask the IT cell or the supplier to trace each disputed figure | Disputes are few, and someone who knows the code is still there | Days of work per question; the answer stays with one person; a fault behind many disputes is seen one at a time |
| A one-off audit of a sample of outputs, worked out again by hand | There is a specific doubt about one calculation | It checks figures, not rules, covers a sample, and goes out of date with the next change |
| Write down the rules the system applies, each traced to the code, for the body’s own officers to judge | The system works out what people owe, are paid or qualify for, and its rules have been changed over years | It costs money and officers’ time, and may turn up rules that are wrong, which raises questions about past cases that are for the body and its legal advisers |
Our recommendation: write the rules down first, starting with the calculations that decide what people are charged, paid or allowed. Each rule goes into plain language, traced to the place in the code where it is applied, for the section that owns it to read. The officers decide whether each rule is what the department intended; we do not. We do not investigate past cases, decide disputes or say that a system is correct. A written rule does not make the system right. It makes it checkable.
Where You Are Now, and Where You Should Be
Before any work begins we set a benchmark: a few measures taken from your own systems, each with a target. For this issue it would look like this.
| Measure | Typical starting point | Where it should be | How it is checked |
|---|---|---|---|
| The rule behind each figure | In the code only | Written in plain language for each calculation the system makes | The specification |
| Where each rule is applied | Found by digging, one dispute at a time | Traced to the place in the code where it is applied | The specification, traced to the code |
| Who owns each rule | Nobody named | The section whose order it carries out, as the body names it | The specification, with the owners the body names |
| How a disputed figure is answered | ”The system shows it” | With the written rule behind it, read by the officer who decides | The body’s own record of disputes |
| Rules nobody can explain | Not known | Listed, each put before the section that owns it | The risk register |
The starting points above are what we expect to find; in an engagement they are replaced by your own figures, taken in the first review. The targets are what we would work towards with you. They are not results we have already achieved for a public body.
How We Work With You
- We listen: a free Reliability Review. A first conversation about the figures your system produces and the ones that are questioned. You leave with a short written note of what we found. You can start with a conversation about the system; we ask for read-only code access only when the assessment begins.
- We research. We research your sector, the regulator’s position and what the public record says about firms in your situation. This article is an example of that work.
- We read the system and propose: the Legacy System Specification Assessment. Here the system is the application that works out what people owe, are paid or qualify for. It produces a map of the system as it stands, a written specification of what it does, a risk register, a cost comparison between keeping and replacing the system, and a staged plan with a fixed quote. Here the specification is the part that matters: each rule the system applies, in plain language. AI does a first pass, our senior engineers review it and trace it to the code, and your own experts confirm it. Code access is read-only and logged, never used for training, and deleted within 30 days of the evaluation ending unless we engage. You get the options and our recommendation.
- If you approve, we build. For this issue our offer is narrower than a rebuild: it ends with the written rules and your officers’ decisions on them. If the body separately decides that a part should be replaced, with each rule as its officers decided it, we replace a system one module at a time while the old one keeps running. We never do a single cutover. A traceability matrix links every old function to its replacement. The work is billed by milestone, each milestone tied to a module you accept, and you can pause after any accepted module.
- We review against the benchmark. Before work starts we set the benchmark. After each module we show you where each measure stands, and the filled-in benchmark is yours at the end.
The Review is free. Every paid step is fixed price, agreed at scoping. Stop after any step. The findings are yours. The assessment fee is credited in full against the first build stage if it starts within 90 days of the report.
What we do not do: we do not investigate past errors or past cases, decide who is right in a dispute, act as an expert witness or give legal advice. We do not interpret an order, a rule or a law, and we do not say that a system is correct: we write down what it does, and the body’s own officers judge it. Our published terms on handling cover code and system access, not citizens’ records. We do not maintain the system or run its releases. We do not implement packaged finance products, and we do not work in COBOL or assembly language or on mainframes. We do not say we are empanelled with any government, and we give no advice on procurement rules: how an assessment is bought is the body’s own decision. We have no published case of this work for a public body.
What changes for the business:
| Role | What they gain |
|---|---|
| Head of department | When a figure is disputed, the department can show how it was arrived at, and can find out early if the system is at fault. |
| Head of procurement | The department can say what it expects of the next contract, because it knows which figures trace to a written rule. |
| Head of IT | A disputed figure is traced to a rule and to the code behind it, without an investigation from scratch. |
| Head of service delivery | Counter staff can explain a figure from a written rule, and know when a doubt should be referred upward. |
| Head of audit and vigilance | Before action is taken on the strength of a system’s figure, there is a written rule to check the figure against. |
The cases in this article, and what IOanyT does about that kind of problem:
| The case | What IOanyT does |
|---|---|
| A large UK public body: a supplier-built system, more than 25 years old, which the trade press reported in 2025 performed poorly while its figures were relied on. | Our part is far narrower, and we make no claim about this case. We do not investigate errors, decide disputes or say a system is correct. For a body’s own application, we write down the rules it applies, traced to the code, so its officers can check a questioned figure. |
| A large UK council: custom changes to its new finance system included a reconciliation function that failed, and the council could not produce auditable accounts. | We do not implement packaged finance products, and we are not auditors. For an application a body built or had built, we write down what each calculation does. |
| The US national auditor (July 2025): the 11 most critical federal legacy systems are “about 23 to 60 years old”. | Some use languages and machines we do not work on. For an application on technology we read, we write down the rules years of changes have left in it. |
IOanyT is a consultancy with its own engineering team: we study the problem, recommend, and prove it by building only what you approve. We start by writing down what your system already does, so that your own officers can judge it.
Sources
- Trade press, 2025
- Trade press, 2026
- US Government Accountability Office, GAO-25-107795, 17 July 2025
In this issue
Wrong outputs trusted as correct
Research note 2
Nobody Has Set Your System Beside the Rules It Was Built to Apply. Three Ways to Check, Weighed.
Public systems drift from the orders they apply and nobody compares the two. Three ways to check what a system calculates, and a new accuracy standard.
Research note 3
Every Figure Your System Produces Should Trace to a Written Rule. Here Is the Benchmark We Would Set.
Changes to an old system are checked against memory because its rules were never written down. Four ways to check a change and the benchmark we set.
Issue study
As Per System
System figures relied on that nobody can check: a national system, a failed reconciliation, a new accuracy standard, the options and our benchmark.
More from Government and public sector
Replacement fails or overruns
A Deferred Replacement Is Still a Decision. Four Ways to Replace an Old System, Weighed.
Thin or missing documentation
A New Officer Takes Charge of Your System With Nothing to Read. Three Ways to Change That, Weighed.
Upkeep absorbs the budget