Every Figure Your System Produces Should Trace to a Written Rule. Here Is the Benchmark We Would Set.

It is the evening before a release. The head of IT is looking at one small change: a new slab in the fee table, as ordered last month.

The developer has tried it on three cases, by hand. All three gave the right fee.

The head of IT has a different question. The fee routine is also used by the arrears calculation, and perhaps elsewhere. Has anything there moved? There is nothing to check it against. Nobody ever wrote down what the arrears calculation is meant to do.

What We Found

For this article we went through the public record on what happens around changes to systems that produce figures: reporting in 2026 on a large UK council’s finance system, the US national auditor’s July 2025 report on the most critical federal legacy systems, and reporting in 2025 on a large UK public body’s system.

First, what we did not find: no figure for how often a change to a public body’s system alters a calculation it was not meant to touch, no Indian public body or body of our readers’ size on the record, and nothing from the Middle East. Every body described below is far larger than those we write for.

The function that failed was among the changes. A large UK council replaced its finance system with a packaged product. Custom changes were made, and they included a reconciliation function that failed. The council could not produce auditable accounts. This was a packaged finance product, which is not our kind of work. We take one point, in the source’s terms: the function that failed was one of the changes made to the product.

Old systems are kept for a long time. The US national auditor reported in July 2025 that about 80% of the more than 100 billion US dollars the federal government spends on IT each year goes on running existing systems, and that the 11 most critical federal legacy systems are “about 23 to 60 years old”. Some use languages and machines that are not our kind of work. A system kept that long has, in our reading, been changed many times.

One case shows what is at stake when a system’s figures are relied on. A large UK public body ran a supplier-built system, more than 25 years old, which, as the trade press reported in 2025, performed poorly while its figures were relied on. We add nothing to that account; we do not say what the fault was, and we do not suggest it came from a change. It is a national body’s system, far outside our kind of work.

Our reading: an old system is joined up inside in ways no drawing shows. A routine written for one calculation was borrowed for others. Nothing runs through the system after a change and reports what is now different. The developer tries the cases the change was meant for, and the cases nobody thought of are the ones that go wrong. And even where someone wants to check, there is nothing to check against, because the rules were never written down.

What a Written Rule Opens Up

The usual view is that changes to an old system are a matter of careful testing. Our reading is that a written rule comes first, because a change can only be checked against something, and four things open up once the rule exists.

A dispute is answered with the rule. The officer hearing it reads the rule behind the figure, not only the figure.

A change can be described as a change to named rules. The request says which rules change and which must not. Acceptance then has something to rest on.

Any supplier’s change can be checked against the department’s own record. The written rules belong to the department, not to whoever maintains the system this year.

Audit sees what was checked. Each change is recorded against the written rules it altered.

This is our reading of the evidence. It is not a measured market figure.

Four Ways to Check a Change, Weighed

There are four real options. Each one is right for somebody.

Option	When it fits	What it risks
Accept the supplier’s word that the change “works as per requirement”	The change is small and the supplier has a long, clean record	The department has no evidence of its own; a calculation that shares the code is not looked at
Try the change by hand on the cases it was meant for	The rule is applied in one place only	It says nothing about other calculations that share the same code
Write down the rules, give each an owning section, and check every change against the written rule	The system’s figures decide what people pay, receive or qualify for, and it changes often	It costs money and officers’ time before any change is checked; the written rules must be kept current after each change, which is the body’s own team’s work
As above, and the body’s own team adds automated tests that report when a change alters a result	The body has a team able to build and keep tests	Tests report a difference, not an error, and will protect a mistake already there; this is the body’s own team’s work, not part of our offer

Where You Are Now, and Where You Should Be

Before any work begins we set a benchmark: a few measures taken from your own systems, each with a target. For this issue it would look like this.

Measure	Typical starting point	Where it should be	How it is checked
Figures traceable to a written rule	None, or a few known to one person	Every calculation the system makes has a written rule, traced to the code	The specification
Rules with an owning section	None named	Each rule with the section the body names as its owner	The specification, with the owners the body names
How a disputed figure is answered	”The system shows it”	With the rule, not only the figure	The body’s own record of disputes
How a change is checked	Tried by hand on the cases it was meant for	Described as a change to named rules, and checked against the written rule by the body’s own team	The body’s own release record, set against the specification
Rules nobody can explain	Not known	Listed, each put before its owning section	The risk register

The starting points above are what we expect to find; in an engagement they are replaced by your own figures, taken in the first review. The targets are what we would work towards with you. They are not results we have already achieved for a public body.

How We Work With You

1. We listen: a free Reliability Review. A first conversation about how changes to your system are made and checked. You leave with a short written note of what we found. You can start with a conversation about the system; we ask for read-only code access only when the assessment begins.
2. We research. We research your sector, the regulator’s position and what the public record says about firms in your situation. This article is an example of that work.
3. We read the system and propose: the Legacy System Specification Assessment. Here the system is the application whose figures the department relies on. It produces a map of the system as it stands, a written specification of what it does, a risk register, a cost comparison between keeping and replacing the system, and a staged plan with a fixed quote. Here the specification is the part that matters: the written rules every later change can be checked against. AI does a first pass, our senior engineers review it and trace it to the code, and your own experts confirm it. Code access is read-only and logged, never used for training, and deleted within 30 days of the evaluation ending unless we engage. You get the options and our recommendation.
4. If you approve, we build. For this issue our offer is narrower than a rebuild: it ends with the written rules and your officers’ decisions on them. If the body separately decides that a part should be replaced, with each rule as its officers decided it, we replace a system one module at a time while the old one keeps running. We never do a single cutover. A traceability matrix links every old function to its replacement. The work is billed by milestone, each milestone tied to a module you accept, and you can pause after any accepted module.
5. We review against the benchmark. Before work starts we set the benchmark. After each module we show you where each measure stands, and the filled-in benchmark is yours at the end.

The Review is free. Every paid step is fixed price, agreed at scoping. Stop after any step. The findings are yours. The assessment fee is credited in full against the first build stage if it starts within 90 days of the report.

What we do not do: we do not add tests, maintain the system, keep its written rules current or run its releases; the body’s own team does. We do not say that a system or a result is correct, investigate past cases, decide disputes, act as an expert witness, interpret orders or give legal advice. Our published terms on handling cover code and system access, not citizens’ records. We do not implement packaged finance products, and we do not work in COBOL or assembly language or on mainframes. We do not say we are empanelled with any government, and we give no advice on procurement rules. We have no published case of this work for a public body.

What changes for the business:

Role	What they gain
Head of department	A change to the system is described as a change to named, written rules, so the department knows what it approved.
Head of procurement	The written rules belong to the department, so any supplier’s change can be checked against the same record.
Head of IT	Before a release, the team can see which written rules a change touches and check the results against them.
Head of service delivery	Supervisors know which written rule a change altered, and can explain the new figure.
Head of audit and vigilance	Each change is recorded against the written rules it altered, so there is something to examine afterwards.

The cases in this article, and what IOanyT does about that kind of problem:

The case	What IOanyT does
A large UK council: custom changes to its new finance system included a reconciliation function that failed, and the council could not produce auditable accounts.	We do not implement packaged finance products, and we are not auditors. For an application a body built or had built, we write down each calculation, so that later changes have something to be checked against.
The US national auditor (July 2025): about 80% of federal IT spending goes on running existing systems; the 11 most critical are “about 23 to 60 years old”.	Some use languages and machines we do not work on. For an application on technology we read, we write down the rules it applies.
A large UK public body: a supplier-built system, more than 25 years old, which the trade press reported in 2025 performed poorly while its figures were relied on.	Our part is far narrower, and we make no claim about this case or its cause. We write down the rules a body’s own application applies, so its officers can check its figures.

IOanyT is a consultancy with its own engineering team: we study the problem, recommend, and prove it by building only what you approve. We write down the rules your figures come from, so that every later change has something to be checked against.

Book a free Reliability Review →

Get a fixed quote →

See the Legacy System Specification Assessment →