Nobody Can Promise a Replacement Misses Nothing. A Benchmark Can Show It.

It is the last steering meeting before the new billing module is switched on. The finance director asks one question. Can anyone show that the new module does everything the old one did?

The project lead says every test has passed. People nod.

Nobody says where the tests came from. They were written from what staff remembered the old screens doing. A small rule the old system applies a few times a year, added long ago by a developer who has since left, is on no list.

What We Found

We went through the public record on replacing old health systems: a US government audit of a record-system replacement (February 2025), a vendor’s undated account of an old healthcare system’s size, and a survey of frontline staff reported in late 2025.

Old health systems collect decades of unwritten behaviour. The US Department of Veterans Affairs is replacing a record system that “has been in operation for more than 30 years”. Thirty years is a long time for small rules to gather. The system is far larger than anything our readers run.

They are larger than anyone remembers. One US healthcare organisation’s old administration system, described in a vendor’s case study, was built on thousands of old terminal screens, many unchanged for decades. An earlier, conventional modernisation effort had converted only a small share of its applications. We do not work in COBOL; we cite it for the scale.

A programme can halt part-way. The US auditor (GAO, February 2025) records that VA announced in April 2023, after five sites, that it would halt deployments to further sites, and that VA “identified 14 critical patient safety issues requiring development or enhancement work”. The record does not say what those issues were, and we do not suggest they were missing functions.

Staff already feel the technology they have. In a vendor-sponsored survey of more than a thousand frontline healthcare staff in the US, the UK and Ireland, reported in late 2025, almost all said inefficient technology was causing patient care and safety issues.

Our reading of these together: a test report shows that the listed functions work. It cannot show that the list was complete, and in an old health system the list never is.

What a Written Record Opens Up

A written record of the old system is usually seen as paperwork. Our reading is that it changes what the organisation can decide.

• Sign-off rests on evidence. The person accepting a module looks at a table, line by line, and not at the room’s confidence.
• Clinicians decide what they rely on. The warnings, checks and defaults the old system applies are listed in front of them, and they decide what stays.
• Each stage can be priced and stopped. A list of functions is also a list of work, priced stage by stage.

This is our reading. It is not a measured market figure.

Three Ways to Decide a Module Is Ready, Weighed

Option	When it fits	What it risks
Test the new module against the requirements document	The old system was fully documented, and the document kept up to date	The tests are incomplete where the document is, and missing behaviour fails silently
Run the whole old and new systems side by side, then switch	The system is small enough to compare as a whole	Differences surface late and all at once, close to the switch
Check each function against a written record, one module at a time, with the old system still running	The system is old, built for the organisation and poorly documented	Writing the record takes time before new code is written, and asks for hours from clinicians

Where You Are Now, and Where You Should Be

This is the benchmark we would set before the first module is built, and fill in as each module is accepted.

Measure	Typical starting point	Where it should be	How it is checked
Functions written down before pricing	A requirements document written from interviews	Every rule, report, check and connection in scope has its own numbered line before any price is set	The record is compared with the code, the database and the overnight jobs
The basis for the estimate	A list of modules and a demonstration	The written record and a risk register, with a price for each stage	The staged plan and quote
Functions linked and compared	Tests run against the document	Every line points to its place in the new module, and both are given the same input	The traceability matrix
Where gaps are found	By staff, weeks after go-live	In the check, before the module goes live	The record of gaps found and fixed
Modules accepted	One sign-off for the whole system	Each module accepted by named people; where it touches clinical work, by your clinicians	Signed acceptance for each module
Rules nobody can explain	Carried across unseen, or dropped unseen	Each one listed for the organisation to decide; clinical rules decided by your clinicians	The decision recorded on its line
Share of the old system still running	All of it until go-live, then none	Falling module by module, with the old one available until each module is accepted	Reported at every review

The starting points above are what we expect to find. In an engagement they are replaced by your own position, taken in the first review. The targets are what we would work towards with you. They are not results we have already achieved for a healthcare organisation.

A small part of the finished record might look like this.

Old system	New module	How it was checked	Accepted by
Night handover sheet, printed at ten each evening	Ward module, report 4	A week of sheets compared	Night nursing lead
Different rate for one insurer after the third day	Billing module, rule 12	Three months of claims run through both	Finance head
An on-screen check whose purpose nobody on the project could explain	Pharmacy module, check 7	Sample orders run through both	The hospital’s clinicians, who decided whether to keep it

The rows above are an illustration, not taken from a real system.

How We Work With You, and What You Are Left With

1. We listen: a free Reliability Review. A first conversation about the replacement, the tests it rests on, and who will be asked to sign. You leave with a short written note of what we found. You can start with a conversation about the system; we ask for read-only code access only when the assessment begins.
2. We research. We research your sector, the regulator’s position and what the public record says about firms in your situation. This article is an example of that work.
3. We read the system and propose: the Legacy System Specification Assessment. It produces a written specification of what the system does, a risk register, a cost comparison between keeping and replacing the system, and a staged plan with a fixed quote. AI does a first pass, our senior engineers review it and trace it to the code, and your own experts confirm it. Code access is read-only and logged, never used for training, and deleted within 30 days of the evaluation ending unless we engage.
4. If you approve, we build. We replace a system one module at a time while the old one keeps running. We never do a single cutover. A traceability matrix links every old function to its replacement. The work is billed by milestone, each milestone tied to a module you accept, and you can pause after any accepted module. Where a module touches clinical work, accepting it is your clinicians’ decision.
5. We review against the benchmark. Before work starts we set the benchmark above. As each module is accepted we show you where each measure stands, and the filled-in benchmark is yours at the end.

The Review is free. Every paid step is fixed price, agreed at scoping. Stop after any step. The findings are yours. The assessment fee is credited in full against the first build stage if it starts within 90 days of the report.

This has a price. Writing the record takes time before any new code is written, and asks for hours from senior clinicians. It will turn up rules nobody can explain. What lives only in a spreadsheet or in someone’s habit may still be missed unless your staff point us to it. And we are not a clinical safety assessor: we do not judge whether a rule or a module is clinically right. Your clinicians do, with the evidence in front of them.

At the end you hold the specification, the matrix and the filled-in benchmark. Switching off each part of the old system is your team’s decision. We do not work on packaged health record products, medical devices or plant control systems.

What changes for the business:

Role	What they gain
CEO	The decision to switch each part on is made on evidence.
CTO or CIO	A written record of everything the old system does, and proof for each function that the new one does it too.
CFO	Bills and claims from each new module can be checked against the old ones before it goes live.
COO	Staff find the reports, printouts and small conveniences they rely on still there.
Medical head	The alerts, checks and defaults in the old system are listed in front of clinicians, and whether each one stays is their decision.

The cases in this article, and what IOanyT does about that kind of problem:

The case	What IOanyT does
According to its auditor (GAO, February 2025), the US Department of Veterans Affairs is replacing a record system in operation for more than 30 years, and in April 2023 halted deployments to further sites after five sites; VA identified 14 critical patient safety issues requiring development or enhancement work.	Our part is far narrower. That programme put in a packaged health record product, which we do not implement. For applications an organisation built or had built, we write down what the old one does and match each function in the new module, with a traceability matrix, before the organisation’s own people accept it.

IOanyT is a consultancy with its own engineering team: we study the problem, recommend, and prove it by building only what you approve. We set the benchmark before the first module is built, so “nothing was missed” is a table you can read.

Book a free Reliability Review →

Get a fixed quote →

See the Legacy System Specification Assessment →