"Nothing Will Be Lost" Is a Promise. A Benchmark Is Something You Can Check.
No technology head can honestly promise a new system does everything the old one did. Here is the benchmark we would set before go-live, and how it is checked function by function.
You are in the steering committee for the new lending system. Go-live is six weeks away.
The chief executive asks one question. Can you promise that everything the old system does today, the new one will do on day one?
You look at the test report. Every test has passed. You also know what the tests were written from: a requirements document, which was written from interviews, which captured what people remembered.
The paradox: You are asked to promise that the new system matches the old one, and nobody can tell you what the old one does.
What We Found
We looked at where the rules of an old lending system are kept, and at what the public record says about the moment a replacement goes live.
The rules are in the code. A special rate for one customer group. A waiver agreed in a meeting. A rounding rule added after an audit. Each one was a small change at the time. None of them reached a document.
Some rules are not in the application at all. They sit in the database as stored procedures (small programs that run inside the database). They sit in scheduled jobs that run overnight. They sit in a spreadsheet someone in finance uses to correct a figure every month end.
The scale is beyond memory. When one group of more than 40 credit unions replaced a core system, the work meant rebuilding more than 200 screens and more than 500 interfaces to other systems. No round of interviews brings all of that out.
Going live is when every gap shows at once. When TSB Bank went live on its new platform in 2018, the bank received 225,492 complaints and was later fined £48.65 million.
What our review does not show: The first two findings are our own reasoning about how old systems grow. The record does not say why TSB’s migration failed in terms of testing, and we do not claim it. Both named cases are from outside India, and one is a large bank.
What a Proved Replacement Opens Up
Proof of a match is usually seen as protection: against complaints, corrections and questions from the auditor. It is that. Our reading is that it is also an asset the business did not have before.
- A system that is written down can be changed with confidence. The next product variant or fee change starts from a record of what the rule is today.
- Figures can be explained. When a customer or the finance team asks why a number is what it is, there is a written rule to point to.
- The record is ready when someone asks. One 2025 survey of 500 leaders at large enterprises reported that 94% said regulatory compliance strongly influences their modernisation decisions.
The gap we see: most lenders running an old system cannot state, in writing, what it does. A lender that can has something the others lack. The replacement is the moment to get there, because the work of proving a match produces the record.
This is our reading. It is not a measured market figure.
Three Ways to Decide You Are Ready, Weighed
| Option | When it fits | What it risks |
|---|---|---|
| Test the new system against the requirements document | The old system was fully documented and the document was kept up to date | The tests are incomplete in exactly the places the document is. Missing rules fail silently |
| Run the whole old and new systems side by side, then switch | The system is small enough to compare as a whole | Differences are found late and all at once, close to the switch |
| Check each function against a written record, one module at a time | The system is old, customised and poorly documented | Writing the record takes time before any new code is written |
Our recommendation: for an old, customised system, check function by function, one module at a time. The question changes from “are we confident?” to one that can be answered with a document: for each thing the old system does, where is it in the new system, and how was it checked?
Where You Are Now, and Where You Should Be
This is the benchmark we would set before the first module is built, and fill in as each module is accepted.
| Measure | Typical starting point | Where it should be | How it is checked |
|---|---|---|---|
| Functions with a written line | A requirements document written from interviews | Every rule, report, calculation and interface in the module has its own numbered line | The record is compared with the code, the database and the scheduled jobs |
| Functions linked to their replacement | No link is kept | Every line points to the place in the new system that does the same thing | The traceability matrix |
| Functions compared, old against new | Tests run against the document | The same input is given to old and new, and the results are compared | The result is recorded on each line |
| Where gaps are found | By customers or finance, weeks after go-live | During the check, before the module goes live | The record of gaps found and fixed |
| Who accepts each module | One sign-off for the whole system | A named person signs each module, with the table in front of them | Signed acceptance for each module |
| Rules nobody can explain | Carried across unseen, or dropped unseen | Each one listed, for the business to decide what to keep | The decision is recorded on its line |
The starting points above are what we expect to find. In an engagement they are replaced by your own position, taken in the first review. The targets are what we would work towards with you. They are not results we have already achieved for a lender.
A small part of the finished record might look like this.
| Old system | New system | How it was checked | Result |
|---|---|---|---|
| Penal interest on overdue instalments, rule 14 | Interest module, overdue calculation | 200 sample accounts run through both | Match |
| Monthly report to finance on written-off accounts | Reporting module, report 7 | Last three month ends compared | Match |
| Overnight file of repayments sent to the collections agency | Integration module, job 3 | Seven nights of files compared | One field differs: fixed and re-run |
The rows above are an illustration, not taken from a real system.
How We Work With You, and What You Are Left With
- We listen: a free Reliability Review. A first conversation about the system, the go-live decision, and who is being asked to make the promise. You leave with a short written note of what we found.
- We research. We research your sector, the regulator’s position and what the public record says about firms in your situation.
- We read the system and propose: the Legacy System Specification Assessment. It produces a written specification of what the system does. AI does a first pass, our senior engineers review it and trace it to the code, and your own experts confirm it. You can start with a conversation about the system; we ask for read-only code access only when the assessment begins. Code access is read-only and logged, never used for training, and deleted within 30 days of the evaluation ending unless we engage.
- If you approve, we build. We replace a system one module at a time while the old one keeps running. We never do a single cutover. A traceability matrix links every old function to its replacement. The work is billed by milestone, each milestone tied to a module you accept, and you can pause after any accepted module.
- We review against the benchmark. Before work starts we set the benchmark above. As each module is accepted we show you where each measure stands, and the filled-in benchmark is yours at the end.
The Review is free. Every paid step is fixed price, agreed at scoping. Stop after any step. The findings are yours. The assessment fee is credited in full against the first build stage if it starts within 90 days of the report.
This has a price. Writing the record takes time before any new code is written. It will also turn up rules that nobody can explain, and some will be mistakes the old system has been making for years. The business then decides, rule by rule, what to keep. That is slower than not knowing. It is also the first time anyone has decided.
At the end you hold three things: the specification, the matrix, and the benchmark with every line filled in.
What changes for the business:
| Role | What they gain |
|---|---|
| CEO | A go-live decision made on evidence. |
| CTO or CIO | A documented system, which makes every later change quicker and safer. |
| CFO | Figures that can be explained from the first month end. |
| COO | Staff start on a system that behaves as they expect. |
| Risk and compliance | A record for every function, ready for audit. |
The cases in this article, and what IOanyT does about that kind of problem:
| The case | What IOanyT does |
|---|---|
| TSB Bank received 225,492 complaints when its new platform went live. | We check each module against a written record of the old system, with a traceability matrix, before that module goes live. Gaps are found by us, not by customers. |
IOanyT is a consultancy with its own engineering team: we study the problem, recommend, and prove it by building only what you approve. The next time someone asks whether anything will be lost, you hand them the table.
Sources
In this issue
Fear of a failed system replacement
Research note 1
Lenders Do Not Fear Their Old Systems. They Fear One Weekend.
We reviewed the public record of system replacements in financial services: what lenders fear, what waiting costs in growth, and the benchmark we would set.
Research note 2
Before You Ask What a Replacement Costs, Ask What It Is Worth.
Three quotes to replace an old lending system, far apart. Why they differ, what they leave out, the ways to buy, and the benchmark for the budget decision.
Issue study
When Replacing the System Is the Biggest Risk
TSB Bank's 2018 migration from the public record: the decision at stake, a lender's options, what we would advise against, and the benchmark we would set.
More from Banking
Unsupported software and hardware
"Accepted Again" Is Not a Plan. A Shrinking Register Is.
Outages at peak load