Finished Should Mean Matched. The Benchmark We Would Set for a Replacement.
A replacement reported as "seventy per cent complete" can still be years from right. Here is the benchmark we would set for replacing an old booking, dispatch or billing application, how each new part is checked against the old one, and what we will not claim.
It is month twenty of an eighteen-month project. The monthly report says the new booking application is seventy per cent complete. Three months ago it said sixty-five.
The new application has been on trial at one branch since spring. Each morning its daily report is laid beside the old system’s, and each morning the totals differ a little. Last week a returning group was quoted more than last year, because the old system gave such groups a small discount that nobody had listed. The week before, a part-cancelled trip was refunded twice.
The project team fixes one difference and another appears. Nobody can say which system is right, because nobody ever wrote down how the old one reaches its figures.
The paradox: The project reports how much has been built, and nobody can say how much of it is right.
What We Found
We looked at what the public record says about changing old systems without a record of what they do, at why firms hesitate to replace them, and at one long-serving piece of airline software.
First, what we did not find: no case of a mid-size logistics, travel or hotel firm whose replacement stalled because new and old did not agree, and no figure for rework. The one organisation we name is far larger than our readers.
Changing an old system with no record of what it does is what engineers fear most. One developer, writing about working on old codebases, put it plainly: “There’s nothing scarier than making changes to a critical legacy system with no tests.” It is one person’s account, and not about this sector. Our point: without a record of what the system does, nobody can tell a fault from a feature.
Firms fear the disruption more than the build. In one cross-sector survey of 504 IT professionals in the US, which is not about this sector, 38% named fear of disrupting operations among their reasons for not upgrading, and 39% named high maintenance or support costs among the top issues with the old systems they keep.
Old operations software can stay in service for a very long time. This is a very large airline, and airline operations systems are not our kind of work. At a large US airline, crew-scheduling software was “First implemented in 2004”. A report of July 2026 says “The airline hopes to complete the transition by 2028”, a move of its systems to a new architecture. We report that as the airline’s hope, not as a fact or a model. Our narrow point: a system that has served that long holds years of rules someone must account for.
What our review does not show: We found no firm in this sector of our readers’ size whose replacement stalled on differences between old and new, and nothing from the Middle East on this issue. The developer’s words are one account, cross-sector. The survey is cross-sector, from a firm that sells related services. The airline is far beyond our readers’ scale, its 2022 disruption is an older case and its 2028 date is a hope. That replacements lose their later months to differences nobody recorded, and that checking each new part against the old one moves that work earlier, is our reasoning, not a measured finding. We found no evidence on how much rework such checking removes, and we claim none.
What a Record of the Old System Opens Up
A record of what an old application does is usually seen as paperwork. Our reading is that it is what lets anyone call a part finished.
- “Finished” gets a meaning. A new part is finished when it does what the old one did, function by function, or differs only where somebody chose a difference.
- Differences are found where they were made. Each module is checked against the old application before the next is begun, so a misunderstanding from the second month is found in the second month.
- Progress can be checked by the people who use it. An accepted module in use is progress the branch can see.
The gap we see: most replacement projects can say how much has been built and how much has been spent. Few can say, part by part, what the old application did, whether the new part does the same, and who accepted it. A firm that holds that record can tell a finished part from a nearly finished one.
This is our reading. It is not a measured market figure.
Three Ways to Know a New Part Is Finished, Weighed
| Option | When it fits | What it risks |
|---|---|---|
| Report percentage complete, and compare old and new in one trial at the end | The application is small, and one go-live is safe | Differences surface at the end, when the money is nearly gone and every part changes at once |
| Test each part against requirements written in workshops | The old application’s rules are well documented | Each part passes against what people remember. What nobody remembered is found on live bookings |
| Read and write down what the old application does first, then rebuild module by module, each checked against the old one with a traceability matrix and accepted before the next is begun | The application is the firm’s own, much changed over the years, and the business cannot stop | It costs money before new code is written. Old and new run side by side for longer, and the whole may take as long as one project, or longer |
Our recommendation: write down what the old application does first, confirmed by your own experts. Then rebuild it one module at a time beside the old application, with a traceability matrix that links every old function to its replacement, and accept each module only when it matches or differs where you chose. If your own team wants automated tests built from the old application’s outputs as well, that is their option and their work; we do not claim it as our service. This is for applications a firm built or had built, in technologies we read. We do not keep the old application running, and we do not move or reconcile your records as a stated service.
Where You Are Now, and Where You Should Be
This is the benchmark we would set before the first module is rebuilt.
| Measure | Typical starting point | Where it should be | How it is checked |
|---|---|---|---|
| Modules in use after the first stage | None until the final go-live | At least one module accepted and in use, with the old application doing the rest | Acceptance recorded for each module |
| A price for each stage | One figure for everything | A price for each stage, known before that stage is approved | The staged plan with a fixed quote |
| What the old application does | In the heads of two or three people | Written down, function by function, and confirmed by your own experts | The specification |
| Old behaviour matched | Compared once, in a trial at the end | Every old function linked to its replacement and checked before the module is accepted | The traceability matrix, module by module |
| Differences between old and new | Found on live bookings | Each one listed, and either fixed or chosen and signed off | The traceability matrix and the acceptance record |
| The size of one commitment | The whole replacement, or nothing | One module, with the freedom to pause after any accepted module | The staged plan |
| Progress | Percentage complete | Modules accepted and in use | Reported at every review |
The starting points above are what we expect to find. In an engagement they are replaced by your own position, taken in the first review. The targets are what we would work towards with you. They are not results we have already achieved for a logistics, travel or hospitality firm.
A small part of the record might look like this.
| Module | Checked against the old application | Status |
|---|---|---|
| Group fares and discounts | Each fare rule linked to its replacement; a month of quotes run through old and new | Accepted by the head of sales; in use |
| Cancellations and refunds | Each refund rule linked; part-cancelled trips compared case by case | Accepted by the finance head; in use |
| Agent statements | Not yet | Quoted, not started; the firm chose to pause here |
The rows above are an illustration, not taken from a real firm.
How We Work With You, and What You Are Left With
- We listen: a free Reliability Review. A first conversation about the application and how the replacement’s progress is reported. You leave with a short written note of what we found. You can start with a conversation about the system; we ask for read-only code access only when the assessment begins.
- We research. We research your sector, the regulator’s position and what the public record says about firms in your situation. This article is an example of that work.
- We read the system and propose: the Legacy System Specification Assessment. Here the system is the booking, dispatch or billing application your firm built or had built, in a technology we read: VB6, Delphi, AngularJS, PHP 4 to 8, Java 8, or .NET Framework and classic ASP. It produces a written specification of what it does, a risk register, a cost comparison between keeping and replacing the system, and a staged plan with a fixed quote. AI does a first pass, our senior engineers review it and trace it to the code, and your own experts confirm it. Code access is read-only and logged, never used for training, and deleted within 30 days of the evaluation ending unless we engage.
- If you approve, we build. We replace a system one module at a time while the old one keeps running. Here that means your own application, rebuilt beside the old one, each module checked against it before you accept it. We never do a single cutover. A traceability matrix links every old function to its replacement. The work is billed by milestone, each milestone tied to a module you accept, and you can pause after any accepted module.
- We review against the benchmark. Before work starts we set the benchmark above. As each module is accepted we show you where each measure stands, and the filled-in benchmark is yours at the end.
The Review is free. Every paid step is fixed price, agreed at scoping. Stop after any step. The findings are yours. The assessment fee is credited in full against the first build stage if it starts within 90 days of the report.
This has a price. Writing down what the old application does costs money before any new code is written. Old and new run side by side for longer, which has a cost, and bookings and records must be kept in step between them; we do not move, clean or reconcile records as a stated service, and your own team or supplier keeps the old application running until each module is accepted. A traceability matrix shows that a module does what the old one did; it does not show that the old one was right, and it does not show how the module behaves at the heaviest volume. We do not implement, upgrade or move a packaged reservation, booking, warehouse, property management or ERP product. We do not work on the systems that run aircraft, trains or vehicles. We do not say a staged replacement is quicker or cheaper overall. We have no published case of this work for a logistics, travel or hospitality firm.
At the end you hold the specification, the traceability matrix and code for every module we rebuilt, and the filled-in benchmark. Switching off each part of the old application is your team’s decision.
What changes for the business:
| Role | What they gain |
|---|---|
| CEO | Progress is reported as parts that match the old application and are in use, and not as a percentage nobody can check. |
| CTO or CIO | A written record of what the old application does, and a traceability matrix for each module, say when a new part is finished, so the second year is not spent repairing the first. |
| CFO | Rework is found within the stage that caused it, and is not carried forward into a later budget. |
| COO | Staff start on screens that give the answers they are used to, and any difference that remains is one somebody chose. |
| Head of sales and marketing | Agents and customers see the same prices and confirmations as before on the day a part changes, unless a change was intended. |
The cases in this article, and what IOanyT does about that kind of problem:
| The case | What IOanyT does |
|---|---|
| A large US airline’s crew-scheduling software was first implemented in 2004; the airline hopes to complete the transition of its systems to a new architecture by 2028. An older case for the 2022 event. | Our part is far narrower, and we do not work on airline systems or say how the airline’s transition should be done. For an application a firm in this sector built or had built, we write down what it does and check each rebuilt module against it before it takes over. |
IOanyT is a consultancy with its own engineering team: we study the problem, recommend, and prove it by building only what you approve. We would rather report one module that matches the old application than seventy per cent of a system nobody has checked.
Sources
In this issue
Too slow and costly to replace