A New System Can Pass Every Test and Still Miss the Old One's Rules.

The steering committee meets for the last time before go-live. The test report is on the table: four hundred and twelve test cases, all passed.

The head of operations has one question. Has anyone put last Tuesday’s orders through the new system and compared what came out?

Nobody has. The tests were written from the requirements document, and the document was written from workshops. In the workshops nobody mentioned that the old system holds back stock from a batch still waiting for its quality result, because the old system has done it by itself for eleven years.

What We Found

For this article we went through the public record on manufacturers whose results fell after a new system went live, looking for what the record says was missing, and at one survey on why firms hold back from replacing old systems.

The faults showed themselves after go-live, in the business. An avocado supplier turned on a new ERP system in November 2021. Afterwards it no longer knew for sure “how many avocados it had on hand”. It hired a consultant to sort out the system, at a cost of 3.8 million US dollars “over the following nine months”, and said that a 22.2 million US dollar year-on-year drop in gross profit “for the quarter following the go-live was primarily due to the ERP problem”. The same account notes a poor avocado harvest at the time. This is an older case.

The loss can run for a year. At a German confectionery maker, problems at the go-live of a packaged ERP product were followed by “a 25% decline in sales” of its best-known sweet in 2018. This is an older case.

Moving the data worries firms as much as moving the system. In one survey of 504 IT professionals in the US, run by a firm that sells software services, 35% named data migration concerns among their reasons for not replacing old systems. That survey covers every sector, not manufacturing alone.

The record does not list which rules the new systems left out, and we do not guess. Our reading is narrower: an old order, stock and dispatch application is the sum of years of small decisions, few of which were written down. Which stock is promised first. Which customer may take a part load. Which batch is held for a quality result. A test written from a requirements document can only check what the document holds.

What Opens Up When the Old System Can Be Asked

The usual way to make a replacement safer is more testing at the end. Our reading is that the larger opening is earlier.

• The replacement can start. A firm that knows what its old applications do can replace them a part at a time, and stop waiting for a quarter it can afford to lose.
• New work stops landing on old ground. Once a customer rule is written down and rebuilt on current technology, the next new rule can be built beside it, not inside an application everyone is afraid to touch.
• An ERP change can be checked. When the firm’s own applications are described in writing, a change to the packaged ERP beside them can be checked against what they must keep doing.

This is our reading of the evidence. It is not a measured market figure.

Four Ways to Know a New Part Is Ready, Weighed

There are four real options. Each one is right for somebody.

Option	When it fits	What it risks
Test against the requirements document	The requirements were written from the system itself, and recently	A rule nobody mentioned is missing from the document and from the tests, in the same place
A user trial with sample orders	The screens are new but the rules behind them are simple	Sample orders are tidy. The exceptions that happen twice a month are not among them
Run the whole new system beside the old one and compare by hand	The firm can staff two systems for weeks and compare everything at once	Differences are found in bulk and late, and nobody can say which rule caused each one
Write down what the old applications do from their code, then check each new module against it	The rules are old, many and unwritten, and dispatch must keep running	It costs money before anything changes, and the record will show rules nobody wants to own

Where You Are Now, and Where You Should Be

Before any work begins we set a benchmark: a few measures taken from your own applications, each with a target.

Measure	Typical starting point	Where it should be	How it is checked
Rules the old applications apply to orders, stock and dispatch	In the code, and in people’s memory	Each written down, including the exceptions	The specification
Where each rule lives	Not known	Named: which application, which part	The specification and the map of the system
Rules nobody owns	Not known	Listed for the business to decide	The risk register
What each new module is checked against	A requirements document	The old module’s functions, one by one	The traceability matrix
Where a missing rule is found	At the dispatch desk, after go-live	Before the module is accepted	Reported at every review

The starting points above are what the public record suggests and what we expect to find. In an engagement they are replaced by your own figures, taken in the first review. The targets are what we would work towards with you. They are not results we have already achieved for a manufacturer.

How We Work With You

1. We listen: a free Reliability Review. A first conversation about your applications, the test plan for the replacement, and what nobody is sure the old system does. You leave with a short written note of what we found. You can start with a conversation about the system; we ask for read-only code access only when the assessment begins.
2. We research. We research your sector, the regulator’s position and what the public record says about firms in your situation. This article is an example of that work.
3. We read the system and propose: the Legacy System Specification Assessment. Here the system is the order, stock and dispatch applications you built or had built beside your ERP. It produces a map of the system as it stands, a written specification of what it does, a risk register, a cost comparison between keeping and replacing the system, and a staged plan with a fixed quote. AI does a first pass, our senior engineers review it and trace it to the code, and your own experts confirm it. Code access is read-only and logged, never used for training, and deleted within 30 days of the evaluation ending unless we engage. You get the options and our recommendation.
4. If you approve, we build. We replace a system one module at a time while the old one keeps running. Here that means your own order, stock and dispatch applications, each new module checked against the record of the old one. We never do a single cutover. Each module has a traceability matrix. The work is billed by milestone, each milestone tied to a module you accept, and you can pause after any accepted module.
5. We review against the benchmark. Before work starts we set the benchmark. After each module we show you where each measure stands, and the filled-in benchmark is yours at the end.

The Review is free. Every paid step is fixed price, agreed at scoping. Stop after any step. The findings are yours. The assessment fee is credited in full against the first build stage if it starts within 90 days of the report.

This approach has limits. A record of what the old system does shows what it does, not whether it is right: an old mistake is written down as faithfully as a good rule, until your people decide about it. A module that matches the old one function by function can still behave differently under a full day’s load, and we promise nothing about load. We do not keep the old applications running or secure them in the meantime; your own team or supplier does. We do not install, configure, upgrade or change a packaged ERP product, and we do not work on plant control systems or machines. We have no published case of this work for a manufacturer.

What changes for the business:

Role	What they gain
CEO	A replacement plan whose readiness the managing director can question in plain words, part by part.
CTO or CIO	Each new module checked against what the old one actually does, not against what a workshop remembered.
CFO	Money released for a module once it matches the old one, not for a pass rate on a report.
COO	The rules stores and dispatch rely on, from part loads to quality holds, written down before anything changes.
Head of risk	Evidence, held by the company itself, of what each module was checked against before it went live.

The cases in this article, and what IOanyT does about that kind of problem:

The case	What IOanyT does
After an avocado supplier turned on a new ERP system in 2021, it no longer knew for sure how many avocados it had on hand, and said a drop in gross profit was “primarily due to the ERP problem”.	Our part is far narrower than this case. We do not install packaged ERP products, and we draw no cause from the record. For a manufacturer’s own applications, we write down what the old ones do and check each new module against it.
A confectionery maker’s go-live problems were followed by “a 25% decline in sales” of its best-known product in 2018.	The same limit applies. Each module we rebuild is accepted by your own people, one at a time, while the old application keeps running.

IOanyT is a consultancy with its own engineering team: we study the problem, recommend, and prove it by building only what you approve. We check every new part against the old one, because the old one is the only full record of what the business needs.

Book a free Reliability Review →

Get a fixed quote →

See the Legacy System Specification Assessment →