Before Old Data Moves, Someone Should Write Down Where It Cannot Be Trusted.
A new system starts with whatever data the old one held, and its faults are usually found after go-live. Here is the benchmark we would set for retail data, and the path from a traced cause to a move that starts with known data.
The new system has been live for three days. It is fast, the screens are clear, and the staff like it.
On the fourth day the warehouse reports that it cannot find two hundred items the system says are there. Customer service finds that a third of loyalty accounts have no email address. Finance finds products with a cost price of zero.
None of this is the new system’s doing. Every one of those records came across from the old one, exactly as it was. The project plan had one line for it: migrate data.
The paradox: The business bought a new system to get figures it could trust, and filled it on the first day with the figures it did not.
What We Found
We looked at what the public record says about data when a retail system is replaced, and at the systems it comes from.
A new system can start with data nobody had measured. When a large retailer’s Canadian business rolled out a packaged ERP product in 2013, its supply chain failed. A later press account says: “An investigation found that only about 30% of the data in the system was actually correct.” An older case, and a packaged product, which is not our kind of work; the record does not say how the data came to be wrong. We take one narrow point: the state of the data decided what the new system could do.
Data worries hold back change. In a vendor’s survey of 504 IT professionals in the US, 35% named data migration concerns among the reasons for not upgrading old systems. It covers every sector, not retail alone.
The copies disagree before anything moves. In a vendor-sponsored survey of “more than 100 senior retail and brand leaders” in the UK: “Among direct-to-consumer brands, 70% said their core systems were not yet fully integrated”. Where systems are not integrated, each tends to keep its own copy of a customer, and a move must choose between them; that point is our reasoning.
What our review does not show: We found no figure for how often retail data is wrong or what correcting it after go-live costs; the numbers in the scene above are imagined. The one named case is from 2013, concerns a packaged product at a very large retailer and records no cause. Both surveys come from firms that sell related services; the US survey is not about retail, and the UK survey is about integration. We found no named mid-size retailer with this problem, and nothing from India or the Middle East. The link between describing old data first and a calmer go-live is our reasoning.
What a Written Description of the Data Opens Up
Moving data is usually planned as the last, simplest step of a replacement. Our reading is that knowing the data first changes more than that step.
- The move gets a real price. The work of putting data right is priced before the contract, not found in the first week.
- “Done” gets a meaning for data. A matching count says nothing was dropped. A description of each field says what the records should hold, so your own staff can check them.
- Less has to move. Old fields nobody needs can be left behind on purpose, decided before the project, not by guess during it.
The gap we see: most retailers plan a replacement around its new screens and give the data one line. The difference between that and a go-live without surprises is a written description of the old data, made from the code that wrote it: what each field means and where it cannot be relied on. A retailer that holds it starts the new system with known data.
This is our reading. It is not a measured market figure.
Three Ways to Bring Old Data Into a New System, Weighed
| Option | When it fits | What it risks |
|---|---|---|
| Move the data as it is and correct it in the new system | The data is small, recent and mostly trusted | The old faults arrive with a fresh date. The correcting is found after go-live and is in no budget |
| Clean everything before the move | The firm has the staff and time, and no cause of the errors is still running | The rules that made the errors run until the day of the move, and staff clean fields nobody can explain |
| Trace where the wrong figures are made, rebuild those parts with checks at the point of entry, and write down what the old data means and where it is weak before the move | The data is old, much corrected, and spread across systems the firm built or had built | It takes stages and costs money before the move starts. What it finds is usually unwelcome |
Our recommendation: trace each key figure to where it is made, and rebuild the parts that make figures wrong, one module at a time, with checks at the point of entry. Before any move, write down what each field of the old data means and where it is weak. Your own staff, or the firm doing the move, then measure and correct the affected records. We do not clean, correct or reconcile records, count stock or move data. We read the code and the structure of the database, not the records.
Where You Are Now, and Where You Should Be
This is the benchmark we would set before the first module is rebuilt, and fill in as each module is accepted.
| Measure | Typical starting point | Where it should be | How it is checked |
|---|---|---|---|
| Key figures (stock, price, product, customer) | Known to be wrong, not traced | Each traced to the rule, join or manual step that produces it | The specification |
| Checks at the point of entry | An old screen accepts blanks, duplicates and impossible values | Each rebuilt part refuses an entry that makes no sense, with the checks written down | The specification and the traceability matrix, for modules we rebuild |
| The old data’s weak fields | Unknown until go-live | Each described, from the code, before any move: what it means and where it cannot be relied on | The specification and the risk register |
| Disagreement between systems on a key figure | Noticed, never measured | Measured by your own staff on a sample they choose, and falling as each part is rebuilt | Your staff’s own count, reported at every review |
| The plan’s line for data | ”Migrate data” | Each weak field with a decision: correct, leave behind, or carry knowingly | The staged plan, agreed with your own people |
The starting points above are what we expect to find. In an engagement they are replaced by your own position, taken in the first review. The targets are what we would work towards with you. They are not results we have already achieved for a retailer.
Two lines need a word. The traceability matrix shows that each old function has its replacement, not that every record is right; that is why disagreement between systems is its own measure, counted by your own staff. And describing the old data’s weak fields goes beyond what we have published: it extends the specification from the code to the data the code wrote. Our delivery team has been asked to confirm it.
A small part of a finished description might look like this.
| Field | What the code says it means | Where it cannot be relied on | Decision |
|---|---|---|---|
| ”Status 2” on an order | 9 means a return was opened in store and not yet completed | Never cleared when a return is cancelled | Your staff correct the open ones; the field is not carried across |
| Cost price on a product | Copied from the supplier’s file each night | Blank when the file is late; the old screen saves the product anyway | Rebuilt entry check refuses a blank cost; your staff fill the gaps before the move |
The rows above are an illustration, not taken from a real retailer.
How We Work With You, and What You Are Left With
- We listen: a free Reliability Review. A first conversation about the replacement you are planning. You leave with a short written note of what we found. You can start with a conversation about the system; we ask for read-only code access only when the assessment begins.
- We research. We research your sector, the regulator’s position and what the public record says about firms in your situation.
- We read the system and propose: the Legacy System Specification Assessment. Here the system is the code your firm built or had built that writes stock, price, product and customer data, and the structure of its database. It produces a written specification of what it does, a risk register, a cost comparison between keeping and replacing the system, and a staged plan with a fixed quote. AI does a first pass, our senior engineers review it and trace it to the code, and your own experts confirm it. Code access is read-only and logged, never used for training, and deleted within 30 days of the evaluation ending unless we engage.
- If you approve, we build. We replace a system one module at a time while the old one keeps running. Here that means the parts that make figures wrong, rebuilt beside the old system on technology that is still supported, with checks at the point of entry. We never do a single cutover. A traceability matrix links every old function to its replacement. The work is billed by milestone, each milestone tied to a module you accept, and you can pause after any accepted module.
- We review against the benchmark. Before work starts we set the benchmark above. As each module is accepted we show you where each measure stands, and the filled-in benchmark is yours at the end.
The Review is free. Every paid step is fixed price, agreed at scoping. Stop after any step. The findings are yours. The assessment fee is credited in full against the first build stage if it starts within 90 days of the report.
This has a price. It adds work and cost before the replacement starts, and what it finds is usually unwelcome. A description of the data corrects nothing: the records are still to be measured and corrected by your own people, or by the firm that moves the data. We do not clean, correct or reconcile records, and we do not count stock. Where the new system is a packaged shop platform or packaged ERP product, we do not install or configure it, and our part ends with the description. Errors that begin on the shop floor are untouched, and nothing here blames the people who work there. We have no published case of this work for a retailer.
At the end you hold the specification, the risk register, the traceability matrix for every module we rebuilt, and the filled-in benchmark. Choosing the new system and the firm that moves the data is your decision.
What changes for the business:
| Role | What they gain |
|---|---|
| CEO | A replacement is approved knowing what data it will start with. |
| CTO or CIO | A written description of the data for whoever does the move: what each field means and where it cannot be relied on. |
| CFO | The cost of putting the data right is in the plan before the contract is signed. |
| COO | The first weeks on a new system are not spent correcting records by hand. |
| Head of marketing | Customer history and product details arrive in the new system in a state a campaign can use. |
The cases in this article, and what IOanyT does about that kind of problem:
| The case | What IOanyT does |
|---|---|
| After a large retailer’s packaged ERP roll-out in Canada in 2013, “only about 30% of the data in the system was actually correct”. | Our part is far narrower than this case. We do not implement packaged ERP products or move data, and we do not say why that data was wrong. For systems a retailer built or had built, we read the code and write down what its data means and where it cannot be relied on, before a move is planned. |
IOanyT is a consultancy with its own engineering team: we study the problem, recommend, and prove it by building only what you approve. The new system should start with data someone has described, not data someone has merely counted.