Your System Applies Its Rules Every Day. Nobody Can Say What They Are.
We reviewed the public record on heavily customised systems in financial services. Here is what we found, what unwritten rules hold back, the options, and the benchmark we would set.
The auditor asks a simple question. How is penal interest on an overdue instalment calculated?
Finance says the system does it. Technology says it was set up years ago. The product note has one sentence on it, written when the product was launched. The developer who made the later changes has left.
After two days of looking, the answer that goes back to the auditor is, in effect, whatever the system does.
The paradox: The firm applies its rules to every customer, every day, and cannot say what they are.
What We Found
For this article we went through the public record on old, heavily customised systems in financial services: a vendor’s published case study of one replacement, and the UK regulators’ notices on one failed migration. The record on this issue is thin, so we say which findings are our own reasoning.
A customised system holds more than anyone can list. A group of credit unions shared one heavily customised core system. Replacing it meant rebuilding hundreds of screens and hundreds of interfaces to other systems. Every screen and every interface carries rules of its own.
A rule is changed many times, and its document once. This is our reasoning, not a finding from a named case. Take a penalty rule, as an illustration, not taken from a real system. At launch it is a fixed monthly rate. In year 3 a grace period is added. In year 5 one employer’s salary scheme is exempted, through a table of codes. In year 7 rounding is added inside the database. In year 9 a holiday rule goes into the overnight job. In year 12 a cap is added by a developer who has since left. By year 15 the product note still shows one sentence, and the rule has six parts in four places.
The rule is not always where people look. This is also our reasoning. Parts of a rule can sit in the database, in small programs called stored procedures; in jobs that run overnight or at month end; in tables of codes and rates; or outside the system, in a spreadsheet one person adjusts every month. A record of the screens holds only part of the rule.
Go-live is when a difference reaches customers. In April 2018, TSB Bank in the UK moved its customers to a new IT platform, and the migration failed. Between 22 April 2018 and 7 April 2019 it received 225,492 complaints. The regulators’ finding, in December 2022, was about how TSB managed its outsourcing arrangement with a critical supplier, not about unwritten rules, and we do not say otherwise. We use it for one point: when a new system behaves differently, customers notice first.
What our review does not show: This is the thinnest public record of any banking issue we have studied. We found almost no public case that names unwritten rules as the cause of a failure. The credit union figures come from a vendor’s own case study; they show how much a customised system holds, not what went wrong. The second and third findings are our reasoning. Both named cases are from outside India, and we do not say any mid-size lender has had the same events.
What the Firm Cannot Do While Its Rules Are Unwritten
Unwritten rules are usually treated as a documentation problem. Our reading is that they are a business problem, because they set the pace of every change.
It cannot change its products quickly. A fee change or a new product variant should start from the rule as it stands. When the rule is unwritten, the change starts with a search through the code and goes in without anyone seeing the other parts. Small changes take weeks, and the business learns to ask for fewer.
It cannot explain its own figures. When an auditor asks how penal interest is calculated, or a customer asks why a charge is what it is, the firm has nothing to point to. Finance signs for figures it cannot inspect, and nobody can say whether the system applies a rule as intended.
The gap we see: many lenders want to change their products more often and to answer questions about their figures from a page. Few can, while the only complete record of their rules is the system itself. A lender that writes its rules down once can do both, whether it later keeps, changes or replaces the system.
This is our reading of the evidence. It is not a measured market figure.
Four Ways to Write the Rules Down, Weighed
| Option | When it fits | What it risks |
|---|---|---|
| Interview the staff and write up what they say | The system is small, and the people who built it are still there | It records what people remember. The missing rules are the ones nobody remembers |
| Leave it alone | The system is stable, rarely changed and due to retire | Every change is made blind, and any replacement is specified from memory |
| Re-specify from the old requirements documents | The documents were kept up to date | They describe the system at launch, not the years of changes since |
| Take the specification from the system itself, then the business decides each rule | The system is old, customised and poorly documented | Time passes before anything visible changes, and it turns up rules nobody can explain |
Our recommendation: take the specification from the system itself, then put each rule in front of the business. Write what the system does in plain words, and list every rule nobody can explain for the business to decide. Interviews can tell you why a rule exists. They cannot tell you what the system does.
Where You Are Now, and Where You Should Be
Before any work begins we set a benchmark: a few measures taken from your own system, each with a target.
| Measure | Typical starting point | Where it should be | How it is checked |
|---|---|---|---|
| Rules with a written line | One sentence per product, from launch | Every rule in scope has its own numbered line, in plain words | The specification, confirmed by your own experts |
| Rules traced to the code | None; a rule is found only when someone goes looking | Each line points to where the code carries it out | Our senior engineers’ review, line by line |
| Rules nobody can explain | Left in place, often under a “do not change” note | Each one listed, with the business’s decision recorded on its line | The decision, recorded against the line |
| Figures that can be explained | ”The system calculated it” | A charge, fee or interest figure can be explained from a written rule | Figures from recent statements, explained from the record |
The starting points above are what we expect to find. In an engagement they are replaced by your own figures, taken in the first review. The targets are what we would work towards with you. They are not results we have already achieved for a lender.
How We Work With You
- We listen: a free Reliability Review. A first conversation about the system, the figures nobody can explain, and the changes that keep waiting. You leave with a short written note of what we found.
- We research. We research your sector, the regulator’s position and what the public record says about firms in your situation. This article is an example of that work.
- We read the system and propose: the Legacy System Specification Assessment. It produces a written specification of what the system does, a risk register, a cost comparison between keeping and replacing the system, and a staged plan with a fixed quote. AI does a first pass, our senior engineers review it and trace it to the code, and your own experts confirm it. Code access is read-only and logged, never used for training, and deleted within 30 days of the evaluation ending unless we engage. Each rule nobody can explain is listed for the business to decide. You get the options and our recommendation.
- If you approve, we build. Where the business decides to replace a module, we build it with each rule as the business decided it. We replace a system one module at a time while the old one keeps running. We never do a single cutover. Each module has a traceability matrix. The work is billed by milestone, each milestone tied to a module you accept, and you can pause after any accepted module. Changing a rule inside the old system stays with your current team or vendor.
- We review against the benchmark. Before work starts we set the benchmark. After each module we show you where each measure stands, and the filled-in benchmark is yours at the end.
The Review is free. Every paid step is fixed price, agreed at scoping. Stop after any step. The findings are yours. The assessment fee is credited in full against the first build stage if it starts within 90 days of the report.
What changes for the business:
| Role | What they gain |
|---|---|
| CEO | The firm can explain how its own products behave, to its board, its regulator and its customers. |
| CTO or CIO | Changes are made knowing which rule they touch and what depends on it. |
| CFO | Interest and fee figures can be traced to a written rule. |
| COO | Staff can explain a number to a customer, and fewer disputes are passed upward. |
| Risk and compliance | The logic behind a calculation is on paper when the auditor asks. |
The cases in this article, and what IOanyT does about that kind of problem:
| The case | What IOanyT does |
|---|---|
| A group of credit unions rebuilt hundreds of screens and hundreds of interfaces when their heavily customised core was replaced. | Our assessment takes the specification from the system itself, so what a customised system does is written down before anyone rebuilds it. |
| TSB Bank received 225,492 complaints after its 2018 migration. The regulators’ finding was about how it managed a critical supplier. | Our part is narrower than that case. When we replace a module, a traceability matrix links each old function to its replacement before it goes live. Managing its suppliers is the firm’s own job. |
IOanyT is a consultancy with its own engineering team: we study the problem, recommend, and prove it by building only what you approve. The rule should be on a page before anyone changes it, replaces it or is asked about it.
Sources
- A vendor's case study
- Bank of England, 20 December 2022
- FCA Final Notice, TSB Bank plc, 2022
In this issue
Business rules buried in code
Research note 2
The Rules in Your System Were Decisions. The Business Can Make Them Again.
A rule nobody remembers is still applied to your customers. What deciding each rule again gives back, four ways to get there, and the benchmark we would set.
Research note 3
"We Documented It" Is Not the Same as Having the Rules. A Benchmark Shows the Difference.
A document of the application can still miss the rules held in the database, night jobs and spreadsheets. The benchmark we would set, and the path.
Issue study
When the System Is the Only Record of the Rules
When the system is the only record of a lender's rules: the public record, the options, what we would advise against, and the benchmark we would set.
More from Banking
Regulator restrictions and penalties
A Replacement Plan Needs Stages Someone Can Check. Here Is the Benchmark We Would Set.
Dependence on one vendor or shared provider
A Supplier Runs Your System. Here Is the Benchmark for Overseeing It.
Weak patch and change process