When the Busiest Day Brings the System Down

1. At a Glance

Issue study: a public case, not an IOanyT client. None of the organisations named here was an IOanyT client, and IOanyT had no part in the events described. Everything about them comes from the public record.

Organisation type	A large UK bank whose outages fell on payday, set beside nine UK banks’ outage record and two Indian banks restricted by the RBI after repeated outages
Size band	All large
Region	United Kingdom and India
The decision at stake	What to do about a system that copes on an ordinary day and fails on the busiest one
The public outcome	In the UK, 56% of one bank’s online payments failed over three days that fell on payday, with compensation expected of £5 million to £7.5 million. In India, the regulator answered repeated outages at two banks with limits on new business
Why it matters to a mid-size lender	A lender’s busiest days are its growth days. An old system that fails on them costs the days the business was planned around

2. The Business Problem, in the Buyer’s Words

“Every month end we add servers and put the team on standby, and every few months something we were not watching gives way. Each incident is fixed and closed. After the last one the board asked for options, and we brought two: replace the whole platform, or buy more hardware. They approved the hardware again. Nobody can tell me which part will fail next.”

These words are imagined. They describe the decision as the research shows it, not a real conversation.

The firm is caught between a fix that is too small to reach the cause and one too large to start. Meanwhile its busiest days keep testing the system on its customers.

3. What We Found

Method. We went through the public record on IT outages in banking: the figures UK banks have reported on their outages, and the RBI’s enforcement actions as reported in the press, from 2020 to 2025.

The payday outages. The January and February 2025 outages at a UK bank fell on payday. The record describes “severe degradation” of its mainframe processing.

What	Figure
When	The January and February 2025 outages, which fell on payday
What the record describes	”Severe degradation” of the bank’s mainframe processing
Effect on customers	”56% of online payments failing over a three-day period”
Compensation	Expected to be “between £5 million and £7.5 million”

It is not only one bank. Nine UK banks reported “803 hours of outages between January 2023 and February 2025, spanning at least 158 separate incidents”. Each incident would have been fixed and closed on its own. Together they form a record.

How the RBI has answered repeated outages.

• Kotak Mahindra Bank. On 24 April 2024 the RBI barred the bank from taking new customers through its online and mobile channels and from issuing new credit cards. Among the reasons it cited were IT inventory, patch and change management, vendor risk and “frequent and significant outages”. The restrictions lasted about ten months.
• HDFC Bank. After two years of outages, the RBI ordered the bank on 2 December 2020 to stop new digital launches and stop sourcing new credit card customers. An older case, used here as background.

In both cases the remedy fell on new business. The cost of repeated outages was counted in growth the bank was not allowed to pursue.

Our reading of why old systems fail at peak. These four are our reasoning from the cases above. None is a regulator’s finding, and none is a diagnosis of any named bank.

1. The system was sized for another time. It now also serves channels that did not exist when it was built.
2. The weak part is hidden on ordinary days. There is room to spare most of the month, so nobody can see which part will give way first.
3. One part holds up the rest. When a shared part runs out of room, everything waiting on it stops, and the whole system appears to fail.
4. Each outage is fixed and the cause stays. The fix restores service; it does not move the limit, so the next peak finds it again.

4. The Options a Firm in This Position Had

Option	When it fits	What it risks	Our view
Add hardware before each peak	The servers really are the limit	When the limit is one table, queue or job, the same part gives way at a slightly higher load	Worth checking first; often not the limit
Restore service fast and close each incident	Peak failures are rare and unrelated	The pattern stays hidden and the record grows until someone else counts it	Necessary on the day; not a plan
Replace the whole platform	The system is small, or failing everywhere	Years of work and risk for functions that were never failing; the large option often never starts	We would advise against this when only a few parts are failing
Find the part that gives way first, and move it first	A few shared parts of an old system run out of room under load	The join between new part and old system must be built with care, and a second weak part will follow	Our recommendation for the systems around the core

5. Our Recommendation, and the Decision It Asks of the Board

For the systems a lender has built around its core (loan origination, collections, agent and branch portals, reporting): lay the incidents of the past two years beside a map of the system, find the part that gives way first, and move that part first, one module at a time, while the old system keeps running. Then the next. Confirming the exact load at which a part fails takes a test on a copy of the system, which is more work and needs the firm’s own team. Until each weak part has moved, the firm’s current team or vendor keeps the system running through each peak.

6. Benchmark: Where You Are Now, and Where You Should Be

These are targets, not results. In an engagement the starting points are replaced by the firm’s own figures, taken in the first review.

Measure	Typical starting point	Where it should be	How it is checked
Peak-day incidents with a root cause	”Under investigation”, then closed	Every one with a cause, traced to a part of the system	The incident record beside the map of the system
The part known to fail first	Found on the day it fails	Named, with what depends on it, and first in the staged plan	The risk register and the staged plan
Recovery rehearsed before each peak	Staff on standby, nothing rehearsed	Rehearsed by the firm’s own team before each payday, month end and festival season	The firm’s rehearsal record
The moved module tested at peak volume	Not tested before customers use it	Tested at the volume of the busiest day before it is accepted	The acceptance record for each module
Repeat incidents	The same weakness under a new name	None from a part that has moved	Reported at every review

7. How IOanyT Would Work

1. We listen: a free Reliability Review. A first conversation about the system, the outages of the past two years and the days they fell on. You leave with a short written note of what we found.
2. We research. We research your sector, the regulator’s position and what the public record says about firms in your situation. This study is an example of that work.
3. We read the system and propose: the Legacy System Specification Assessment. It produces a map of the system as it stands, a written specification of what it does, a risk register, a cost comparison between keeping and replacing the system, and a staged plan with a fixed quote. AI does a first pass, our senior engineers review it and trace it to the code, and your own experts confirm it. Code access is read-only and logged, never used for training, and deleted within 30 days of the evaluation ending unless we engage.
4. If you approve, we build. We replace a system one module at a time while the old one keeps running, starting with the part that gives way first. We never do a single cutover. A traceability matrix links every old function to its replacement. The work is billed by milestone, each milestone tied to a module you accept, and you can pause after any accepted module.
5. We review against the benchmark. Before work starts we set the benchmark. After each module we show you where each measure stands, and the filled-in benchmark is yours at the end.

The Review is free. Every paid step is fixed price, agreed at scoping. Stop after any step. The findings are yours. The assessment fee is credited in full against the first build stage if it starts within 90 days of the report.

8. What the Client Would Own

• The map and the specification: what the system does and what depends on what, in writing and portable to any supplier.
• The risk register: the weak parts, with what depends on each, listed before any price is set.
• The traceability matrix: every old function linked to its replacement, with how it was checked.
• The code: the new modules are the client’s. No runtime or platform fee.
• The filled-in benchmark: each measure, where it started and where it stands.

9. What This Approach Does Not Cover

• IOanyT does not monitor, tune or run an on-premise core banking system or a mainframe, and does not run the old system through a peak. The client’s current team or vendor does that, including rehearsing recovery.
• IOanyT does not state that its assessment includes a load test, or that it identifies the load at which a part fails.
• IOanyT does not implement packaged core banking products, and does not work on mainframe platforms.
• Where a moved part could run in the cloud, that applies only to systems the firm is allowed to run there.
• Taking a plan to the regulator is the firm’s own job. Our assessment gives it the evidence to do so.
• This study does not say IOanyT would have prevented the events described, or that IOanyT has done this work for a bank.

10. Sources

Source	Used for
The Global Treasurer, 6 March 2025	Nine UK banks’ outage hours and incidents; one UK bank’s payday outages and compensation
TechCrunch, 24 April 2024; News On AIR; RBI press release, 24 April 2024; RBI press release, 12 February 2025	Kotak Mahindra Bank restrictions, the reasons cited and their end
Outlook India	HDFC Bank order of 2 December 2020

IOanyT is a consultancy with its own engineering team: we study the problem, recommend, and prove it by building only what you approve. The busiest day should be the one the business was planned for, not the one it fears.

Book a free Reliability Review →

Get a fixed quote →

See the Legacy System Specification Assessment →