Almost Right, but Not Quite. Four Ways to Handle Code Your AI Tools Wrote, Weighed.

The new feature was demonstrated on Thursday and shipped on Friday. In the demonstration it was faultless.

On Monday a customer in another country finds that their dates are a day out. On Tuesday another finds that a discount of exactly one hundred per cent produces an error. On Wednesday a third finds that when two people save the same record at once, one person’s changes are lost.

None of these was in the demonstration, because nobody thought to try them. The tool that wrote the feature was not asked about them either. It answered the question it was given, and answered it well.

What We Found

For this article we went through the public record on what happens after code is written with AI tools: one large survey of developers, one large software firm’s engineering account, and one consultant’s account of repairing such code.

Three findings stand out.

Almost right is the common complaint. In one large survey of developers in 2025, about two-thirds named answers from AI tools that are almost right, but not quite, as a frustration they had met, and nearly half said debugging code written by AI tools takes longer. The same survey found that more developers distrusted the accuracy of AI tools, nearly half, than trusted it, about a third, while more than four in five were using or planning to use AI tools when they write code. Most developers use the tools, and most of them check what the tools produce. The survey is about developers in general, not about software firms alone.

Code can look right and behave differently. One large software firm, writing about its own move of old code with AI tools, found that generating code without the code it depends on gave partial rewrites that looked correct as code but behaved differently when they ran. Nothing failed to run. It simply did something else. That firm’s results are its own, and we use only the limit it states.

The cost of almost right shows up later. One consultant who sells repairs of such code quotes an anonymous founder for whom each small change cost days of debugging things that broke elsewhere. It is one founder’s experience, reported by someone who sells the cure.

Our reading is that almost-right code is the hard kind of wrong. Code that is plainly broken is fixed the same day. Code that works for the case someone tried goes out, and its faults are found later by whoever meets them first, often a customer. The tools are not careless. They answer the question they were asked, and the awkward cases are the ones nobody asks about.

What a Firm Cannot Do While Its Code Is Almost Right

A firm whose code is almost right is not standing still. It is shipping quickly. What it cannot do is change the product with confidence, or say how its code was made and checked.

Each change is a gamble. Without tests that say what the product is meant to do, nobody can tell whether a fix for one customer has broken something for another until the second customer writes in. So the team slows down where it should be fastest, or keeps its speed and lets customers do the checking.

Each question about the product is awkward. A buyer, a partner or an investor’s adviser who asks how the code was checked gets “we tried it, and it works”. What buyers ask about this was not researched; our expectation is that the question is coming, and that “it works” will not be enough.

And the speed is spent twice. The time a tool saved in writing a feature is paid back later in debugging, and the consultant’s founder put a number on it for one product.

This is our reading of the evidence. It is not a measured market figure.

Four Ways to Handle Code Your AI Tools Wrote, Weighed

There are four real options. Each one is right for somebody.

Option	When it fits	What it risks
Keep re-prompting the tools, and fix what customers report	Customers are few, close and forgiving, and the product changes weekly	Each fix is tried once, like the feature; faults reach customers first, and support pays for the testing
Rewrite it by hand	The product is small, the firm can pause new work, and the team wants to own every line	Months with nothing new, the tools’ speed given up, and a hand-written product that is not tested either unless someone writes the tests
A reviewed rescue: read it first, establish ground truth, put tests around the parts you must modify first, harden, hand back	The product sells, mostly works, and the team cannot say where it is not quite right	It costs effort before the next feature, writing tests takes far longer than the tool took to write the code, and tests cover only the cases somebody thought of
A staged rebuild, one module at a time	Parts have been patched so often that nobody can vouch for them	Old and new run side by side for a time, and it is the slowest of the four on the calendar

Where You Are Now, and Where You Should Be

A recommendation is only useful if you can tell later whether it worked. So before any work begins we set a benchmark: a few measures taken from your own product, each with a target.

For this issue, the benchmark would look like this.

Measure	Typical starting point	Where it should be	How it is checked
What each feature is meant to do, written down	Known to whoever asked the tool for it	Every feature in plain words, including its awkward cases, confirmed by your product people	The specification
Behaviour covered by tests	Few or none	The parts you must change first covered, then the rest as they are touched	The test run, shown against the specification
How a change is judged ready	”It worked in the demonstration”	The tests pass, and any difference is explained before release	The record of each release
Changes that go through the same checks	Depends on who made them and how	Every change, whoever or whatever wrote it	The record of each release
Faults found by customers after a release	Not counted	Counted, and falling	Your own support records

The starting points above are what the public record suggests and what we expect to find. In an engagement they are replaced by your own figures, taken in the first review. The targets are what we would work towards with you. They are not results we have already achieved for a software firm.

How We Work With You

1. We listen: a free Reliability Review. A first conversation about your product, the faults customers have found and how fixes are made today. You leave with a short written note of what we found. You can start with a conversation about the system; we ask for read-only code access only when the assessment begins.
2. We research. We research your sector, the regulator’s position and what the public record says about firms in your situation. This article is an example of that work.
3. We read the system and propose: the Legacy System Specification Assessment. Here the system is your own product’s code, and for code written with AI tools the reading is done as an AI Code Rescue assessment: we read it first and establish ground truth about what is tested and what is not. It produces a map of the system as it stands, a written specification of what it does, a risk register, a cost comparison between keeping and replacing the system, and a staged plan with a fixed quote. AI does a first pass, our senior engineers review it and trace it to the code, and your own experts confirm it. Code access is read-only and logged, never used for training, and deleted within 30 days of the evaluation ending unless we engage. You get the options and our recommendation.
4. If you approve, we build. Here that means an AI Code Rescue: we put tests around the parts you must modify first, harden the code, and hand back a codebase your team can own. Where a module has to be replaced, we replace a system one module at a time while the old one keeps running. We never do a single cutover. Each rebuilt module has a traceability matrix. The work is billed by milestone, each milestone tied to a module you accept, and you can pause after any accepted module.
5. We review against the benchmark. Before work starts we set the benchmark. After each module we show you where each measure stands, and the filled-in benchmark is yours at the end.

The Review is free. Every paid step is fixed price, agreed at scoping. Stop after any step. The findings are yours. The assessment fee is credited in full against the first build stage if it starts within 90 days of the report.

This approach has limits. Tests cover the cases somebody thought of, so a case nobody imagined can still get through; tests show that the product does what it is meant to in those cases, and they do not make it secure. Hardening finds what our engineers know to look for, not everything. We do not write your firm’s rules or policy for AI tools, and we do not train your staff. We do not decide or manage access to your live product; that is your decision. We do not back up, restore or recover data. We do not choose, recommend, compare or supply AI tools, and we name none. We do not deliver on-premise or self-hosted AI. We do not maintain the product or keep the tests current afterwards; your own team does. We are not a staffing firm. Our AI security assessment is a different service and is not what this article describes. We are not saying AI-written code is bad: AI does a first pass in our own work too, and our senior engineers review it. We have no published case of this work for a software firm.

What changes for the business:

Role	What they gain
CEO	The speed the tools gave the firm is kept, and customers stop being the ones who find where the code is not quite right.
CTO or CIO	Tests that say what the product is meant to do, so a change is checked before it goes out, whoever or whatever wrote it.
Head of product	A fix for one customer stops breaking something for another.
CISO	The code has been looked over by senior engineers for what fast-written code tends to leave out, before anyone else finds it.
Head of people	Engineers use AI tools with a safety net under them, and are judged on work that has been checked.

The cases in this article, and what IOanyT does about that kind of problem:

The case	What IOanyT does
This article names no organisation. Its figures come from one large survey of developers, one large software firm’s engineering account and one consultant’s account of a repair.	We put tests around the parts of your product you must change first, harden the code and hand it back, so that every later change, from a person or a tool, can be checked. We do not choose your tools, write your rules or maintain the product afterwards.

IOanyT is a consultancy with its own engineering team: we study the problem, recommend, and prove it by building only what you approve. We keep the speed your tools gave you and add the check, so that almost right becomes known to be right in the cases that matter.

Book a free Reliability Review →

Get a fixed quote →

See the Legacy System Specification Assessment →