From Policy Document to Rules You Own

The underwriting guide is long. It was last revised in the spring. Every underwriter has read it, and every underwriter applies it a little differently, because some clauses say “generally” and some say “at the underwriter’s discretion”.

Now the firm wants AI to help. The first proposal is to give a model the guide and let it answer questions about cases. The demo goes well. Then someone asks which clause the model’s answer came from, and whether it would give the same answer next week.

The paradox: The policy is written down, and still nobody can say exactly what it decides.

What We Found

Before we recommend anything about turning a policy into decision logic, we look at what the public record says about where a decision’s authority comes from. For this article we went back over a tribunal decision on an AI answer about a policy, and three sets of rules we have read in full: a US credit notice rule, US bank supervisors’ model risk guidance, and an insurance regulators’ model bulletin.

Four findings stand out.

An AI answer about a policy can state a policy that is not the policy. In February 2024 a Canadian tribunal decided Moffatt v. Air Canada. The airline’s website chatbot had told a customer he could book at the normal fare and apply for a bereavement discount afterwards. That was not the airline’s policy. The tribunal held the airline responsible: “it makes no difference whether the information comes from a static page or a chatbot.”

Some rules require the decision to know its own reasons. Under the US credit notice rule, Regulation B, the reasons given for declining credit “must relate to and accurately describe the factors actually considered or scored by a creditor”, according to its official commentary. A decision that applies written rules knows which factors it used. A decision made by reading a document and generating an answer may not.

Fixed rules are governed differently from models. The US bank supervisors’ revised model risk guidance (SR 26-2, April 2026) defines a model as a quantitative method applying statistical, economic or financial theories, and excludes “deterministic rule-based processes and software where there are no statistical, economic, or financial theories underpinning their design or use”. Rules written from a policy are governed as policy and software, not as models.

Insurers are expected to describe how their AI is governed. The US insurance regulators’ model bulletin, adopted by 29 jurisdictions as of 8 October 2026, expects insurers to keep a written AI systems program covering governance, risk management and internal controls. A decision traced to written rules is easier to describe in such a program than one generated from a document.

What our review does not show: None of these sources studies a policy-to-rules project, and the tribunal case concerns a chatbot, not a rules system. The three sets of rules apply to particular firms in particular places, and nothing here is legal advice. We use them to show why the source of a decision matters, not to say what any firm must do.

The Policy Nobody Can Run

Our reading is that most firms’ policies are written for people to interpret, and are now being handed to systems that interpret differently. A person applying an underwriting guide uses judgement and experience. A model answering from the same guide produces a fluent answer that may or may not match either the guide or the person. Neither leaves a record of which clause decided the case.

Rules written from the policy change that. Each rule says what it decides and which clause it comes from. Where a clause is genuinely a matter of judgement, the rule says so and hands the case to a person. The result is a policy that can be run the same way every time, tested against cases your experts have already decided, and changed on purpose when the policy changes.

The gap we see: many firms want AI to help apply their policies, and start by giving a model the document. Firms that turn the policy into rules first, clause by clause, get decisions they can explain to a customer, an examiner or an auditor, and an AI that helps with reading and drafting without deciding what the policy means.

This is our reading of the evidence. It is not a measured market figure.

Four Ways to Put a Policy to Work, Weighed

Option	When it fits	What it risks
People apply the document	Low volume, decisions that are mostly judgement	Different people decide the same case differently; no record of which clause applied
A model answers from the document	Internal questions where a person reviews every answer	Answers that are not the policy, or that change between runs
Rules configured in a vendor’s product	The policy matches what the product already models	The rules may live in the vendor’s format; tracing them to your clauses depends on the vendor
Rules written from your policy, each traced to its clause, confirmed by your policy owners, tested on decided cases, and owned by you	Volume, stakes and a policy clear enough to write down	Up-front work; ambiguous clauses have to be settled or handed to people

Our recommendation: turn the parts of the policy that decide cases into rules traced to their clauses, confirmed by the people who own the policy and tested on cases they have already decided. Keep judgement clauses with people, by design. Let AI help read cases and draft letters, not decide what the policy says.

Where You Are Now, and Where You Should Be

Measure	Typical starting point	Where it should be	How it is checked
Where the policy lives	A document people interpret	Rules, each traced to the clause it comes from	A trace from each rule to its clause
Who confirmed the rules	Nobody, or one expert	The policy’s owners confirm each rule	Sign-off records
How rules are tested	Not tested	Against cases your own experts have decided	A test set with results
Judgement clauses	Mixed in with everything else	Listed, and handed to a person by design	The rules, and the hand-off record
When the policy changes	The document is updated; practice catches up	The affected rules change, are re-tested and released as a new version	The change log

The starting points above are what we expect to find. In an engagement they are replaced by your own figures, taken in the first review. The targets are what we would work towards with you. They are not results we have already achieved for a client.

How We Work With You

1. We listen: a free Reliability Review. A first conversation about the policy, how it is applied today and where AI is being considered. You leave with a short written note of what we found.
2. We research. We research your sector, the regulator’s position and what the public record says about firms in your situation. This article is an example of that work.
3. We diagnose: the Workflow Diagnostic. It sorts each step of the workflow into compile, keep on an AI model, or don’t automate, and includes a policy-to-rules map, a 100-rerun consistency test of your current pilot, the evidence gaps, the cost per case today against projected, and a change plan. It may recommend another firm or tool, or not automating at all. We take no referral fees on model or platform recommendations.
4. If you approve, we prove it: a Proof Sprint. One workflow, on your own cases, measured for consistency and for accuracy against your own experts. You keep the code.
5. We review against the benchmark. Before work starts we set the benchmark. At each stage we show you where each measure stands, and the filled-in benchmark is yours at the end.

In policy-to-rules work, we turn a written policy into executable rules, each traced to the clause it comes from. AI does a first pass, our senior engineers review it, and your own policy owners confirm each rule. Each rule ships with tests built from cases your own experts have decided. The rules are delivered as code and documentation you own, with no IOanyT runtime or platform fee.

The Review is free. Every paid step is fixed price, agreed at scoping. Stop after any step. The findings are yours. The Diagnostic fee is credited against a Proof Sprint within 90 days.

What changes for the business:

Role	What they gain
CRO or Head of Compliance	Every decision traces to a rule, and every rule to the clause of the policy it comes from.
COO	Teams apply the policy the same way, and the cases that need judgement reach a person by design.
CTO or CIO	Policy as versioned, tested code, changed on purpose when the policy changes.
Head of AI or Data	AI helps read and draft, while the policy itself is decided by rules the business has confirmed.
CEO	A policy the firm can explain to a customer, an examiner or the board, and that belongs to the firm.

The cases in this article, and what IOanyT does about that kind of problem:

The case	What IOanyT does
In Moffatt v. Air Canada, a chatbot stated a policy that was not the airline’s policy.	Policy answers come from rules confirmed by the policy’s owners and traced to their clauses, not from a model’s reading of the document.

IOanyT is a consultancy with its own engineering team: we study the problem, recommend, and prove it by building only what you approve. The rules are yours, whoever runs them.

Book a free Reliability Review →

Get a fixed quote →

See the Workflow Diagnostic →