Keeping an AI system honest

Problem

AI agents sound confident. Some of what they say is wrong. I saw three patterns:

• Drift. A fact changes, but the old version stays in another file. Later, someone trusts it.
• False claims. “Fixed”, “restarted” or “I can’t do that” without a check behind it.
• Over-engineering. Full planning and audit rounds for a two-line change.

What I built and changed

• Drift log. I logged each drift incident by type. A script counts them, and refuses to print a number if the log is malformed.
• Single source for counts. Each number lives in one place. Other files link to it, so they can’t go stale.
• Claim rules. No “fixed” without a check that was actually run. Every work report names its evidence.
• Stopping rules. Three rules with no code:
  1. Three rounds with nothing changing means stop.
  2. A second rejection of the same work from a blind cross-model review means stop.
  3. Before building a control, estimate the saving. If the saving is smaller than the cost, don’t build it.
• Less background work. I turned off a daily self-maintenance loop.

How it’s checked

• A second auditor on a different model family reviews key decisions blind.
• Tests are set up before the run, with a pass mark agreed in advance.

Results

• Drift: 38 incidents in 8 types, script-counted (log covers 6–8 September 2026). The costliest type: a fix landed on one copy of a fact but not the others. The 9 “never checked” claims each fell apart once someone ran the check.
• Over-engineering: I dropped a planned enforcement tool before deploying it. Measured against three real cases (September 2026), it would have prevented 4 of their 27 dispatches. Shipping it would have cost 5 to 8 more. Shipped instead: the three rules above.
• Slowness, measured: the same audit role ran about 7 times slower on the new model family than on the old one. Median time in the same week was 30 seconds on the old family (18 runs) and 220 seconds on the new one (45 runs) (September 2026).
• A tuning test: lowering reasoning effort cut median time from 163 to 129 seconds (5 runs each, one fixed task, September 2026). That missed the pass mark set in advance, and its effect on quality was unmeasured, so I reverted it.
• A correction: I first reported about 10% of runs killed by a timeout. A recount found one confirmed case in 96 runs (14–20 September 2026), about 1%. I corrected it.

What’s still open

• No drift checks are built yet. Four of the eight types have a designed check; the other four have no proposed check at all.
• Four drift incidents are open with no owner (as of 8 September 2026).
• Output quality by model is not measured. Quality claims are asserted, not tested.
• Capability claims still slip. Saying “I can’t” when a working path exists has come back before.