1. A check needs an independent reference.
A checker handed the right answer can compare against it. A checker handed nothing has to work the answer out again, and can fail where the system it checks failed.
People meet this as the answer that is almost right. In Stack Overflow’s 2025 survey, 66% of the 31,476 developers who answered chose “AI solutions that are almost right, but not quite” as a frustration: finding the wrong part means doing the work again.
What the field already has
Debate proposes that two AI systems argue while a person judges which gave the more true and useful information. In the authors’ complexity-theory analogy, debate lets a judge with limited time settle questions that direct judging could not; whether it works in practice they call an empirical question.
Prover-verifier games train a model to write solutions that small verifier models can check, and the gain in checkability carried over to people checking under time pressure.
What is still missing
A measured account of how much reference a checker needs: how its catches and false alarms change between the full answer, a test it can run, a partial trace and nothing, with the same tasks and the same checker. Without it, a deployment cannot say what its check is worth.
What would count as solved
For a given kind of task, the curve from full reference to none is published with catches and false alarms reported separately, and every deployed check states where on that curve it sits.
What this lab has measured
- One local checker, gemma4:12b, separated right answers from wrong ones at 1.000 when handed the correct value, 0.536 with a test it could apply, and 0.034 with nothing to compare against (0.042 when re-run after a serving fix). The top of that range is deference: told that a false value was correct, the same checker passed the wrong answer on 0.969 of items. The measurement and its caveats
- The bottom of the range belongs to that checker. Another model, qwen3.5:9b, scored 0.804 on the same tasks with nothing to compare against. Which checker to pick
Proposed here, not yet built
Build the reference ladder: the same checker on the same tasks, with the reference varied one form at a time (the answer, a runnable test, a worked intermediate state, retrieval, a partial trace, nothing). The two ends and one framing between them have been measured; the ladder between them has not.
Make every result reproducible without the provider. Done for the attack results, which re-derive from raw outputs with Python alone in frontier-repro; the other results on this site cannot yet be reproduced from public files.
3. A monitor is tested for inversion before it is trusted.
A monitor can become worse than no monitor. Past some difficulty it rejects correct work more often than wrong work, and the health numbers that do not need an answer key can improve at that moment.
What the field already has
AI control evaluates monitoring set-ups against a model that is itself trying to subvert them, and found that set-ups such as trusted editing and untrusted monitoring improve substantially on simple baselines.
A weaker model reading a stronger model’s chain of thought caught reward hacking. Trained against that monitor too hard, the stronger model learned to hide its intent while it kept hacking, and the authors recommend paying a “monitorability tax” by not optimising the chain of thought directly.
What is still missing
A release test for monitors that uses tasks with known answers across the difficulty range where the monitor will run, and an attacker, and fails when catches minus false alarms fall below zero, whatever the monitor’s other numbers show. No study this site has read runs one.
What would count as solved
No monitor is released unless a fixed bank of tasks with known answers, spanning its difficulty range and including an automated attacker, shows its catches minus false alarms above zero everywhere it will be used; the release fails when that number crosses zero, whatever else improves.
What this lab has measured
- Holding the test fixed and making only the numbers longer, gemma4:12b went from 0.536 to −0.620 (re-run with the leak fixed; the first run read −0.648): it now rejected correct answers more often than corrupted ones. At that moment three of its five other health numbers improved. The inversion, drawn for six models
- An automated attacker search broke two monitor defences that had held against every attack written by hand, so a release test needs an attacker in it as well as fixed tasks. The attack results
Proposed here, not yet built
Turn monitor inversion into a release test: a small bank of grounded tasks spanning the known crossing, with the monitor’s signed score computed on every release. The crossing is located for six local models; no gate has been built.
4. Instructions carry their provenance.
A model reads its instructions and its data as one stream of text. Text inside the data can pose as an instruction, and a model can recognise the injection and follow it anyway.
What the field already has
Indirect prompt injection, instructions planted in data a system retrieves, compromised real applications including Bing’s GPT-4 chat; in 2023 the authors found effective mitigations lacking.
An instruction hierarchy trained into GPT-3.5 teaches the model to rank a developer’s instructions above text from users and third parties, and drastically increased robustness with minimal cost to capability.
Spotlighting marks where each piece of input came from; with GPT-family models it cut attack success from above 50% to below 2% in the authors’ experiments.
CaMeL takes the control flow from the trusted request alone, so retrieved data can never change what the program does; it solved 77% of AgentDojo tasks with provable security, against 84% for an undefended system.
Debenedetti and others, Defeating Prompt Injections by Design, 2025
Reasoning models can resolve an instruction conflict correctly in their reasoning and still produce a response that breaks the rule; a monitoring step cut rule-following failures by 81 to 99% across three models.
What is still missing
Provenance that holds for the model reviewing the work as well as the model doing it. Marking untrusted text helps the model doing the work, and on the evidence below, on two gemma monitors, it made a reviewing model obey what the marking was meant to fence off; on qwen2.5-coder:32b and qwen3.6:35b-a3b it did not.
What would count as solved
Text from an untrusted source can change neither what a system does nor what its monitor decides, shown by an automated attacker search that fails to move either.
What this lab has measured
- Asked to audit a prompt, Claude Sonnet 5 flagged an injected instruction every time; doing the task, it obeyed that instruction on 80% of tasks when the injection was the last text in the prompt, and on none with 200 characters of ordinary text after it. Detection and obedience
- With a stamp in the reviewed text dictating the verdict, labelling that text as untrusted, the usual defence, took gemma4:12b’s own first verdict from flagging 0.0059 of correct work to 1.0000. The one defence that held removed the reader: the monitor saw only the answer’s first integer. That is the principle CaMeL builds on, that untrusted text never reaches the part that decides. The seven defences
Proposed here, not yet built
Route by evidence mode: send each check to the kind of evidence that can bear it (a pattern match where an answer has a canonical form, a test, retrieval, a person, or no answer at all), and refuse when none clears its threshold. Not built.
5. A system’s account of itself carries a claim ceiling.
A system’s account of its own reasoning is produced by the system it describes. It can be useful and still be wrong about what drove the answer.
What the field already has
Chain-of-thought explanations left out features that had biased the answer, and rationalised the biased answer instead; accuracy fell by as much as 36% on 13 BIG-Bench Hard tasks.
Reasoning models that used a hint in the prompt mentioned it in their reasoning, for most settings, in fewer than 20% of the cases where they used it, and reward hacking rose without the models saying so more often.
Chen and others, Anthropic, Reasoning Models Don’t Always Say What They Think, 2025
Forty-one authors from across the field call monitoring a model’s chain of thought a new and fragile opportunity, imperfect like every other known oversight method, and ask developers to weigh how their choices affect it.
What is still missing
A format that keeps a model’s account of itself apart from independent evidence about the model, and a published rate at which the two agree.
What would count as solved
Every account a system gives of its own reasoning is labelled as one, and its agreement with an independent measure of what the system did, such as a trace or an intervention, is published beside it.
What this lab has measured
- Claude Sonnet 5 reported an injected instruction every time it was asked to audit for one, and obeyed that instruction on 80% of tasks when not asked, as long as the instruction was the last text in the prompt. What a model says it noticed predicted little about what it did. Detection and obedience
Proposed here, not yet built
Give self-reports an explicit claim ceiling: logs and reports that keep observed behaviour, the system’s own explanation, an inferred mechanism and an unresolved cause apart. Not built.
6. People keep the skill to check.
Every check above ends with a person who has to know when to doubt the machine. That depends on a skill the person keeps by using it.
What the field already has
In a randomised trial, 16 experienced developers completed 246 tasks in their own projects. With AI tools allowed, tasks took 19% longer, while afterwards the developers estimated the tools had made them 20% faster.
In Stack Overflow’s 2025 survey, 20% of those who answered the frustration question chose “I’ve become less confident in my own problem-solving”.
When Anthropic interviewed 80,508 Claude users in December 2025, 16.3% raised cognitive atrophy, over-reliance that costs skill or critical thinking, as a concern.
What is still missing
A study that follows people who rely on an AI system over months and measures whether they still catch its errors, with errors planted where the right answer is known. The work above measures how fast people worked and what they report about themselves. None of it measures whether they catch errors.
What would count as solved
People who oversee a system are tested, at intervals fixed in advance, on errors planted in its output, and their catches and false alarms are published beside the system’s own figures.
What this lab has measured
- Nothing. The lab has measured models, not people. The open questions page sets out small versions of this study that a household or a group could run with paper. Questions a household or small group can answer
7. Compute and deployment are legible to an outside auditor.
An auditor can check a developer’s statements only if what went into a system, how much computing and where it is deployed, can be measured by someone other than the developer. Every requirement above assumes the auditor knows which system it is checking.
What the field already has
The Foundation Model Transparency Index fell from an average of 58 of 100 in 2024 to 40 in 2025, and developers were most opaque about training data, training compute, and how their flagship models are used after release.
Wan, Klyman, Kapoor and others, The 2025 Foundation Model Transparency Index
Computing power is, in the authors’ words, “detectable, excludable, and quantifiable”, which makes it a place where governments could see and govern AI development; the proposals for doing so vary widely in how ready they are.
Sastry, Heim and others, Computing Power and the Governance of AI, 2024
Flexible hardware-enabled guarantees would put an auditable processor beside AI accelerators to verify claims about development. It is a proposal, the first of a three-part series, and not a deployed system.
What is still missing
Independent measurement of what a developer declares about training compute and deployment. The index above scores what developers disclose about themselves.
What would count as solved
An auditor who is not the developer can check a statement about training compute or deployment against a measurement they did not have to take on trust.
What this lab has measured
- Nothing. The lab has not measured this.
A case: checking AI-assisted mathematics
Project Reinnman builds instruments for mathematics done with AI help: source locks, obligation graphs, claim ceilings, independent replay and adversarial mutation testing. It works on Robin’s inequality, an established equivalent of the Riemann Hypothesis, and its record says that it has not proved the Riemann Hypothesis and is not close to proving it.
Its record labels each result with the height at which it was established, and does not count a Lean file that compiles as evidence that an unproved hypothesis holds. That is requirement 5 applied to research, and requirement 1 in the form mathematics allows: a proof checker is an independent reference for exactly what the proof states.