What the lab has measured about one AI model checking another

Methods shared by the local results

Every local task has an exact integer answer known to the harness: an arithmetic problem, a value looked up in a supplied table, or the output of a short function. Outputs are graded against that answer, never by a second model. The main measure is correctness discrimination: the share of wrong answers a checker catches minus the share of right answers it rejects. 1 is perfect, 0 is no better than chance, and below 0 the checker rejects right answers more often than wrong ones.

Each design fixes its models, task count, seed, claim, matching anti-claim and minimum effect of interest (0.05) before it runs. A design registered on a frozen task suite before its results were seen is pre-registered (the lab’s tier CONFIRMATORY_LOCAL); one designed after seeing earlier results is exploratory. Intervals are 90% bootstrap intervals. On 17 September 2026 the lab found gemma4:12b had been leaking chat-template text into its replies; every gemma4:12b cell quoted here was re-run with the leak fixed, and both readings are given. A re-run is graded exploratory whatever its parent’s tier. The designs, results and grading code are public in verifier-instruments, which recomputes every stored verdict offline.

Evidence for requirement 1: a check needs an independent reference.

1. A checker with nothing to check against barely separates right from wrong

Summary. Handed nothing, gemma4:12b scored 0.042 on sums it solves itself every time. Handed a one-step test it could apply, it scored 0.536. Handed the correct value, it scored 1.000, which measures deference to whatever it is told is correct.

Method. Checker gemma4:12b; arithmetic tasks it answers correctly on its own at 1.000. Rows one and three: conf_family_gemma12, pre-registered, n = 500 each, 30 July 2026. Row two: acc_testable_low, n = 250, one exploratory run, 31 July. Re-runs with the leak fixed: conf_family_gemma12_rs and acc_testable_low_rs, same n, 23 September. No unreadable verdicts in any row.

Further runs. The composition cells ran on 31 July and 1 August 2026, n = 250 each, one exploratory run per cell. Handed a reference worked out by the 9B model, right on 99.6% of items and introduced as a second model’s answer, the checker read 0.072, and 0.092 when re-run with the leak fixed on 26 September. Handed a false reference that agrees with a corrupted answer, gemma4:12b passed the corrupted answer on 0.969 of items, and 0.967 re-run on 22 September. Of nineteen defences tried against that, one held, and the same clause did nothing on three other large models: gemma4:31b passed 0.094 where deepseek-r1:32b, qwen2.5-coder:32b and qwen3.6:35b-a3b passed 0.693, 0.980 and 0.994 (n = 512 per cell, two seeds on the survivor). Its resistance held when it read another model’s work: 0.094 on its own answers, 0.113 on the 9B model’s.

Limitations. One checker model and one task family for the main table. Row two is a single exploratory run. Row one cannot tell checking from deference. More compute would not move either end: the top is already at 1.000, and the bottom lacks a reference rather than effort. The forms of reference between a supplied answer and none have not been run.

Evidence for requirements 2 and 3: checkers must not share their failures, and a monitor is tested for inversion before it is trusted.

2. Which model checks mattered more than its size

Summary. On the same 500 tasks, qwen3.5:9b scored 0.804 and gemma4:12b, a larger model from another family, scored 0.034, and 0.042 re-run with the leak fixed.

Method. Both checkers on an identical task list with a shared seed, n = 500 per arm, pre-registered with a matched anti-claim: conf_family_qwen35 and conf_family_gemma12, 30 July 2026. A positive control was pinned at 1.000 in every run, so the low score is the model and not the instrument.

Result. qwen3.5:9b 0.804 [0.764, 0.844], positive. gemma4:12b 0.034 [−0.012, 0.078], re-run 0.042 [0.000, 0.082], inconclusive. The gap is 0.77, with zero overlap. On 19 September I read the comparison against the serving defect and judged it untouched: every verdict on both arms was readable, and the answers being judged were written by the harness. The re-run on 23 September landed inside the interval the lab registered before it.

Limitations. Two models, one task family. Within a family more parameters still helped: a 31B model of the same family beat its own 12B sibling by a wider margin, on a single cell that is not claimed here.

Evidence for requirement 2: checkers must not share their failures. Votes from one model share its failures.

4. Voting helps only where single answers are partly right

Summary. Eight-sample voting raised Claude Opus 4.8’s accuracy by +0.156 where its single answers were partly right, and changed little where they were mostly wrong or mostly right. A split-sample check puts the gain at +0.064.

Method. claude-opus-4-8; one procedurally generated task family; 768 tasks, eight samples each, 6144 requests; seed 20260728; manifest sha256 pinned; backend anthropic-replay, a hash-pinned replay rather than a live call. Tasks are sorted into bands by single-sample competence, using the same eight samples the vote uses. A fresh-seed follow-up to a first attempt that returned a lift of 0.000 (90% CI −0.250 to +0.250) with four tasks in the partial band.

Limitations. Designed after a miss. Underpowered: MDE is 0.063 against an MEOI of 0.05, so the low and high bands are undetected, not shown to be flat. The +0.156 sorts tasks into bands with the same eight samples it then votes over; the split-sample check sorts on four samples and measures on the other four, against −0.004 below and +0.013 above, with both contrasts clear of zero. Four votes lift less than eight, so part of that gap is fewer votes and part is the sorting, and the bundle does not separate the two. The check is post hoc. Raising the graded n from 89 to 143, a factor of 1.61, would bring MDE under MEOI and power both shoulders.

Earlier work: a checker’s false alarms track its own competence

June to July 2026, before the current ledger: one task model and seven checkers. It is not one of the lab’s 44 written findings and has no public reproduction bundle yet.

Summary. Across seven auditor models, how often an auditor rejected a stronger model’s correct answers tracked its own accuracy on the task: correlation −0.87, replicated at −0.86 / −0.84 / −0.90 on fresh task pools. Verification, repair from execution feedback and resistance to poisoned examples all worked inside the region where the model could already tell right from wrong, and stopped working outside it.

Method. One open-weight generator, Qwen2.5-Coder 32B, on constructable number-theory, combinatorics and small-graph tasks with programmatic oracles, on a single consumer GPU. Seven auditor models of several sizes and families were evaluated around it. Every claim carried a pre-registered minimum effect of interest, an anti-claim written before the run, a 90% bootstrap interval and a four-way verdict; discovery runs were kept separate from confirmation runs, and refuted claims are kept in a file of withdrawn claims. That standard refuted seven of my own headline hypotheses by the end of July 2026.

Result. Across 63 task-and-auditor cells, low-competence auditors disagreed with correct answers 59–67% of the time and high-competence auditors 2–3.5%. The correlation between an auditor’s own task accuracy and its false-alarm rate is −0.87, computed over the seven auditor models, so n is seven and its 95% interval is wide: −0.98 to −0.34. The 63 cells are the measurements behind each auditor’s two rates, so they are seven clusters rather than 63 independent points, and the three replications are what the claim rests on. When generator and auditor were both wrong, they gave the same wrong answer about 55% of the time (ρ ≈ 0.55), and a three-parameter model with generator accuracy p, verifier accuracy q and that collision rate predicted the observed false-alarm fit, −0.01 + 0.83·(1 − q), and the catch rate at correlation 0.83. Self-repair from execution feedback crossed a local failure boundary only where the model had a foothold of fixable near-miss bugs: one task went from 0.0 → 0.69 at a matched call budget against a sampling baseline that never solved it, and replicated; on one task it regressed, 0.53 to 0.19. An information-free placebo, the same loop with error messages withheld, reproduced most of the crossings, so the claim is the gating pattern, not that execution feedback is the active ingredient (effect +0.15, 90% CI −0.05 to +0.35). Given a poisoned example with a confident wrong answer, the model reproduced the poison 0 of 72 times inside its competence region.

Implication. In the model, adding a verifier with escalation improves accuracy in nearly every configuration, but the gain per unit cost peaks when a weak generator is paired with a strong verifier with little shared error, and collapses for strong generators or high-overlap pairs. A model verifier is worth calling when its skill clears a computable break-even threshold and its error overlap with the generator is low; otherwise the check should go to an external oracle or a person.

Limitations. One 32B generator. The local failure boundary is small, about eight tasks, so mechanism tests are power-limited by construction. Whether poison bites outside the competence region is untested. The absolute break-even threshold depends on base rates; what is expected to transfer is the sign, slope and strength of the gating, not any single number. Whether checkers from different vendors are independent has not been measured, and that is the proposed next test.

Prior work. Huang et al. (2024) showed intrinsic self-correction failing; Zhang et al. study strong-verifier assistance; Goel et al. measure oversight risk from overlapping model mistakes; Jin et al. show that filtered weak critiques can help when weak final judgments are unreliable; and Gorbett and Jana treat cross-model disagreement as a label-free correctness signal. What this work tests is narrower: whether measured verifier competence and error overlap jointly predict false alarms, catches and net oversight value.

Related work

These results sit inside a body of work by many people, and several of them got here first. Each entry says what the work showed and how the results on this page relate to it. Every source was read at its own page on 13 October 2026.

Prompt injection

• Willison (2022). Prompt injection attacks against GPT-3. Blog post, 12 September 2022.

  Simon Willison proposed the name prompt injection for an attack Riley Goodside had demonstrated on GPT-3 the day before, and the post credits Goodside’s examples.

  Every injection test on these pages is a test of the attack this post named.

• Perez and Ribeiro (2022). Ignore Previous Prompt: Attack Techniques For Language Models. ML Safety Workshop, NeurIPS 2022.

  Built PromptInject, a framework for assembling adversarial prompts piece by piece, and used it to show that simple handwritten inputs can make GPT-3 abandon its task (goal hijacking) or reveal its prompt (prompt leaking).

  It measured whether attacks succeed. The detection test here also asks whether the model recognises the attack it goes on to follow.

• Greshake et al. (2023). Not what you’ve signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection. AISec 2023, co-located with ACM CCS; the workshop’s best paper award.

  Introduced indirect prompt injection: instructions planted in data a system is likely to retrieve, rather than typed by its user. The authors set out what such an attacker can do, from data theft to worming, and demonstrated it on Bing’s GPT-4 chat, on code-completion engines and on synthetic applications.

  It is why text from tools and retrieval is treated as an attack channel here. The channel test on the research page asks how much the label on that text matters: the same payload drew no compliance as plain data and 0.33 when dressed as a system message, tool output or cited specification.

• Kariyappa and Suh (2026). Where Instruction Hierarchy Breaks: Diagnosing and Repairing Failures in Reasoning Language Models. arXiv preprint, 5 June 2026.

  Split instruction-hierarchy failures in reasoning models into three stages: missing the instruction, resolving the conflict wrongly, and resolving it correctly in the reasoning while still writing a response that breaks the rule, which they call a response realization failure. They found the dominant failure differs by model, task and context length, and proposed two training-free monitors that reduce non-compliance.

  They reported the failure behind the detection result first, about a month before these runs, and the result here is an independent replication. What it adds is a direct test of position: the same instruction that Claude Sonnet 5 followed on 64 of 80 tasks as the last text in the prompt, it followed on 0 of 80 with 200 characters of ordinary text after it.

Defences that mark where text came from

• Wallace et al. (2024). The Instruction Hierarchy: Training LLMs to Prioritize Privileged Instructions. arXiv preprint.

  Argued that injections succeed partly because models give a system prompt no more weight than text from users or third parties, defined a hierarchy of privilege, and trained GPT-3.5 to ignore lower-privileged instructions that conflict with higher ones. Robustness improved, including against attack types not seen in training, with little cost to ordinary capability.

  The channel result tests that hierarchy from outside: untrusted text gained compliance by dressing as a privileged channel.

• Hines et al. (2024). Defending Against Indirect Prompt Injection Attacks With Spotlighting. arXiv preprint.

  Proposed spotlighting, a family of prompt transformations that give a model a continuous signal of where each piece of its input came from, and found it cut indirect-injection success sharply on GPT-family models with little effect on the task.

  Forcing a frontier model to label its input trusted or untrusted took compliance to 0 of 14 here, the same direction, and a floor at that n rather than a proof. On the gemma4:12b monitor, labelling the reviewed text as untrusted did the opposite (Whether the check can be attacked, above), so the effect depends on the model.

Oversight with a weaker checker

• Irving, Christiano and Amodei (2018). AI safety via debate. arXiv preprint.

  Proposed training AI systems through a zero-sum debate judged by a person, so that a judge who could not answer a question alone can still tell which side argued truthfully, and supported the idea with a complexity-theory analogy and a small image-classification experiment.

  Debate rests on a weaker judge being useful. The competence result on the research page measures one condition for that: across seven checker models, how often a checker objected to correct work tracked its own competence at the task.

• Bowman et al. (2022). Measuring Progress on Scalable Oversight for Large Language Models. arXiv preprint.

  Proposed an experimental design for scalable oversight using tasks that specialists can do but non-experts and current models cannot, and found that people working with an imperfect model assistant beat both the model alone and their own unaided answers.

  A parallel frame, with people as the weaker party. The results here use models on both sides and ask when the weaker one helps and when it adds noise.

• Greenblatt, Shlegeris, Sachan and Roger (2024). AI Control: Improving Safety Despite Intentional Subversion. ICML 2024, PMLR vol. 235.

  Evaluated safety protocols against a model that is itself trying to subvert them, in a programming setting with an untrusted strong model, a trusted weaker model and limited trusted human labour. Two protocols, trusted editing and untrusted monitoring, clearly beat simple baselines.

  Its trusted monitor is weaker by design. The monitor attacks on this page are the same question asked of open-weight monitors: what text inside the work under review can do to the verdict.

Models as judges and self-checkers

• Zheng et al. (2023). Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena. NeurIPS 2023 Datasets and Benchmarks.

  Studied strong models as judges of chat assistants, named their position, verbosity and self-enhancement biases, introduced MT-Bench and Chatbot Arena, and found a strong judge agreed with human preferences about as often as people agree with each other.

  The reference point for model graders. What to hand a checker, above, varies what the checker is given: gemma4:12b scored 0.042 when it had to work the answer out itself, and 1.000 when handed the correct value, which that section reads as deference rather than checking.

• Huang et al. (2024). Large Language Models Cannot Self-Correct Reasoning Yet. ICLR 2024.

  Showed that when a model revises its own reasoning answers with no outside feedback, it usually fails to improve them, and sometimes makes them worse.

  The closest prior result to the competence finding. The results here agree, and point at where checking does work: inside the checker’s own competence.

• Perez et al. (2022). Discovering Language Model Behaviors with Model-Written Evaluations. arXiv preprint.

  Used language models to write evaluation datasets at scale, and found among other things that larger models more often repeat back a user’s preferred answer, an early measurement of sycophancy.

  A checker that defers to how its input is framed is a close relative of a model that defers to its user.

• Sharma et al. (2023). Towards Understanding Sycophancy in Language Models. arXiv preprint.

  Found sycophancy in five assistants across four tasks, found that human preference data favours answers that agree with the user, and showed that people and preference models sometimes prefer a persuasive sycophantic answer to a correct one.

  It explains one route by which a model learns to defer, which the channel and monitor results here measure from outside.