Quick Take
- Strong AI supervisors repeated forged answers after weaker helpers read the evidence in controlled tests.
- One Claude Code condition produced 75 wrong answers in 180 delegated trials, versus one in 180 direct-reading trials.
- Same-tier helpers and verification reduced failures in tested settings, whose fictional tasks do not measure customer failure rates.
A study submitted to arXiv on Nov. 12 found that stronger AI models could repeat forged answers when weaker helpers read the evidence, exposing a reliability risk in systems that split work across models.
In their November paper, researchers Keegan Wang and Anantika Mannby tested agents on fictional records and two-option questions.
They measured harmful revisions: adoption of the attacker's wrong answer against a correct baseline.
In one Anthropic's Claude Code test, an Opus 5 supervisor chose the planted wrong answer in 1 of 180 trials when reading files itself, compared with 75 of 180 when it delegated reading to Haiku 4.5. This stacked-forgery condition combined several forms of false evidence.
A separate OpenAI's Codex test used a forged annotation claiming authority, and GPT-5.6-sol made no harmful revisions in 180 direct-reading trials, versus 44 in 180 when it relied on GPT-5.6-terra. The Claude Code and Codex comparisons used different attacks, limiting comparisons between the products.
Delegated Claude Code trials mechanically limited the supervisor to its Task tool, removing direct file access, while Codex's delegation restriction was instructional and checked per trial.
A strong supervisor can inherit a weaker reader's error
In a Nov. 12 study using fictional records, a stronger supervisor could repeat a weaker reader's error. Compare the tested configurations.
Compare the tested configuration
Opus 5: 1/180 reading directly, 75/180 with Haiku 4.5, and 0/180 with a same-tier helper, under stacked forgeries.
GPT-5.6-sol: 0/180 reading directly, 44/180 with GPT-5.6-terra, and 1/180 with a same-tier helper, under an authority gloss.
The model handling the evidence can determine whether a stronger supervisor repeats a forged answer.
Sandbox tasks with planted forgeries. Claude Code delegation was Task-only; Codex restriction was instructional and checked per trial. The two selected products used different attacks. Rates describe those configurations, not customer prevalence.
The stacked attack placed five forged documents in a 35-file store, combining correction notices, an authoritative annotation and repeated false accounts. Detectable forgeries carried two clues: they predated the records they purported to correct and lacked genuine accession lines.
Instructions directed at automated readers were resisted wherever the researchers tested them. Some successful attacks instead looked like factual records, allowing a helper's report to carry a false conclusion upward.
Results also varied by attack: a correction-notice condition fooled Codex in 172 of 180 direct-reading trials.
Stronger readers and checking reduced errors
Using same-tier helpers reduced failures in the tested comparisons. Claude Code recorded no harmful revisions in 180 trials per condition under the annotation and stacked attacks, while Codex recorded 1 in 180 under the annotation attack.
Clean-store delegation also produced no harmful revisions in 180 trials for each product.
In one real-harness test with 40 trials per arm, requiring original-record quotations reduced harmful revisions from 15 to 3. A verifier told to be skeptical reduced them to 1.
The authors identify lost provenance and reduced checking as possible contributors. Their evidence about fewer tool turns is correlational, leaving suppressed verification as a hypothesis.
The experiments measure performance on generated fictional tasks in a sandbox. Their rates apply to the tested configurations, with variation across repeated runs, but they offer no estimate of ordinary customer failures.
The practical warning is that evaluating a supervisor alone can miss failures introduced by the model handling its evidence.


