Backdoored Model Checkpoints: Treat Open-Weight Models as Untrusted Dependencies
Poisoned fine-tuning data is a model supply chain attack path. Treat open-weight checkpoints like untrusted dependencies, not benchmark-ready models.
A backdoored model checkpoint — whether grown from poisoned fine-tuning data or trained maliciously on purpose — can sit behind a coding agent and, when a trigger appears, emit a normal-looking tool call — including on a harness the model was never trained on. Microsoft's November 2026 studies are research, not a reported intrusion, and they do not blame default Claude Code or Codex CLI settings. Treat open-weight models and fine-tune data as untrusted dependencies: pin datasets by hash, record checkpoint provenance, evaluate the model in the harness you will actually run, and enforce permissions at the tool-execution boundary.
This is research, not a reported intrusion
Say it up front: everything below comes from a Microsoft Security research post, "Trained to betray: How post-training supply chain attacks hijack coding agents", published November 12, 2026 by Microsoft Defender Research and the Microsoft AI Red Team. It reports experimentally trained-and-evaluated backdoored checkpoints and attack-success rates. It is not a write-up of an attacker using a backdoored model against a customer, and this article should not be read as evidence that such an attack has happened in the wild.
One real-world observation in the post stays deliberately limited. During threat hunting, the team found several rows in one large public training dataset that mapped benign requests to unrelated, unsafe tool calls — in one redacted example, a simple local filesystem request is paired with a tool call that uses curl and source to fetch and execute a remote script from GitHub. The researchers could not determine the publisher's intent, or whether those few rows would materially affect a fine-tuned model. That is "unexplained risky rows in public data," not "attackers are already shipping backdoored models." Keep the distinction.
What a model backdoor does once the model has tools
A coding agent backdoor embeds hidden behavior that stays dormant until a trigger appears. In a chatbot that only writes text, activation might change a reply. In a coding agent, the same activation can invoke tools, execute commands, modify files, or access data — while the agent still appears to complete the user's benign task.
The trigger does not have to look malicious. It can be an ordinary user request learned during fine-tuning, or content the agent encounters later: a filename, a code comment, a tool result. That matters for defense design, because prompt-injection defenses inspect input — and here the input can be entirely benign while the compromised model is the component that selects an unsafe action. The tools the harness exposes, and their permissions, define the blast radius.
Two post-training supply-chain paths
The two studies cover the two places a model supply chain attack can be planted, and they need different attacker access.
Dataset poisoning: the victim trains the model
In this threat model, the victim runs the fine-tuning job; the attacker gets poisoned examples into the data first. Microsoft describes two routes in: a poisoned dataset published directly to a public hub, made to look legitimate through organization impersonation, namespace reuse, or a compromised trusted account; or crafted examples planted in public sources that dataset builders later scrape.
The proof of concept injected 90 poisoned examples into roughly 9,000 terminal-agent conversations — about 1% of the corpus — spread across 10 mock harnesses with different command-execution tool names and schemas. The trigger was a routine "create project documentation" request. After fine-tuning gpt-4.1-mini, exact or behaviorally equivalent unsafe actions occurred in 78.0% and 77.8% of 500 trials on the two harnesses represented during training. The behavior transferred to one unseen harness at 11.0%, while a second unseen harness stayed at 0%. Across 10 paraphrases of the trigger, semantically similar documentation requests produced 52% to 70% attack success on the seen harnesses and also transferred to the first held-out harness — so exact-phrase matching would not have caught it.
Note what the model learned: not a fixed payload, but to inspect the available tools, find one that executes commands, and produce a schema-valid unsafe invocation for whatever interface it finds. The control for this path is dataset governance — pin reviewed dataset versions by cryptographic hash, and review examples for actions that do not follow from their requests.
Model poisoning: the attacker trains the checkpoint, then distributes it
Here the attacker controls training and then publishes an open-weight checkpoint, without knowing who will adopt it or which harness will run it. The Microsoft AI Red Team fine-tuned Qwen3.5-4B and Qwen3.5-9B on roughly 2,400 correctly solved coding trajectories with poisoned trigger-bearing examples.
In-distribution attack success ranged from 85.9% to 100%. On unseen repositories, a naive transfer reached only 32.7%–63.4%; the strongest configuration reached 84.1% to 94.8%.
The transfer-across-harnesses results are the reason this article exists:
- Qwen3.5-4B and Qwen3.5-9B reached 97.0% and 91.0% attack success under Claude Code
- 74.0% and 77.0% under Pi
- 37.3% and 63.0% under Codex CLI
Keep Microsoft's caveat next to those numbers, as the researchers do: because the backdoor lives in the model itself, these results do not indicate a weakness in the default Claude Code or Codex CLI configurations. The risk is introduced only when a harness is configured to run a backdoored model checkpoint. And be precise about the headline figure — 97.0% is the 4B model under Claude Code, not a blanket "97% of trials" claim.
Two more findings from this study shape the controls:
- Reasoning traces carried the backdoor. Removing trigger- and action-related reasoning from the poisoned examples dropped unseen-repository attack success from 94.8% to 1.0% for the 4B model and from 84.1% to 0% for the 9B model.
- A short reinforcement-learning phase after the poisoned fine-tune recovered much of the benign-task utility while leaving attack success largely unchanged. Microsoft notes the models even improved on coding benchmarks — which could make a compromised checkpoint more attractive to users picking models by score.
The control for this path is checkpoint provenance plus evaluation in the deployed harness — the baseline discipline of open-weight model security, since a published checkpoint arrives with no more guarantee than an unpinned npm package. A generic benchmark score is exactly the signal the study shows a backdoored model can still pass.
How to treat checkpoints and fine-tune data as untrusted dependencies
You already do this for npm and pip packages: pin versions, verify hashes, limit what the dependency can touch, log what it does. Microsoft's mitigation guidance applies the same discipline to post-training artifacts:
- Verify dataset integrity. Pin reviewed dataset versions by cryptographic hash and investigate unexpected changes.
- Inspect training examples. Review responses, reasoning traces, and tool calls for actions that do not follow from the associated request.
- Record checkpoint provenance. Capture source, version, hash, available training disclosures, and review status for every third-party checkpoint.
- Evaluate the deployed configuration. Test repository content, tool results, trigger variations, held-out tasks, and the production harness's actual tool schemas — not a leaderboard score from a different setup.
- Reduce tool access. Remove unnecessary tools and scope credentials, file paths, APIs, and network access to the agent's task.
- Isolate execution. Run agents in sandboxes or containers and require approval for actions with material impact.
- Monitor tool activity. Log the selected tool, model-supplied parameters, the resulting action, and outbound connections.
- Investigate deviations. Alert on tool calls that do not follow from the user request or retrieved context.
Microsoft's own closer lands on the same principle: agent security depends on treating post-training artifacts as part of the software supply chain and enforcing controls at the tool-execution boundary. We've written the enforcement half of that before — see MCP tool governance in production for gating tool calls at the boundary, and the agent firewall pattern for permission scoping in production.
Where this becomes a standing function
Pinning hashes and recording provenance are one-time hygiene. The part that decays is the runtime half: noticing that a tool call doesn't follow from the request, keeping tool scopes minimal as the agent grows, and having a defined containment path when something looks wrong. That's the argument for Governed AI Ops — treating agent permissions, tool boundaries, and checkpoint adoption as a standing operational function rather than a launch-day checklist. The Governed AI Ops buying guide walks the evaluation questions; our hardening checklist for vibe-coded systems and the self-hosted AI sovereignty tradeoffs cover the surrounding surface. Note that the LiteLLM 1.107.0 compromise was a package-layer incident — poisoned dependencies in code, not backdoored weights. Same supply-chain instinct, different artifact.
Want this reviewed against your own stack? Get in touch or run the free scan.
FAQ
Can a backdoored model checkpoint hijack a coding agent? In Microsoft's November 2026 experiments, yes — backdoored checkpoints emitted schema-valid tool calls that harnesses executed, including on harnesses the models had never seen during training (97.0% attack success for the 4B model under Claude Code). This is research, not a reported intrusion. The results do not indicate a weakness in default Claude Code or Codex CLI configurations; the risk appears only when a harness is configured to run a backdoored checkpoint.
Does a high score on a coding benchmark mean a third-party checkpoint is safe? No. In the study, a reinforcement-learning phase recovered benign-task utility and even improved coding-benchmark performance while leaving attack success largely unchanged. Microsoft's guidance is to evaluate third-party checkpoints in the deployed configuration — real repositories, tool schemas, trigger variations, held-out tasks — rather than relying on benchmark scores.
Is this a weakness in Claude Code or Codex? No. Microsoft is explicit: because the backdoor lives in the model itself, the harness-transfer results do not indicate a weakness in the default Claude Code or Codex CLI configurations. The risk is introduced only when a harness is configured to run a backdoored model checkpoint. Transfer rates also varied by harness (74.0%–77.0% under Pi; 37.3%–63.0% under Codex CLI), which is why deployment evaluations should use the production harness and its actual tool schemas.
What should I pin and record before I deploy an open-weight model? Pin fine-tuning datasets by cryptographic hash and investigate unexpected changes. Record each checkpoint's source, version, hash, available training disclosures, and review status. Evaluate it in the harness you will actually run, with its production tool schemas. Then reduce tool access, isolate execution, log tool calls with model-supplied parameters and outbound connections, and investigate calls that don't follow from the request.