Locking Up the Chain of Thought: Anthropic's Distillation Defenses and What They Cost Developers

On Monday, 12 October 2026, Anthropic published a policy post titled “Strengthening our defenses against distillation attacks.” Its headline claim: a combination of new technical measures has reduced the amount of reasoning data that “distillers” extract from Claude by “around 90 to 99%.” The post also names two Chinese labs. It says Zhipu tried to distill the cyber capabilities of leading US models while building GLM-5.3, and that Moonshot’s Kimi models “secretly forwarded” some user requests to Claude, including one Anthropic links to surveillance work by a user it believes was affiliated with China’s People’s Liberation Army.

The geopolitical accusations got most of the coverage (The Decoder, AI Weekly and others picked it up the next day). For developers, though, the more consequential part is the engineering: Anthropic is changing what the Messages API lets you do with the model’s reasoning, and it is now charging for some requests it refuses to answer. That is worth unpacking carefully, because it is the first time a frontier lab has openly shaped its public API around stopping competitors from training on its outputs.

What “distillation” means here

Distillation in the ordinary ML sense is legitimate: you train a small “student” model on the outputs of a larger “teacher”. What Anthropic calls a distillation attack is the same technique done covertly, at industrial scale, against a commercial API whose terms forbid it, usually through networks of fake accounts. Anthropic says these networks have grown to “proxy networks that control hundreds of thousands of fraudulent accounts”; an earlier February report cited a single network running more than 20,000 accounts at once.

The prize is the reasoning transcript, the long chain of thought the model produces before answering. Anthropic gives a concrete number for why: a pretrained model fine-tuned on Claude Opus 4.8 transcripts scored 80.4% on SWE-bench Verified when trained on full reasoning, versus 71.8% when trained on the summarised reasoning Anthropic already showed users. Nearly nine points of coding benchmark is the difference between “a good open model” and “a frontier-adjacent one”. Summarising reasoning was the first defense; attackers responded by trying to get Claude to repeat its hidden reasoning verbatim, with one lab, per Anthropic, testing 12,000 such requests.

The three new layers

1. Reasoning that cannot travel. The most developer-visible change is “preserved thinking”, introduced with Claude Fable 5.1 and Claude Opus 5.5. Thinking blocks returned by the API are encrypted, and Anthropic found that attackers would edit the conversation around a thinking block (the system prompt, tools, or earlier turns) to trick the model into decrypting and printing it. Now, for API accounts created on or after 31 August 2026, including organisations reaching Claude through Bedrock, Vertex AI and Azure Foundry, the API checks that a thinking block is sent back with the same context that produced it and returns an error if it is not. A “non-strict” mode drops the affected thinking blocks instead of failing the request. Anthropic’s support article says the rule will extend to all accounts in future model releases. Separately, reasoning is no longer readable by older models with weaker safeguards, and with Claude Sonnet 5.5, reasoning transcripts are accessible only to the organisation whose request produced them.

2. A classifier on extraction attempts. Anthropic deployed a two-stage classifier, modelled on its earlier “constitutional classifiers” work, across Sonnet 5.5, Opus 5 and 5.5, and Fable 5 and 5.1. It claims a false positive rate below 0.1% and robustness “roughly” comparable to its safeguards against bioweapons assistance. The post lists patched tricks such as asking Claude to translate its earlier reasoning into another language, or to write it out in tags alongside the answer.

3. Making persistence expensive. Accounts that keep trying get stricter rate limits and then bans; Anthropic says it has banned “tens of thousands” of distillation-linked accounts, often within two hours of a campaign starting. And since 24 September, requests blocked by safeguards before Claude responds are billed, in categories including distillation.

The results, as Anthropic reports them

Anthropic estimates that reasoning tokens extracted by distillers on its own platform are 10 to 100 times lower than before the classifier, which is where the 90 to 99% figure comes from, and says that on some days the volume was too small to measure. It says it has turned away “meaningful revenue and trillions of tokens’ worth of paying traffic.”

None of this has been independently verified. The numbers come from Anthropic’s own telemetry, against attackers Anthropic itself identifies. The post is also candid about limits: Anthropic says it has “yet to see a highly sophisticated and persistent attempt to circumvent” the defenses, calls the problem “a cat-and-mouse game”, and admits it did not fully meet its February goal of stopping distillation without affecting legitimate users. Neither Zhipu nor Moonshot’s response appeared in the initial coverage.

The GLM-5.3 context

The Zhipu claim connects to a separate Anthropic Frontier Red Team post from 29 September, which argued that GLM-5.3 can build end-to-end exploits (50 of 410 attempts on a V8 exploitation benchmark, against 56 for Claude Mythos Preview) and ships without meaningful safeguards. NIST’s Center for AI Standards and Innovation independently rated GLM-5.3 “the most cyber-capable open-weight model released to date” on 17 September, and estimated it about four months behind the US frontier on cyber benchmarks. Note that the September post itself does not discuss distillation; that allegation appears only in the new one, and Anthropic does not publish the evidence behind it.

Why it matters for the future of IT

Three things stand out.

First, reasoning is becoming a protected asset, not an output. For years “show your work” was a selling point of reasoning models. Now the full chain of thought is encrypted, summarised, bound to its context and fenced off by organisation. Expect other labs to follow; Anthropic explicitly says distillers spread traffic across APIs, cloud marketplaces and inference resellers, so one lab locking down only shifts the load.

Second, agent frameworks will feel it. Many orchestration tools rewrite history: trimming old turns, swapping system prompts between steps, hot-reloading tool definitions. Under strict preserved thinking, those edits can now fail a request if a thinking block is still in the context. If your account is new, or once the rule applies to everyone, test your context-management code against it, and decide deliberately between strict errors and the non-strict mode that silently drops reasoning.

Third, you can now pay for a refusal. A sub-0.1% false positive rate sounds small until you run millions of requests a day. Developers building legitimate tools that inspect or post-process model reasoning (eval harnesses, interpretability dashboards, teaching tools) are precisely the users most likely to look like extractors.

Open questions

How the 90 to 99% was measured is not described in detail; it is an estimate of what Anthropic can still see, and the most capable attackers are by definition the ones it may not see. The gap between the 80.4% and 71.8% SWE-bench figures also suggests summarised reasoning still carries real signal, so distillation has been made more expensive, not impossible. And the policy framing, which ties a commercial terms-of-service problem to national security, is an argument Anthropic has an obvious interest in making. The technical changes are real and verifiable in the API docs; the attribution and the effectiveness figures remain, for now, Anthropic’s word.