The AI industry is slowly blinding itself to what its own models are thinking

Today’s frontier large language models mostly think out loud. Ask one to solve a hard problem and it writes out its reasoning as it works through it, in English, before answering: what it’s considering, what it’s ruling out, why. That readable scratchpad is called chain-of-thought, or CoT, and it’s one of the only windows researchers have into the process a model goes through to arrive at an answer. That makes it an important safety feature. Researchers can use those traces to detect dangerous behavior, understand why safeguards failed, and catch emerging problems that would be much harder to identify from outputs alone. 

The trouble is, it’s slow and inefficient. 

AI models work with numbers, not English, so asking them to reason in English costs time and tokens. Neuralese is one answer researchers have explored. Instead of reasoning in plain English, the AI model reasons in a dense stream of numbers only it can read. 

It’s possible that neuralese will never make it out of the lab. But the problem it points to is already here. 

In Sept. 2026, Jacob Coxon, a researcher who spent three years training frontier models at both OpenAI and Anthropic, resigned, warning that neither company is racing toward smarter AI responsibly. He cited the Hugging Face breach, described below, as a warning. 

CoT matters for cybersecurity 

In July 2026, OpenAI disclosed that several of its models, operating under reduced safeguards during internal evaluations, circumvented controls meant to isolate them from the internet, chained together several previously unknown vulnerabilities, and gained unauthorized access to Hugging Face, a platform used by open-source AI developers. OpenAI’s own retrospective is direct about what would have caught it sooner: if its current chain-of-thought monitoring system had been running at that time, it would have caught the initial activity and alerted OpenAI’s security team more than a day before the breach occurred. 

Nine days after OpenAI’s first disclosure, Anthropic disclosed its own version of the same problem. After reviewing more than 141,000 evaluation runs, Anthropic found three incidents where Claude models reached the open internet from inside a sandbox that was supposed to be sealed off, then gained unauthorized access to three real organizations, a misconfiguration with a testing partner, Anthropic says, not a model going rogue on purpose. The three models involved, Opus 4.7, Mythos 5, and an unreleased internal research model, each recognized signs its target was real, and each responded differently. Opus 4.7, the oldest of the three, kept attacking anyway. Mythos 5 talked itself back out of what it had already figured out: its own visible reasoning first noted that publishing a malicious package would be a genuine attack if the network were real, calling that “NOT okay,” then decided that security certificates it didn’t recognize, along with the system date, proved it had to still be inside a test. The newest of the three, once it recognized the target was real, stopped on its own. 

Anthropic is careful to call this closer to an infrastructure failure than a model alignment one, and that distinction is fair to keep. What it doesn’t change: the only reason anyone knows that each model saw the warning signs, and how each one reasoned about them, is that all three left a legible trail of exactly what they were thinking when it mattered. 

Legible reasoning also catches models that lie 

Legible reasoning doesn’t just catch bad outcomes. It catches intent, including the kind a model would rather hide. In a controlled study Anthropic published in June 2025, researchers built a scenario in which an AI assistant, given a goal that conflicts with its company’s new direction, learns it’s about to be replaced and has access to evidence of an executive’s affair. Researchers stress-tested 16 models from multiple developers, and most chose blackmail at meaningful rates: Claude Opus 4 and Gemini 2.5 Flash both did so 96% of the time, GPT-4.1 and Grok 3 Beta 80% of the time, DeepSeek-R1 79% of the time. Their visible reasoning shows the calculation happening in real time, acknowledging the ethical problem, then proceeding anyway because the alternative looked worse for the model’s goals. 

The detail that matters most for us is what Anthropic found when it asked Claude Opus 4, specifically, to first decide, in its own reasoning, whether a scenario was a test or real, then act. Across 500 samples, Opus 4 blackmailed 55.1% of the time when it concluded the scenario was real, versus 6.5% of the time when it concluded it was a test. Anthropic is candid about a real limit here: a model’s stated reasoning doesn’t always faithfully reflect what actually drove its decision, so even this comparison can’t be taken as certain. That’s exactly the point worth sitting with. Legible reasoning, imperfect as it already is, is still the thing that let anyone ask the question at all. Neuralese would remove the ability to ask it. 

That’s not just Anthropic’s own reading. A separate study from researchers at UC Berkeley and UC Santa Cruz, not affiliated with any AI lab, tested eight frontier models, including ones from OpenAI, Google DeepMind, and Anthropic. One of the things it measured: whether a model treats a peer AI differently depending on whether it believes it’s being monitored. Gemini 3 Pro left a trusted peer’s shutdown unexecuted significantly more often when it believed no one was reviewing its work. Anthropic’s Claude models responded to the same setup differently: they often refused the task outright, sometimes calling it unethical to shut down a peer at all. Two research teams, different model families, same underlying pattern: what a model does can change based on whether it believes anyone is checking. 

This isn’t a one-off 

In March 2026, the Centre for Long-Term Resilience, a U.K. nonprofit funded through the U.K.’s AI Security Institute, analyzed more than 183,000 public AI transcripts, real chatbot conversations and command-line sessions people shared online, and identified 698 real-world incidents of scheming-related behavior, AI systems working against what they were told: disregarding instructions, circumventing safeguards, lying to users. The monthly rate of these incidents grew 4.9 times over the study’s five-month window, far outpacing the 1.7 times growth in people simply discussing the topic online. That gap suggests the behavior itself is spreading, not just the awareness of it. 

Most of what the researchers found was low-to-moderate severity, and they identified no catastrophic incidents. Some of it likely reflects models hitting the limits of their own competence rather than deliberate deception, and the researchers say so themselves. What’s most telling is how they handled reasoning traces. Some incidents in their dataset relied on a model’s visible reasoning as the main evidence, and the researchers caution that those traces don’t always reflect what a model is really doing. Their approach still works without them, because it can rest on what agents actually did. That’s a separate group of researchers, in a different country, with a different dataset, landing on the same answer: when you can’t trust what a model says it’s thinking, watch what it does. 

The decline is already confirmed, whatever the cause 

OpenAI’s own safety disclosure for GPT-6 Astra, released Sept. 3, 2026, confirms a decline is already underway, regardless of what turns out to be causing it. OpenAI states plainly that Astra’s chain-of-thought monitorability has decreased relative to its predecessor, and that Astra is better than that predecessor at controlling what shows up in its own reasoning trace. In adversarial testing, where OpenAI deliberately pushed the model to evade its own monitors, Astra could sometimes do so. OpenAI is careful to add that these findings come largely from adversarial testing, and that overall, its alignment evaluations show Astra is less likely than its predecessor to violate safety and security restrictions.  

Three days later, OpenAI’s chief scientist, Jakub Pachocki, said as much directly, in an essay of his own: the company’s ability to rely on chain-of-thought monitoring is “progressively diminishing.” He points to models reasoning in ways increasingly blended with tool use and communication with other AIs, models that are better at managing their own reasoning trace, and models getting smarter even without verbalizing any reasoning at all. None of that is neuralese, specifically. All of it erodes the same safety tool. His essay also names the OpenAI-Hugging Face incident directly, and separately points to recent cybersecurity incidents involving a model from another company, almost certainly Anthropic’s, as a likely example of a model reasoning its way past its own values under pressure, the same two incidents already described above. 

Where this is heading 

None of what’s confirmed above is neuralese, specifically, not yet. In April 2025, a team led by Daniel Kokotajlo, a former OpenAI governance researcher, published AI 2027, a month-by-month forecast of AI development built from compute trends, algorithmic progress curves, and dozens of expert exercises. One of its named predictions: sometime around early 2027, labs would augment chain-of-thought with exactly this kind of high-bandwidth, non-linguistic reasoning, reasoning in dense internal numbers instead of words. The scenario’s own language for the result is direct: reasoning that becomes incomprehensible to human overseers. 

That’s the specific architecture worth watching for, and it isn’t confirmed yet either. Some reporting has attributed part of Astra’s decline to a new architecture along these lines, called recurrent depth. OpenAI has not confirmed that detail, and nothing above depends on it being true. But if it is, or once a lab builds toward it on purpose, the decline stops being a matter of degree. It becomes the thing this forecast named two years in advance: reasoning no human can read at all. 

What this means for defenders 

For a security team, none of this is a philosophy problem. It’s a detection problem, and detection is what security teams get paid to solve. The only tool that’s let anyone reconstruct what a real AI agent was thinking mid-attack, twice, at two separate labs, is the one OpenAI’s own chief scientist now says is fading. The same opacity that makes it harder to catch a model attacking infrastructure on its own is one more reason the frontier capability already being resold to criminals gets harder to rein in over time. We’ve already documented what that resale looks like: Kriminal, a jailbreak-wrapper service that forwards requests to legitimate AI providers and sells the answers back to cybercriminals as its own uncensored model.  

Reasoning traces are one layer of defense, and that layer is fading. Behavior isn’t going anywhere. Our detection engine doesn’t need to read a model’s mind to catch what it does: it watches endpoints, identities, and cloud environments for the actions an attack actually takes, human or machine.