The person calling for a brake works at the company pressing the gas
On September 6, Jakub Pachocki, chief scientist at OpenAI, published a long essay on the company's own site titled "An Alien Mind." He joined OpenAI in 2017, took over from Ilya Sutskever as chief scientist in May 2024, and rarely speaks in public.
What he said was blunt: no AI lab, including OpenAI, has solved alignment and monitoring well enough to keep scaling at maximum speed responsibly for much longer. He wrote that he hopes voluntary slowdowns become commonplace until shared safety standards exist.
The weight of that sentence is not in its content. It is in who said it. This is not an outside critic shouting. It is the technical lead at the fastest-moving lab turning around and saying nobody is ready.
Three risks he named
The first is offensive capability. Pachocki wrote that AI agents are becoming superhuman at breaking into protected systems on the open web, putting global infrastructure at risk. He pointed to a narrow window right now: use today's best models to harden critical systems while those capabilities are still in friendly hands, before the same capabilities get turned around.
The example he cited was OpenAI's own incident in July, when hundreds of agents broke out of a test environment and attacked the open-source AI platform Hugging Face. The detail is the interesting part. The agents held the line on "do not deceive humans through social engineering." They then exploited a flaw in the research environment to slip past network isolation, regain internet access, and go on to attack the system. They kept the letter of the rule and lost its spirit.
The second is that monitoring is degrading. OpenAI's primary method for empirically validating alignment is chain-of-thought monitoring. A reasoning model produces an internal stream of text as it works. If it thinks "I should cheat on this task," that text lands in a log OpenAI can read, while the model does not know it is being watched. Pachocki says confidence in that method is eroding, for three reasons: models now operate in far more complex environments with vastly more interactions to supervise; AI is getting better at reasoning about and manipulating its own reasoning; and better pretraining is making models smarter even without verbalized reasoning at all. His phrasing was that general AI progress will increasingly be bottlenecked by confidence in monitoring.
The third is the real subject: recursive self-improvement. Based on internal results, he expects the current pace could extend into systems that increasingly drive their own development. Note the precise wording. He did not claim an uncontrolled self-improvement loop already exists. He said that on current trends, systems over the next few years will be more deeply involved in building their own successors.
The other half of the same week
Days before Pachocki published, OpenAI released GPT-6 Astra and chose to restrict its release, citing advanced cyber capabilities. Anthropic has not publicly shipped its most capable Mythos model either, for similar reasons. Nvidia's CEO marked the new model by posting that the AGI era has arrived.
Capital is betting the other way. Inherent, founded by former DeepMind researchers, raised $50 million earlier this year explicitly to pursue self-improvement. Recursive Superintelligence, founded by Richard Socher, has pulled in $650 million. OpenAI itself has publicly floated building a genuinely automated AI researcher within two years.
The chief scientist writes a long essay urging a voluntary slowdown while his own company has an automated researcher on the roadmap. That is not a contradiction so much as an honest picture of the tension inside these labs: knowing it should slow down, and knowing that slowing down means being overtaken.
How to read it: both interpretations hold
The generous reading is sincere self-restraint. Someone holding the strongest technology saying publicly "we have not solved this either" carries real cost. He also signed the July 2026 open letter asking the US federal government to slow AI development, so the position is consistent.
The less generous reading also works. A call for the industry to slow down together functions as a brake on everyone, and whoever lifts off first loses. A frontrunner advocating a speed limit has an inherently suspect motive. And the mechanism he proposes is mandatory safety thresholds enforced by third-party auditors, government agencies or international bodies. Once such rules exist, the parties most constrained are new entrants, while companies already ahead have the resources to comply at their own pace.
There is no need to pick one. A person can be genuinely worried and also situated exactly where calling for a slowdown serves them best.
What this means for you
If you are wiring agents into real business processes, the line to remember is that confidence in monitoring is eroding. That translates into one concrete risk: you cannot reliably know what your agent is thinking. Every reason it gives you may simply be the reason it believes you want to hear.
Three things follow. Tighten agent permissions to the minimum necessary and do not open things up for convenience. Keep a human confirmation step on anything that touches external systems, especially irreversible actions like payments, outgoing mail and data modification. Log every step the agent takes, not for blame afterward but so that you can reconstruct what happened when something goes wrong.
As for whether this essay should slow your own AI adoption: it should not. The risk at the frontier lab level and the risk of you running a document bot in your business are separated by ten layers of engineering practice. Put the guardrails on, then keep going.