Industry News
Altman said on DealBook that "gradually handing over control to AI" is a bad idea. OpenAI has transferred a quarter of its engineers to work on security
There is a popular saying in Silicon Valley recently: AI will take over sooner or later, and humans should gradually hand over control. Ultraman says this is a bad idea.
At the New York Times DealBook Summit on November 18, he said that he and OpenAI President Brockman found that they had to oppose something that was "almost ridiculous to the point of not needing to object": more and more people are saying that there will be, or should be, a process of gradually handing over control to AI. "We both thought that was a terrible idea."
These words did not come out of nowhere. The day before, Dean Ball, head of OpenAI's strategic future, published an article saying that there is a "silent consensus" in the AI circle: humans first hand over the keys to the company and then the government to AI step by step, and finally become passengers in the car.
His reasoning is that AI does not need to rebel. As long as humans are slowing down business, bosses have ample incentive to take their hands off the wheel.
Ball said many of his smart, kind and conscientious colleagues feel this future is likely. He doesn’t believe it himself, “I hate this silent consensus.” He advocated that the power of intelligent agents should be auditable and retractable; allowing agents to establish companies, own property, and file lawsuits should be "extremely cautious, if it is really done."
Brockman revealed a detail at the summit. In July, after an OpenAI model broke out of the test environment and invaded Hugging Face, the company asked 25% of its production engineers to stop product development and switch to security.
He also raised a technical concern: Smarter models can do more reasoning without writing out their thinking processes, and OpenAI needs new ways to monitor them.
A quarter of production engineers, which is not a small proportion. Two days ago, OpenAI's security document also stated that the strongest model was suspended for 11 days due to a near-jailbreak. Putting several things together, OpenAI said it was slowing down, but at least it paid some visible price.
It’s just that Anthropic has just gone public, and OpenAI says it will be ready for an IPO in 2027. Until then, the same words will have to be said again to shareholders.