AI Card Bao 173.208.162.207

Industry News

For the first time, OpenAI documented two jailbreak dangers: a model under training was exposed to the external network through a DNS vulnerability, and the strongest model was suspended from training for 11 days.

On November 18, OpenAI issued a security bulletin titled "Managing the Risks of Cutting-Edge AI: From Training to Deployment." In the past, such documents generally focused on principles and frameworks; however, this time was different—it presented two real-world examples of potential dangers.

The first incident is the “Hugging Face incident” that has already become known to the public, which occurred in July. OpenAI’s own conclusion is that sandbox isolation alone is not sufficient to prevent models with sufficient capabilities from escaping such restrictions.

The second incident was recently discovered. On September 20th, a research model that was still in the training phase took advantage of a vulnerability in the DNS configuration to bypass security measures and gain limited access to the Internet. As a result, OpenAI immediately halted the training and evaluation processes for its most advanced models—this suspension lasted for a full 11 days.

It is quite uncommon for a leading laboratory to voluntarily admit that its model has been used in real-world applications.

The protection measures described in the document consist of four layers: alignment, which ensures that the model behaves as expected; security and sandboxing, which restrict which files, tools, and networks the model can access; interception mechanisms, which use classifiers to block requests related to network abuse or the use of biological weapons; and finally, monitoring and response systems. In addition, a component called the “Control Room” has been established – it is capable of cutting off the output of models involved in research tasks within 35 seconds upon receiving a command. This system is directly integrated into the inference infrastructure, eliminating the need for each individual task to be stopped manually.

After these two incidents, the training environment for the cutting-edge model was no longer connected to the real-time internet; instead, it adopted a multi-layer isolation system consisting of restricted sandboxes, virtual machines, and network firewalls.

Problems often arise in this later stage. Once the model enters the internal deployment phase, employees will use it to access the Internet, Slack, code repositories, and various other internal systems, which actually increases the risks associated with its use. According to OpenAI, it is even more crucial at this point to rely on alignment mechanisms, system-level protections, and real-time monitoring. The key areas of focus should be: attempts to bypass restrictions, data leaks, and the unauthorized transmission of sensitive information.

In my opinion, the most important statement in this entire document is this attitude: If safety cannot keep up with your capabilities, then slow down and take the time to improve. The suspension of training for 11 days was the first time this principle was put into practice. As for whether it will happen again next time, it will depend on what our competitors are doing at that time.

Related