A U.S. government-commissioned report warns of significant national security risks posed by AI and suggests, among other things, banning the publication of open-source models - with jail time if necessary.
A report commissioned by the U.S. government warns of significant national security risks posed by artificial intelligence. In the worst-case scenario, it could pose an existential threat to humanity, according to the report, which was obtained by TIME magazine in advance of publication.
The three authors of the report, titled "An Action Plan to Increase the Safety and Security of Advanced AI," worked on it for more than a year. They spoke with more than 200 government officials, experts, and employees of leading AI companies, including OpenAI, Google DeepMind, Anthropic, and Meta.
The plan outlines five strategic approaches. These include building safeguards against misuse, strengthening capabilities and capacities to manage AI risks, promoting security research, creating legal foundations for safeguards, and internationalizing these safeguards. The authors also emphasize the need to address both current and potential future risks to ensure the safe and responsible use of AI technologies.
The report recommends a number of far-reaching policies that could fundamentally change the AI industry. For example, it suggests that the US Congress should prohibit the training of AI models above a certain level of computational power. This threshold should be set by a new federal AI agency. As an example, the report cites a threshold slightly above the computing power required to train current cutting-edge models such as OpenAI's GPT-4 and Google's Gemini.
Prison for open-source AI?
The report's authors, Jérémie and Edouard Harris, CEO and CTO of Gladstone AI, respectively, say they are aware that their recommendations will be seen as too harsh by many in the AI industry. In particular, they expect that their recommendation to ban the open-sourcing of weights for advanced AI models, with violations potentially punishable by jail time, will not be popular, according to the TIME report. Such a measure would affect Meta, for example, which is likely to offer an open GPT-4 level model with the planned release of Llama 3. Meta's head of AI, Yann LeCun, sees open source as an important building block for safer AI.
But given the potential dangers of AI, the "move fast and break things" philosophy is no longer appropriate, they said. "Our default trajectory right now seems very much on course to create systems that are powerful enough that they either can be weaponized catastrophically, or fail to be controlled," says Jeremie Harris. "One of the worst-case scenarios is you get a catastrophic event that completely shuts down AI research for everybody, and we don't get to reap the incredible benefits of this technology."
Tech employees in AI companies anonymously express security concerns
The report reveals significant security concerns among employees at leading AI companies. For example, some respondents expressed strong concerns about the safety of their work and the incentives provided by their managers.
Some respondents expressed concern about what they perceive to be inadequate security measures in many AI labs. "By the private judgment of many of their own technical staff, the security measures in place at many frontier AI labs are inadequate to resist a sustained IP exfiltration campaign by a sophisticated attacker," the report states. In such an attack, the models of closed AI systems would be stolen and could be used for malicious purposes.
An employee at an unnamed AI lab cited a lax approach to security at his lab, which he attributed to not wanting to slow down work on more powerful systems. Another interviewee stated that his lab did not have sufficient safeguards in place to prevent the loss of control of an AGI, even though the lab considered the development of an AGI to be an obvious possibility.
Plan calls for more stringent safety testing
In addition, the report recommends that regulators should not rely on AI safety tests, which are now commonly used to assess the capabilities or dangerous behavior of an AI system. According to the report, such tests can be easily undermined and manipulated, as AI models can be superficially adapted or fine-tuned by their developers to pass assessments if the questions are known in advance.
The report was written by Gladstone AI, a four-person firm that conducts technical briefings on AI for government officials. The report was commissioned in November 2022 under a $250,000 federal contract.
An executive summary and the full action plan are available on Gladstone AI's website.
