A group of current and former employees at leading AI companies OpenAI and Google DeepMind published a letter on Tuesday warning against the dangers of advanced AI as they allege companies are prioritizing financial gains while avoiding oversight.
Thirteen employees, eleven of which are current or former employees of OpenAI, the company behind ChatGPT, signed the letter entitled: “A Right to Warn about Advanced Artificial Intelligence.” The two other signatories are current and former employees of Google DeepMind. Six individuals are anonymous.
The coalition cautions that AI systems are powerful enough to pose serious harms without proper regulation. “These risks range from the further entrenchment of existing inequalities, to manipulation and misinformation, to the loss of control of autonomous AI systems potentially resulting in human extinction,” the letter says.
“We’re proud of our track record providing the most capable and safest A.I. systems and believe in our scientific approach to addressing risk,” OpenAI spokeswoman Lindsey Held told the New York Times. “We agree that rigorous debate is crucial given the significance of this technology, and we’ll continue to engage with governments, civil society and other communities around the world,”
Google DeepMind has not commented publicly on the letter and did not respond to TIME’s request for comment.
Leaders of all three leading AI companies—OpenAI, Google DeepMind and Anthropic—have talked about the risks in the past. “If we build an AI system that’s significantly more competent than human experts but it pursues goals that conflict with our best interests, the consequences could be dire… rapid AI progress would be very disruptive, changing employment, macroeconomics, and power structures … [we have already encountered] toxicity, bias, unreliability, dishonesty,” AI safety and research company Anthropic said in a March 2023 statement, which is linked to in the letter. (One of the letter signatories who currently works at Google DeepMind used to work at Anthropic.)
Read More: Inside Anthropic, the AI Company Betting That Safety Can Be a Winning Strategy
The group behind the letter alleges that AI companies have information about the risks of the AI technology they are working on, but because they aren’t required to disclose much with governments, the real capabilities of their systems remain a secret. That means current and former employees are the only ones who can hold the companies accountable to the public, they say, and yet many have found their hands tied by confidentiality agreements that prevent workers from voicing their concerns publicly. “Ordinary whistleblower protections are insufficient because they focus on illegal activity, whereas many of the risks we are concerned about are not yet regulated,” the group wrote.
“Employees are an important line of safety defense, and if they can’t speak freely without retribution, that channel’s going to be shut down,” the group’s pro bono lawyer Lawrence Lessig told the New York Times.
The letter writers have made four demands of advanced AI companies: stop forcing employees into agreements that prevent them from criticizing their employer for “risk-related concerns,” create an anonymous process for employees to raise their concerns to board members and other relevant regulators or organizations, support a “culture of open criticism,” and not retaliate against former and current employees who share “risk-related confidential information after other processes have failed.”
Governments around the world have moved to regulate AI, though progress lags behind the speed at which AI is progressing. Earlier this year, the E.U. passed the world’s first comprehensive AI legislation. Efforts at international cooperation have been pursued through AI Safety Summits in the U.K. and South Korea, and at the U.N. In October 2023. President Joe Biden signed an AI executive order that, among other things, requires AI companies to disclose their development and safety testing plans to the Department of Commerce. However, these disclosures are not required to be made public—potentially preventing the broader oversight that the letter’s signatories want.
-With additional reporting by Will Henshall/Washington