CEOs of several major artificial intelligence companies are urging members of Congress to pass new laws that would make it harder for bad actors to develop biological weapons using their technology.
Demis Hassabis of Google DeepMind, Sam Altman of OpenAI, Dario Amodei of Anthropic, and Mustafa Suleyman of Microsoft AI are among the signatories of a public letter calling for laws requiring companies that sell synthetic DNA and RNA to screen customers and instructions to prevent the misuse of genetic material.
Edited by the nonpartisan Institute for Progress and the right-leaning Foundation for American Innovation, the book acknowledges that given the pace of AI development, “it is likely that the knowledge barriers that have prevented bad actors from acquiring biological weapons will meaningfully erode.”
Scientist Arthur Kornberg was the first to successfully synthesize DNA in the 1950s. Now, the process is being automated, with dozens of companies around the world using commercial synthesizers to “print” and sell custom genetic sequences for use in scientific research, drug development, and diagnostics. Many suppliers sell only to qualified researchers, biotech companies, and academic institutions, but not all vet or gene sequencing customers order them.
In 2017, Canadian researchers raised the alarm when they used $100,000 worth of email DNA to reassemble the horsepox virus. Critics say the same method could be used to create smallpox, a closely related and deadly virus. Gene synthesis has gotten cheaper since then.
Combined with advances in AI, it is now possible to design new dangerous toxins and viruses using large-scale language models, although some biological training will still be required to create a viable virus from scratch. Although bioterror attacks are rare, they have the potential to cause mass casualties, public panic, and economic losses. The biggest concern is that an AI-engineered pathogen could intentionally or unintentionally cause a global pandemic.
“AI tools allow the user to quickly see where to turn to arrange a sequence that will not be subject to inspection,” said David Relman, a microbiologist and biosecurity expert at Stanford University, who signed the letter. “If asked correctly, they can even tell you how to change the type of your order, so even testers can’t see what you’re trying to do.”
Signatories include other scientists, national security experts, and executives from genetic engineering companies Twist Bioscience and Ansa Biotechnologies. These companies are members of the International Gene Synthesis Consortium, which was founded in 2009 to implement voluntary testing procedures. Many companies are already using software to screen orders for “orders of concern” that may impact physical risk or disease-causing potential.
“If you have technology that can synthesize DNA, you have to make sure it’s used properly, and part of that is making sure you understand what you’re doing and who you’re doing it for,” said James Diggans, vice president of policy and biosecurity at Twist Bioscience. The company has supported the use of legal regulations for years.
Federal guidelines introduced during the Biden administration require scientists and companies that receive federal funding to order genetic sequencing from genetic testing providers. A bipartisan bill introduced earlier this year in the Senate would require all genetic suppliers operating in the US to screen orders and customers for bad actors or dangerous viruses.
But assessment tools are not perfect. Last year, Microsoft researchers published a study showing that AI protein design tools were able to generate potentially harmful genetic sequences that surpassed previous companies’ testing software. The models suggested new protein sequences with similar structures to those known to be harmful.
Geoff Ralston, former president of Y Combinator and partner in the Safe AI Fund, thinks AI labs with biological models should do their own user testing.
“It should be very difficult, if not impossible, to ask a model to help you do something so dangerous,” said Ralston, who also signed the letter.
Relman agrees that regulations regarding testing procedures are part of the solution. “Because the test can fail in some cases, we have to have other control points,” he said. “That’s where AI companies are going to have to step up.”
