The CEOs of Several major artificial intelligence companies are urging members of Congress to pass new laws that would make it harder for bad actors to develop biological weapons using their technology.
Demis Hassabis of Google DeepMind, Sam Altman of OpenAI, Dario Amodei of Anthropic and Mustafa Suleyman of Microsoft AI are among the signatories to an agreement. public letter calling for laws requiring companies that sell synthetic DNA and RNA to vet customers and orders to prevent misuse of genetic material.
Organized by the nonpartisan Institute for Progress and the right-wing Foundation for American Innovation, the letter acknowledges that given the pace of AI development, “there is a real possibility that the knowledge barriers that have historically prevented bad actors from obtaining biological weapons will significantly erode.”
Scientist Arthur Kornberg was the first to successfully synthesize DNA in the 1950s. Today, the process is automated, and dozens of companies around the world use commercial synthesizers to “print” and sell custom genetic sequences used for scientific research, drug development, and diagnostics. Many suppliers sell only to qualified researchers, biotechnology companies, and educational institutions, but not all verify customers or the genetic sequences they order.
In 2017, Canadian researchers alarm triggered when they used $100,000 worth of mail-order DNA to reconstitute the extinct smallpox virus. Critics said the same methodology could be used to construct smallpox, a closely related and deadly virus. Since then, genetic synthesis has become cheaper.
Combined with advances in AI, it is now possible to design dangerous new toxins and pathogens using large language models, although some training in biology would likely still be required to create a functional virus from scratch. Although bioterrorism attacks are rare, they have the potential to cause mass casualties, public panic, and economic losses. A major concern is that an AI-engineered pathogen could trigger, intentionally or unintentionally, a global pandemic.
“AI tools allow a user to very quickly identify where to turn to order sequences that will not be subject to control,” explains David Relman, a microbiologist and biosecurity expert at Stanford University, who signed the letter. “If prompted appropriately, they can also tell you how to change the nature of your order, so that even those doing the checking are much less able to detect what you’re trying to prepare.”
The signatories include other scientists, national security experts and executives from the gene synthesis companies Twist Bioscience and Ansa Biotechnologies. These companies are members of the International Gene Synthesis Consortium, created in 2009 to implement voluntary screening practices. Many companies already use software to screen orders for “sequences of concern” that may contribute to an organism’s toxicity or ability to cause disease.
“If you have technology that can synthesize DNA, then you need to make sure it’s used responsibly, including making sure you understand what you’re making and who you’re making it for,” says James Diggans, vice president of policy and biosafety at Twist Bioscience. The company has supported the implementation of formal rules for years.
Federal guidelines introduced under the Biden administration required scientists and companies that receive federal funding to order synthetic genetic sequences from suppliers who select purchases. A bipartisan bill introduced earlier this year in the Senate would require all gene synthesis suppliers operating in the United States to screen orders and customers for bad actors or dangerous pathogens.
But screening tools are not perfect. Last year, Microsoft researchers published a study showing that AI protein design tools were capable of generating potentially dangerous genetic sequences that evaded companies’ screening software. The models suggest new protein sequences with structures similar to those known to be dangerous.
Geoff Ralston, former president of Y Combinator and partner at the Safe AI Fund, thinks AI labs with biological models should do their own user screening.
“It should be very difficult, if not impossible, to ask a model to help you do something dangerously imminent,” says Ralston, who also signed the letter.
Relman agrees that regulations regarding screening procedures are only part of the solution. “Since screening can fail in some cases, then you need to have other checkpoints,” he says. “This is where AI companies are going to need to step up their efforts.”
