Is artificial intelligence paving the way for biological weapons?

While the opportunities offered by artificial intelligence technologies in the fields of health and biotechnology are increasing every day, the possibility of these same technologies being used for biological weapon development is causing serious concern in the scientific community. A comprehensive assessment published in Nature journal reveals that AI-powered biological design tools have reached the capacity to design many biological agents, from deadly toxins to next-generation viruses, while scientists discuss the measures that need to be taken against biosecurity risks. 

While the opportunities offered by artificial intelligence technologies in health, drug development, and biotechnology fields are increasing every day, the possibility of these same technologies being used for biological weapon development has become the center of a growing debate in the scientific world. Especially the risks that could arise if AI systems used in the design of viruses, toxins, and other biological agents fall into the hands of malicious actors have alarmed scientists and policymakers. 

A comprehensive assessment published in Nature journal brought up the question of whether biological artificial intelligence tools pose a new threat to humanity. At the heart of the discussions is an AI system developed by Chinese researchers, used to design conotoxins which can possess deadly properties.

A snail's venom is being redesigned with artificial intelligence

One example highlighted by scientists is the extremely potent toxins produced by cone snails. The stings of cone snails, living in the seas, contain a complex mixture of venom composed of proteins called conotoxins, which can affect the nervous system. The stings of these creatures contain a cocktail of small proteins called conotoxins, some of which can block ion channels in the nervous system. There is no known antidote for this venom. 

On the other hand, not all of these molecules are harmful. In fact, some can be used in medical treatments. Some drugs used in chronic pain treatment are directly derived from conotoxins. However, due to the strict control of research on dangerous conotoxins in many countries, the announcement by Chinese researchers in 2024 that they designed new conotoxins using artificial intelligence raised serious questions in biosecurity circles. 

The study in question was considered a potential biosecurity risk

In private correspondence obtained by Nature, a high-ranking official in the US government was revealed to have assessed the study as a potential biosecurity risk. The official stated that it was particularly concerning that the system used was based on an open-source protein language model developed by US scientists. 

However, Weiwei Xue, a computational chemist from Chongqing University in China and one of the study's co-authors, argues that the criticisms are unfair. Xue states that their work aims directly at drug development, saying:

“In our lab tests, we obtained some conotoxins with therapeutic potential. It is important to consider the possibility of misuse of the artificial intelligence tool, but the system was not developed to produce harmful proteins.”

The researcher also emphasizes that advanced expertise and expensive laboratory infrastructure are required to produce the designed molecules in the real world. 

The scenario that keeps scientists awake at night

Nevertheless, many experts believe that biological artificial intelligence tools could have much more serious consequences in the long term. Martin Pacesa, a structural biologist at the University of Zurich in Switzerland, points out that, theoretically, extremely deadly and difficult-to-detect toxins could be produced.

Pacesa expresses his concern with these words:

“This is exactly what keeps me awake at night. Theoretically, a person could develop agents at the level of ricin or other lethal agents that are almost impossible to detect.”

According to experts, artificial intelligence could make biological weapon production more accessible. It is particularly considered that even individuals without advanced biology training could learn complex processes by utilizing large language models and biological design tools. 

David Baker, a computational biophysicist from the University of Washington in Seattle, who shared the 2024 Nobel Prize for his pioneering work on protein design, states that they have always made an assessment that the benefits provided to the world far outweigh the dangers. Baker adds that as capabilities increase, this will be an important question to continue to ponder.

Some, however, argue that instead of trying to prevent these by imposing software restrictions, the focus should be on detecting and countering AI-driven biological weapon attacks. Timothy Jenkins, a protein designer from the Technical University of Denmark in Lyngby, states that, in his opinion, that train has already left the station.

The biggest fear: AI-powered pandemic viruses

The scenario researchers focus on most is the emergence of new pandemic viruses.

James Black, a biosecurity researcher and visiting academic at Johns Hopkins University in Baltimore, Maryland, says there are two main threats concerning artificial intelligence and biological weapons. The first is individuals learning how to produce existing biological threats like anthrax using chatbots. The second is states or well-funded terrorist organizations designing new biological weapons using advanced AI systems. 

According to experts, the most realistic threat is not the creation of a new virus from scratch, but rather making existing viruses more dangerous.

Doni Bloomfield, a law professor working on biosecurity at Fordham University in New York, states that some AI tools used today were actually designed to predict virus evolution and develop vaccines, but theoretically could also be used for malicious purposes. The possibility of increasing the ability of viruses like SARS-CoV-2 or influenza to evade the immune system particularly worries experts.

In a preprint study published in 2025, new virus genomes were designed using artificial intelligence, and approximately 5% of these lab-produced viruses were found to be functional. Although these viruses were designed to infect bacteria rather than humans, the study was noteworthy for demonstrating the level of biological design capability reached by artificial intelligence. 

No doomsday scenario yet

However, not all experts agree equally on disaster scenarios.

According to a 2025 report by the US National Academies of Sciences, Engineering, and Medicine (NASEM), there are significant technical barriers preventing artificial intelligence from developing pathogens capable of creating pandemics from scratch today. The report emphasizes the lack of high-quality data revealing the relationship between characteristics such as virus transmissibility or lethality and their genetic structures. Furthermore, laboratory tests and verification processes still largely require human expertise.

Brian Hie, a computational biologist from Stanford University, reminds that mutation techniques already used in biology for decades also carry similar risks.

David Baker, the 2024 Nobel laureate protein designer, also believes that the benefits of biological artificial intelligence currently outweigh its risks. Baker makes the following assessment: “The benefits provided to the world far outweigh the dangers. However, it is also clear that as capabilities increase, this is an important question we need to continue to ponder.”

Can artificial intelligence elevate non-biologists to expert level?

One of the most critical topics of discussion is whether artificial intelligence can replace human expertise.

Research published by Seth Donoughe, AI director at SecureBio, and his colleagues, showed that advanced large language models can bring the performance of individuals with limited biology training closer to the level of PhD-level researchers in some tasks. Particularly striking results were obtained in tasks such as solving problems in virology experiments or generating code for laboratory robots.

Donoughe warns: “Artificial intelligence is becoming increasingly capable in every task we put before it. We should expect both doing good things and doing bad things to become easier.”

In contrast, other research indicates that novice individuals using artificial intelligence have not yet shown a significant advantage over internet-using volunteers in complex tasks like DNA manipulation or virus production. For this reason, some experts believe that existing risks might be exaggerated.

Defense line: DNA synthesis companies

The majority of experts argue that the most effective way to prevent biological threats is not to restrict artificial intelligence, but to control the production of biological materials.

According to researchers, the production of proteins or synthetic genomes designed with artificial intelligence often requires ordering from DNA synthesis companies. A significant portion of these companies use screening systems that detect genetic sequences which could lead to toxin or pathogen production.

However, a study published in 2025 by Microsoft researchers revealed that these systems could occasionally be bypassed with the help of artificial intelligence. The researchers designed 76,000 synthetic biological constructs, demonstrating that some of them could evade existing screening systems. This rate was significantly reduced with subsequent updates, but experts emphasize that the race continues.

Screening controls

According to many scientists, the best way to prevent the development of AI biological weapons is to detect malicious actors during the virus or toxin production phase. “After all, most of the time it doesn't matter what you do on the computer,” says Pacesa. “What matters is how this is turned into a real, physical protein or small molecule.”

The team led by Eric Horvitz and Bruce Wittmann utilized open-source protein design tools to redesign 72 biological molecules that could pose a biosecurity risk. The researchers designed approximately 76,000 synthetic homologs, ensuring that these molecules retained their potentially dangerous functions while also having sufficiently different genetic structures to evade existing screening systems.

The results of the study were striking. Approximately a quarter of the designs developed in the initial phase could not be detected by existing DNA screening software. Later, as companies updated their software, this rate decreased to approximately 3%.

However, a new preprint study published in March showed that if these sequences are broken down into very small fragments, their detection becomes difficult again.

Horvitz and his colleagues chose not to share the details of the designed molecules with the public, stating that they provided controlled access for scientific research.

Making these molecules functional is not as easy as thought

Experts also emphasize that redesigned toxins may not always lead to dangerous outcomes.

In follow-up research to Horvitz's study, a team led by Bruce Wittmann and Elizabeth Strychalski tested the functions of some redesigned proteins in a laboratory setting. Experiments showed that some simple proteins could retain their functions, while enzymes experienced significant loss of function.

Eric Horvitz stated that this result demonstrates the complexity of biological systems, commenting, “This shows how complex and difficult it is to make them functional.”

Screening systems are still a strong line of defense

James Diggans, Vice President of Biosecurity and Policy at California-based DNA synthesis company Twist Bioscience, argued that existing security mechanisms are still effective. Diggans stated, “Currently, even if you are using these artificial intelligence tools in earlier stages of the process to evade detection, screening applications still serve as a highly effective stronghold against misuse.”
Nevertheless, Diggans acknowledged that AI models are rapidly evolving, and more advanced systems in the future may have an increased capacity to bypass existing protection mechanisms.

Screening systems are not mandatory in many countries, including China

Researchers are working on new screening technologies that will allow DNA synthesis orders to be evaluated not only based on genetic sequences but also on the structure and functions of the molecules to be produced.

However, there are significant differences in practices worldwide.

In the US, scientists benefiting from research funds may soon be required to place orders only with companies using screening software. The United Kingdom, the European Union, and New Zealand are also among the countries considering similar regulations.
In contrast, screening practices are not yet mandatory in China, which accounts for over 30% of global DNA synthesis orders. Weiwen Zhang, a synthetic biologist from Tianjin University, states that the Chinese government recommends screening practices to companies but does not mandate them.

Tessa Alexanian, working at the Switzerland-based International Biosecurity and Biosafety Initiative for Science, states that it is still possible to order toxin sequences from suppliers in many parts of the world without triggering any alarms.

Desktop DNA synthesis devices could pose new risks

Another source of concern for experts is "desktop DNA synthesis devices".

These devices, currently capable of producing only short sequences, are expected to reach the capacity to generate longer genetic fragments in the future. Researchers emphasize that this development could democratize biological production processes but also complicate oversight.

Tighter security boundaries proposed for artificial intelligence models

Another topic of discussion in the scientific community is the imposition of stronger security boundaries on artificial intelligence models used in biology. David Baker, a pioneering figure in protein design, states that they conduct regular risk assessments before publishing AI tools developed in his laboratory.

The “responsible AI and biodesign principles” published by Baker and other researchers in 2024 also support this approach. However, many experts disagree on whether self-regulation by the scientific community is sufficient.

OpenAI and other companies implement biosecurity measures

Companies developing general-purpose artificial intelligence systems, meanwhile, implement security measures against requests related to biological weapons or chemical attacks. OpenAI's published safety principles emphasize that models should not provide actionable instructions for the development of chemical or biological weapons.

Despite this, some research shows that security boundaries are not always effective. A study conducted by Donoughe and his team found that approximately 90% of participants could access high-risk biological information through large language models.

Meanwhile, a New York Times report alleged that a person arrested in India on suspicion of planning a terrorist attack obtained information on ricin toxin production from ChatGPT and AI-powered search tools.

OpenAI officials stated that the information in question was already openly accessible on the internet.

Researchers managed to bypass Evo 2's security boundaries

Specialized biological artificial intelligence systems are also at the center of similar discussions. In the training of Evo 2, a genomic language model trained on 128,000 genome sequences, viruses that infect humans were specifically excluded.

However, Le Cong and his team from Stanford University managed to bypass Evo 2's security boundaries using a general-purpose AI agent, having the model design new versions of SARS-CoV-2 and HIV-1 proteins.

Another study revealed that fine-tuning using publicly available genome data of human-infecting viruses enabled the model to regain these capabilities. Patrick Hie, one of Evo 2's developers, stated that this situation was not surprising and argued that the open-source approach contributes to security research.

Access restrictions or open science?

One of the biggest disagreements among scientists concerns whether access to powerful biological artificial intelligence systems should be restricted.

Jassi Pannu and colleagues from Johns Hopkins University suggest that new data related to genetic changes that could lead to pandemics should be subject to controlled access. The pandemic preparedness engine developed by the Coalition for Epidemic Preparedness Innovations (CEPI), described as “ChatGPT for vaccine developers,” is also planned to be accessible only to approved users. Similarly, the GPT-Rosalind model announced by OpenAI will also be available only to verified researchers.

Proposal for tiered access based on risk level

The Nuclear Threat Initiative, an organization, proposes a risk-based access model for artificial intelligence systems.

A report published by RAND Corporation in 2025 stated that 23% of the 57 biological AI tools evaluated fell into the high-risk group. The report prepared by the National Academies similarly recommends predetermining which measures will be implemented when risky developments emerge, using an "if-then" approach.

Artificial intelligence can also be a defense tool

Experts also emphasize that despite all the risk discussions, artificial intelligence offers great opportunities for biological defense. Patrick Hie states that biological artificial intelligence will play a significant role in developing defenses against natural epidemics and malicious actors in the future.

Roman Woelfel points out that artificial intelligence systems capable of designing toxins can also develop antitoxins and new treatment methods. In ongoing work with NATO, systems are being developed for the rapid detection of artificially designed proteins in suspicious samples using mass spectrometry.

Private companies operating in this field are also noteworthy. US-based companies Red Queen Bio and Valthos received investments of 15 million and 30 million dollars respectively, to develop biological defense technologies.

Both very uncertain and very urgent

Experts emphasize that the effects of artificial intelligence on biology cannot yet be fully predicted. Jassi Pannu states that the potential for misuse of artificial intelligence is still unknown, while Tessa Alexanian summarizes the current situation with these words:

“Many aspects of this field appear to me to be both highly uncertain and highly urgent.”

The growing debate in the scientific world continues to bring to the forefront the question of whether artificial intelligence will offer great opportunities for humanity in the field of biology or bring forth new-generation biological threats.

For Details