Understanding AI-Enabled Biological Threats: Hype, Hazard, and Governance

First of all, it’s necessary to clarify what kind of AI it is we’re talking about, not just for the sake of precision, but to move away from the tendency to imbue AI with near-magical qualities.”Dr Lalitha Sundaram

The most worrying prospect would be for a mal-intentioned actor to use AI to aid in the development of biological weapons.⁴ While this kind of scenario obviously captures the imagination (terrifyingly!), it needs careful unpacking across a number of dimensions, technical and otherwise. 

First of all, it’s necessary to clarify what kind of AI it is we’re talking about, not just for the sake of precision, but to move away from the tendency to imbue AI with near-magical qualities. Even for a single type of tool, assessments can vary. On the purely technical level, assessments of Large Language Models for example, have ranged from significantly raising the alarm⁵ to being somewhat similar to a baseline of un-augmented internet searching.⁶ Aside from LLMs, there has also been a proliferation of Narrow Biodesign Tools with relevance for Engineering Biology.⁷ Some are more capable than others, and they each have strengths and limitations - both in terms of utility to Engineering Biology but also utility to a would-be bioweapons developer. 

Conversely, some stages of biological weapons development may be more amenable to “barrier lowering” through AI than others. As previous attempts by States and Non-States to develop biological weapons have shown us, the process is a complex one, and the utility of AI in the different stages will depend on the stage, the actor(s) involved, their existing capabilities, and their absorptive capacity. Moreover, while many assessments have looked in particular at the ‘design’ stage (and this is where AI could likely have an impact), you cannot get away from the need for iterative testing ‘in the real world’. And the transition to the physical world is a significant pinch point. 

Though it has been written about for decades,⁸ the importance of tacit knowledge in biological weapons production has often been overlooked. Having a jail-broken LLM spit out instructions is unlikely to be sufficient to create a usable biological weapon. How might new and more interactive modes of instruction impact this, though? There is a physicality to life science practice that is hard to automate. But we are trying, and for very good reasons of standardisation, reproducibility, efficiency and safety. So, thinking beyond the crude picture of a chatbot giving a list of instructions, we need to think about the developments in AI-assisted training across the digital-physical barrier, as well as developments in EngBio to do with standardisation and automation, and how they might erode traditional tacit knowledge barriers.⁹

The main international agreement dealing with the weaponisation of the life sciences is the Biological and Toxin Weapons Convention. However, given the many applications of Engineering Biology in the production of chemicals, the Chemical Weapons Convention also applies, especially when we consider the place of toxins. In fact, one of the earliest papers that ‘raised the alarm’ about the incorporation of AI tools into biotechnology was about the potential for misuse and development of potential chemical weapons.¹² While there is debate about the real-life utility of such models (plenty of toxins are already well-known, and the tool did not address issues such as stability), the fact that it successfully generated these predictions does rightly draw attention to the capabilities afforded by this kind of technology convergence. 

Do the BTWC and CWC “apply” to the convergence of AI and the life sciences? Both are clear: they are about purpose not about specific agents or technologies (this is often referred to as the ‘general purpose criterion’). Thus, the use or not of AI is immaterial as to whether the Conventions “apply”: no matter what the means, what is proscribed in the BTWC is whether a biological weapon has been “develop[ed], produce[d], stockpile[d] or otherwise acquire[d] or retain[ed]”.¹³ However, the need for the Conventions to incorporate processes and mechanisms to keep pace with emerging technologies has been widely-recognised. The CWC has a Technical Secretariat that is examining the impact of AI,¹⁴ and the Working Group of the BTWC is considered a science and technology review mechanism.¹⁵

2025-04-biosecurity-fig2.jpg

On the flip side, AI tools may well assist in implementation of the Conventions in surveillance, detection, verification, or attribution. Not only that, they may hold promise for other aspects of dealing with biological risk, such as designing decontamination tools or, in ways that intersect with developments in health and infectious disease risk management, such as the development of medical countermeasures.

At CSER, we have organised a set of seminars intending to create a shared understanding of the realistic risks and appropriate safeguards to address the rapid evolution of AI capabilities as applied to the life sciences. Our seminars have explored technical questions, national perspectives and evaluations, and options for international governance.