Rethinking the De-skilling Narrative in AI and Biological Weapons Policy

Concerns that artificial intelligence (AI) could make biological weapons development easier have become increasingly prevalent. These warnings largely reprise a longstanding narrative that new technologies lower barriers to biological weapons development, making them cheaper and more accessible to untrained groups or individuals. Since the 1980s, each innovation—including synthetic biology, DNA synthesis, CRISPR, and now AI tools—has been cast as having a “de‑skilling” effect that erodes traditional requirements of expertise and infrastructure. Yet, historical evidence from both state and non-state bioweapons programs, as well as more recent scientific research projects, suggests that the primary obstacles to successful outcomes have not been in access to technology. Rather, they have stemmed from building and sustaining the necessary reservoirs of tacit knowledge, overcoming integration challenges across scientific, engineering, and organizational domains, and adapting existing protocols and technologies to local knowledge bases and industrial or laboratory infrastructures to turn ideas into operational capabilities.

Current assessments of new technologies in bioweapons development ignore these essential elements and instead focus on a single variable or stage of a bioweapon’s lifecycle, thereby drawing conclusions that mischaracterize threat potentials. Such mischaracterizations, and their resulting exaggerations, produce policies that may not only limit the use of new technologies for beneficial medical or industrial purposes, but also result in costly and ineffective programs. Several biosecurity projects designed in response to the events of 9/11 exemplify the latter consequence. Correcting these conceptual mistakes is essential to producing more accurate threat assessments, which foster the scientific and technological advancements enabled by these novel technologies. This article first highlights the common mistakes made in appraising the threat potentials of new technologies and then proposes recommendations on achieving more accurate threat assessments and developing appropriate responses.

Common Mistakes in Current Threat Assessments

Overstating the Power of Explicit Knowledge

Among the most common analytical weaknesses of current assessments is the disproportionate weight they place on explicit information—open publications, online protocols, and now AI-generated stepwise “recipes,” treating access to such material as a proxy for technical skills. Detailed analyses of bioweapons programs show that protocols, however detailed, are insufficient to carry out an experiment successfully because explicit information is an incomplete representation of its authors’ knowledge. It only captures the data that can be expressed in words or formulas and omits the tacit knowledge—skills, know-how, or practical ways of doing things that cannot be articulated in words—that has helped create a technological artifact or carry out a scientific experiment. Two illustrative examples, the Soviet anthrax and smallpox weapons, demonstrate that explicit data cannot be used effectively without access to the corresponding tacit knowledge.

Both weapons were designed in a Russian institute and were sent to be produced in different facilities. Although the producing facilities received extensive protocols and samples of the bioagents, their personnel were unable to produce the weapons without on-site contribution from the original authors of the weapon. The production of the Soviet anthrax weapon at the Stepnogorsk facility in Kazakhstan took five years to achieve, owing both to gaps in knowledge and to the challenges of adapting the protocol to the new site. The gaps in knowledge were addressed only when a team of scientists from the Russian facility that had designed the weapon came to Stepnogorsk to work alongside their colleagues. However, adapting the protocol to the Stepnogorsk infrastructure, the type of knowledge available onsite, and local environmental conditions required three additional years of experimentation and testing. Notably, the resulting weapon was different from the original designed in Russia.

This example highlights another important weakness of current assessments—it assumes that protocols, recipes, and other scientific documents are universal and can be used without modification. However, explicit data must be adapted to the specific environment of use to account for local environmental, technological, and knowledge conditions, which necessarily differ from those of the laboratory in which the document was produced. For example, differences in water pH, equipment, reagents, and local expertise can introduce variables that standard protocols do not account for. Therefore, when a protocol is used in a location different from the one in which it was produced, it needs to be modified to take the local circumstances into account.

Using Narrow Definitions of Tacit Knowledge

Similarly, it has been argued that AI tools can overcome the tacit knowledge barrier by providing stepwise instructions and helping users troubleshoot difficulties. Yet, this argument overlooks a critical point: if users do not have sufficient initial domain-specific knowledge and skills, they will be unable to evaluate the accuracy of AI‑generated guidance and solutions. Recent experiments demonstrate that AI-generated protocols and troubleshooting instructions can contain critical errors, which a non-expert would be unable to identify. Even error-free instructions would be challenging to use because such instructions are just explicit data and do not confer necessary laboratory skills nor the expertise required to adjust for local conditions. Although experts are better positioned to interpret AI‑generated data, they would still need to validate it in the laboratory. Thus, while AI tools may help experts diagnose and address problems in biological experiments, they do not eliminate the need for further hands-on experimentation and troubleshooting.

Analysts have attempted to circumvent this challenge by defining tacit knowledge narrowly as a set of verbalizable techniques or laboratory tips that can be readily converted into explicit knowledge through improved tools. Although there is indeed a form of tacit knowledge that can be made explicit when scientists work side-by-side and communicate directly, tacit knowledge cannot be reduced to that single form. There are, in fact, several reservoirs of tacit knowledge, each containing a form of knowledge not contained in the others.

For example, individuals serve as reservoirs of personal knowledge, a form of expertise and skill unique to each person and accumulated through years of personal practice. Some of this personal knowledge can be entirely unconscious, such that a person may be unaware that they are performing certain actions in ways that are essential for experimental success. The transfer of this knowledge to another individual requires direct, prolonged side-by-side work, often without either party realizing that important skills have been transmitted or acquired. Furthermore, teams of scientists and technicians working on a common project are reservoirs of communal knowledge, a distinct form of knowledge created by the team that exceeds the sum of individual knowledge and that no single member fully possesses. Therefore, when teams are separated, individual members retain only their specific, personal knowledge, while the communal knowledge tends to disappear. There is also tacit knowledge embedded in organizational or corporate culture, understood as a set of practices and ways of reasoning that shape experimental outcomes. This extends to the structure of work itself, which is more than a succession of tasks; it includes mechanisms that enable—or, when absent, hinder—the creation, transmission, and use of tacit knowledge.

Collectively, these forms and reservoirs of tacit knowledge highlight a key point—no single individual can possess all forms of tacit knowledge. Yet, all are necessary to replicate past work or adapt it to a different purpose because they each contributed in specific ways to the design and production of a technical artifact, and mutually reinforce one another, much as different disciplines contribute to different aspects of a technology. When one or several of these reservoirs of knowledge are unavailable or lost due to decay, replication of past work becomes very difficult or impossible, including for experts in the field, as demonstrated by the Soviet anthrax and smallpox weapons cases cited earlier. A group or country could strive to create these different forms of knowledge over time, but their ability and the speed at which they can do so depend on the type and breadth of expertise available at the outset and their ability to create a work environment that enables learning. However, most covert programs start with little or no relevant expertise and typically adopt organizational and managerial models that hinder learning due to the added constraint of preventing detection. Therefore, it is important to consider the full range of tacit knowledge available to an individual or group when assessing the threat potential of new technologies for bioweapons development.

 Limiting Assessments to the Early Stage of Bioweapons Development

Another important weakness of current assessments of the role of new technologies in bioweapons development is their focus on the first stage of development, that is, how such technologies might enhance the capacity of a lone actor or small group to design a bioagent. Yet, a bioagent, whether designed by humans or by AI, is not a weapon; it must undergo various stages of processing and scale‑up before a dissemination mechanism can be developed. Each stage of development not only requires different types of expertise and skills but also exposes the bioagent to environmental changes and contamination, potentially compromising the whole endeavor.

Both the US and the Soviet bioweapons programs have, at times, faced insurmountable contamination problems as well as bioagents that failed at intermediate stages of the process. The sensitivity of bioagents to their handling and environmental conditions only reinforces the importance of having the skills required for each stage of development. Since most current assessments focus on a narrow portion of a bioweapon’s lifecycle, they cannot fully appraise the challenges that an individual or even a small team would face when lacking the full range of skills required for agent selection, production, scale‑up, stabilization, testing, dissemination, and weaponization.

By ignoring the many variables that affect bioweapons development, these assessments also understate the integration challenge involved in aligning these elements over time and the effects that disruptions caused by a lack of skills or inadequate supplies or resources can have on a program’s outcome. The result is an account of biological weapons development that overestimates the significance of access to information and technology and underestimates the difficulty of constructing and sustaining the sociotechnical systems that offensive programs require, namely the breadth and relevance of skills and expertise needed at each stage of bioweapon development, as well as the organizational and managerial arrangements that support system integration and learning.

Implications and Policy Recommendations

The first key implication from the preceding discussion is the need to shift from the current analytical model that isolates a single variable or stage of bioweapons development. The impact of new technology or data should be evaluated within the context of the entire bioweapon development, production, and weaponization process. This evaluation should address key questions including which stage of development the technology facilitates or accelerates, whether its effects extend beyond that stage, what complications it may introduce, whether it requires specialized expertise, and what might complicate the acquisition and use of this new expertise. Accordingly, analysts and policymakers should move from speculative claims about what a new technology could in theory do and instead undertake a more rigorous analysis of how a technology is actually used, how it evolves over time, and what challenges it poses in order to understand what, if any, effect it has on the overall lifecycle of a bioweapon.

Policy responses should, in turn, focus on key barriers to bioweapon development: the acquisition of expertise and the integration of various stages and parts of a program. In this context, policies should create disruptions that hinder a group or state’s ability to advance a bioweapons program. Existing policy tools provide a strong foundation to this end. For example, export controls pose significant integration challenges for covert programs because proliferants must acquire technology and materials from a variety of sources that may be incompatible with one another or poorly suited to the program’s needs. As a result, such controls force proliferants to conduct further research and experimentation, which exposes gaps in knowledge, creates additional technical and organizational difficulties, and can delay a program or even prevent it from advancing altogether if integration challenges cannot be resolved. Therefore, expanding assistance programs, such as the State Department EXBS and projects under UNSC 1540, that support the development of effective export control systems in countries that have limited or no export controls is of utmost importance.

The Biological Weapons Convention (BWC) provides another essential tool and requires urgent reinforcement. By adopting a verification mechanism for the treaty that includes on-site inspections, BWC members could more effectively disrupt covert programs, even if inspections do not catch violators in the act. Historical cases—including Iraq, the Soviet Union, and Aum Shinrikyo—suggest that the mere threat of detection due to a pending inspection or police activity can lead a program to stop its activities temporarily, move its equipment elsewhere, and potentially destroy documents and portions of its stock of bioagents. These disruptions would affect any weapons program, but they are particularly consequential for bioweapons development given the sensitivity of bioagents to changes in the environment.

Finally, it is necessary to resist the instinct to treat regulation as the initial response to potential technological misuse. Such measures are often based on fears and perceived vulnerabilities rather than empirical data. This is not to suggest that technologies such as AI are benign. However, if regulations are to be developed, they should be based on deep and continuous analysis of the technology. Such analysis should be grounded in empirical data gathered by observing what scientists who use AI tools in their experiments can do with them, what challenges they face, how they solve them, and what AI can and cannot do in real-life science. The National Academies of Sciences or the NIH could work with these scientists to better understand the role of AI in science. The current approach of beginning with worst-case scenarios risks obscuring the challenges that real-world users might have in absorbing and implementing AI-generated instructions. Understanding these challenges, alongside the strengths and limitations of AI tools, will better inform assessments of the actual risks they pose and support the development of appropriately tailored policies.

. . .

Sonia Ben Ouagrham-Gormley is an associate professor and deputy director of the Biodefense Graduate Program in the Schar School of Policy and Government at George Mason University. She holds a PhD in Economics of Development and graduate degrees in Strategy and Defense Policy and Economics and Law. Her research focuses on issues related to biodefense, WMDs, and the impact of emerging technologies on security and health.

Image Credit: CDC worker in the maximum containment virology laboratory by Centers for Disease Control, PDM 1.0, via Wikimedia Commons