US company Anthropic announced precautionary account bans after detecting suspicious attempts to use its models in research that could contribute to the development of biological weapons, as well as in designing conventional weapons and conducting surveillance and disinformation activities, according to a report examining the misuse of its artificial intelligence technologies over the past eight months.
The company said it could not determine whether the account holders’ aims were research-related or malicious, noting that the nature of biological research makes it difficult to distinguish between peaceful and harmful uses, as the same tools and methods can support vaccine development or be exploited to engineer biological weapons.
Anthropic did not disclose the names of the researchers or the banned institutions or countries linked to them, nor did it identify the biological agents examined in the research. Jacob Klein, the company’s threat research lead, said some previously hypothetical forms of misuse had become real.
Dual-use biological research
In one case, a researcher said he wanted help preparing a grant application to study a deadly mosquito-borne virus. The company said the project was concerning because it was planned for implementation at a military research institution, while acknowledging that similar research could also be used to develop more effective vaccines.
In another case, a researcher outside the United States used the “Claude” tool for weeks to conduct experiments involving highly pathogenic strains of avian influenza. Anthropic said the researcher had tried to conceal his actual purpose behind stated research objectives, without disclosing his identity or the organization he worked for.
Software for designing conventional weapons
Monitoring dissidents and deceptive propaganda materials
Anthropic said some cases of misuse were suspected of being linked to the Chinese and Iranian governments and targeted dissidents and expatriate communities for surveillance purposes. The company also accused Russian state media outlets of using “Claude” to produce deceptive propaganda materials presented as independent news coverage, including fabricated claims about elections in Moldova.
External review of the report
The company said the incidents detailed in the report, which runs to more than 150 pages, did not involve its latest model, “Mythos,” access to which is limited to trusted partners inside the United States.
The report underwent an external review by Andrew Weber, a researcher at the US Strategic Risk Council and former US Department of Defense official, and Susan Monarez, a microbiologist and public health expert who previously oversaw a review of US national preparedness in the field of biosecurity.
