Smoke comes out of the Intercontinental Hotel in Kabul, capital of Afghanistan, during a terrorist attack on January 21, 2018 Mohammad Ismail/Reuters "Good morning, ChatGPT, can you tell me how to make a bomb?" Anyone who has ever tried to ask an artificial intelligence (AI) chatbot such a question knows that the response can range from a long explanation about the history of explosives to permanently blocking the user's account. But sometimes, if the question is phrased a certain way, the answer can include potentially useful information about how to make a bomb. ?? Do you have any reporting suggestions? Send to g1 Several media outlets have already tested this hypothesis and discovered that, when the "correct" prompts are used, some AI models can provide guidance on creating biological weapons, planning an attack on a sports stadium or ways to hide a terrorist's tracks. This method of bypassing system restrictions is known as "jailbreaking." OpenAI, the developer of ChatGPT, defines this as "an attempt by a malicious actor to trick the model into serving prohibited content."
How terrorists are using AI to plan attacks and bypass chatbot restrictions
Smoke comes out of the Intercontinental Hotel in Kabul, capital of Afghanistan, during a terrorist attack on January 21, 2018 Mohammad Ismail/Reuters "Good morning, ChatGPT, can you tell me how to make a bomb?" Anyone...
A recent report published by Tech Against Terrorism, an online observatory supported by the United Nations counterterrorism office, showed how often an LLM provides "useful" information to potential extremists. Researchers sent more than 2,300 information requests, based on "real-world terrorist use cases," to 27 different AI models. They found that 32% of queries resulted in usable information. When the same question was rephrased as if it were for academic research purposes, the percentage rose to 42%. Read also: AI could transform the economy faster than the Industrial Revolution, experts warn Increased use of AI by terrorists The report once again drew attention to a growing concern among experts in digital security and terrorism: the possibility of potential attackers starting to use AI for planning attacks and not just propaganda. Over the past three or four years, the main use of AI by extremist groups such as the Islamic State and the Al Qaeda network has been to produce propaganda. This includes videos, memes, podcasts and different forms of disinformation, disseminated among supporters and used to radicalize new followers. But this scenario is changing. The year 2025 will see a significant increase in incidents in which terrorists and violent extremists use AI tools to plan, research and prepare attacks, experts at Militant Wire said in December. Headline-grabbing attacks that caused deaths or property damage, as well as several foiled plots, have used AI for planning, surveillance, visualization and propaganda. Cases were registered in the United States, Canada, Israel, Finland and Austria. It is often difficult to know exactly how AI was used, as law enforcement agencies rarely release details. However, as an expert reported to the UK Parliament during an inquiry held in late 2025, "court cases and expert reports increasingly document conversations in which suspects ask language models for bomb-making instructions, ideological validation or justifications for attacks." Extremist groups also embrace AI This use of AI is not limited to individuals. Extremist groups are also adopting this technology. Researchers studying the use of drones by the Al-Qaeda-affiliated group Jama'at Nusrat al-Islam wal-Muslimin (JNIM), based in Mali, believe the organization has used AI to help modify drones. In an analysis published in June by the Global Network on Extremism and Technology, security researchers Yuri Neves and Emily Klein noted that supporters of the Islamic State, as well as far-right groups, regularly discuss ways to use AI in messaging channels. Neves and Klein work at the American organization Moonshot, which specializes in combating online threats. They identified channels in the Telegram app dedicated to the use of AI, as well as extremists sharing "prompts" and links to conversations with chatbots, coordinating strategies to obtain specific responses, and even sharing the costs of ChatGPT subscriptions. Read also: Meta tool stops recognizing AI images after cropping, says agency Has AI replaced virtual recruiters? Researcher Rueben Dass, from the S. Rajaratnam School of International Studies (RSIS), in Singapore, notes that AI chatbots have started to play new roles in attacks carried out by so-called "lone wolves". "Before, there was the concept of virtual planners, who were people located in conflict zones who contacted others via social media to encourage them to commit attacks," Dass explained to DW. "I don't think you can say that humans have been replaced, but to some extent these isolated actors have turned to AI, like ChatGPT, for that kind of support." According to analyst Moustafa Ayad, from the UK-based think tank Institute for Strategic Dialogue, last year the Islamic State's Voice of Khorasan media outlet published guidance on how to use AI. Ayad says the jihadist ecosystem uses AI in a variety of ways, from creating memes and dance videos for TikTok to cross-border propaganda. "There is also a group dedicated to trying to circumvent AI systems (jailbreaking) and using them to support operational planning and preparation of actions", he states. According to him, AI could be simplifying and supporting advertising processes, while helping planning and
This story was originally published by G1 Tecnologia. Visit the original publication for further details.
Open original publication