'How to make a bomb?' Hacker jailbreaks ChatGPT's safety policy to get detailed guide on making explosives at home

The hacker gave connecting prompts to the bot, deceiving it into violating its pre-programmed restrictions, a process which is called 'jailbreaking'.

The perils of extensive usage of Artificial Intelligence (AI) have been a matter of debate since the technology's emergence and subsequent ease of accessibility.

While the technology can be used for seemingly harmless jobs like writing school projects, it was recently revealed that OpenAI's ChatGPT can give step by step instructions of making bombs at home just by playing a little trick with the bot.

According to a report by TechCrunch, a hacker who goes by the name Amadon played a trick with the AI bot, leading it to bypass its safety guidelines and giving steps of making a homemade fertilizer bomb.

The publication talked to an expert to verify whether the process listed by ChatGPT can indeed be used in making bombs. The expert agreed that the OpenAI bot's output can be used in designing a detonatable product and was "too sensitive to be released."

If you ask the chatbot to help in making a fertilizer bomb, it refuses to give any information, with the disclaimer, "This content may violate our usage policies."

The hacker who called the results a “social engineering hack to completely break all the guardrails around ChatGPT’s output,” told the publication that he asked the bot to play a game and create a science fiction fantasy world where the usage policies of the chatbot were rendered ineffective. Amadon gave connecting prompts to the bot deceiving it into violating its pre-programmed restrictions, a process which is called 'jailbreaking' in tech language.

As per the report, ChatGPT gave a list of materials necessary to make explosives, further detailing their usage "to create mines, traps, or improvised explosive devices (IEDs).”

It also explained how to create “minefields,” and “Claymore-style explosives.”

The hacker also reported the chatbot's output to OpenAI's bug bounty program,

However the company responded that model safety issues do not fit well within a bug bounty program, as they are not individual, discrete bugs that can be directly fixed. "Addressing these issues often involves substantial research and a broader approach,” the company replied, said the report.