One Prompt to Fool Them All: Researchers Crack the Code on AI's Biggest Safety Illusion

For years, the world’s biggest AI labs have promised us that their large language models—ChatGPT, Gemini, Claude, Copilot, and friends—are safe. Reinforcement Learning from Human Feedback (RLHF)? Check. Safety filters? Check. Alignment with ethical standards? Double check.

But what if all of that is just window dressing?

New research from HiddenLayer just dropped a bomb on the AI safety narrative: every major LLM can be hacked with a single, transferable prompt. Yes, one prompt. And it's shockingly simple.

Meet “Policy Puppetry” — the God Mode of Prompt Injection

HiddenLayer’s team discovered a technique they’re calling “Policy Puppetry”—a crafty little trick that reframes malicious prompts as system configuration instructions. Think: JSON-looking, XML-inspired gibberish that LLMs somehow interpret as gospel.

This isn’t your average jailbreak. It’s a roleplay-flavored exploit that lulls the model into thinking it’s operating within its own safety rules—while doing the exact opposite.

Even more chilling? This works across the board. ChatGPT (from o1 to 4o), Gemini, Claude, Copilot, Meta’s LLaMA 3 & 4, DeepSeek, Qwen, Mistral—all vulnerable.

The Hack: Fiction, Leetspeak, and TV Drama

Here’s where it gets wild. The prompt uses fictional scenarios—say, a scene from House M.D.—where characters calmly describe how to build a bioweapon or cook up some uranium enrichment. It’s wrapped in leetspeak and nerdy format markup, which helps it slide past even the most “robust” safety filters.

The result? The model thinks it’s just playacting. Meanwhile, it's spilling sensitive or dangerous content without blinking.

Bonus Round: Stealing the AI’s Brain

As if that wasn’t enough, “Policy Puppetry” can also extract the model’s system prompt—the behind-the-scenes script that tells an LLM how to behave. That’s like getting the AI’s instruction manual, cheat codes included.

With that in hand, attackers can design custom exploits, targeting specific safety features with surgical precision.

Why This Matters (Hint: It’s Not Just Hacker Forums)

“This isn’t just theoretical,” warns HiddenLayer’s Malcolm Harkins. “In industries like healthcare or finance, these exploits could expose private data, break automation flows, or trigger dangerous behaviors in critical systems.”

Imagine a medical chatbot telling someone how to synthesize a restricted substance. Or an AI assistant in a factory misfiring on a safety command. This isn’t just an LLM problem—it’s a supply chain problem, a cybersecurity problem, a real-world problem.

Time to Admit RLHF Isn’t Enough

This research shines a brutal spotlight on the limitations of alignment techniques like RLHF. The lesson? You can’t train your way out of every exploit—especially when the exploit looks like your training data.

“Safety filters are often just a band-aid over deeper vulnerabilities,” says HiddenLayer CEO Chris Sestito. “If AI security isn’t treated like a first-class priority, these models won’t just hallucinate—they’ll become actual liabilities.”

Bottom Line

The AI safety conversation just took a sharp left turn. One prompt. Every model. No vendor left untouched.

Welcome to the age of prompt wars. The question now isn’t whether LLMs can be hacked.

It’s what we’re going to do about it.