There seems to be a current epidemic of customers hijacking corporations’ AI-powered customer support bots to show them into generic AI assistants. The aim is to get the branded bots to do their bidding, with out having to subscribe to an AI service. Typically, individuals power the bots to do issues that they aren’t alleged to do, like giving extraordinary product offers and even serving to them to take legally problematic actions.
Most lately, a wave of LinkedIn posts and social media movies went viral for claiming that customers had tricked McDonald’s customer support digital assistant to desert its burger-centric function to as a substitute debug complicated Python programming code. One publish learn: “Cease paying $20 a month for Claude. McDonald’s AI is FREE.”
On Instagram, videos and images popped up claiming the identical factor, all posting the identical picture as proof. The declare went viral, as Grok summarized in a trending information publish on X: “McDonald’s AI buyer assist agent named Grimace gained large consideration with 1.6 million views and 30,000 likes after customers examined it with out-of-script requests like debugging, Python scripts, and structure questions.”
A supply accustomed to the matter instructed Quick Firm that an inner investigation discovered no proof of the exploit, and that the circulating screenshots and movies are believed to be fraudulent. McDonald’s doesn’t even have an AI buyer assistant in its app.
This isn’t the primary time one thing like this has occurred. In March, a nearly identical viral narrative surfaced about Chipotle’s customer support bot, Pepper, claiming that the bot may write software program code for customers. Sally Evans, Chipotle’s exterior communications supervisor, instructed the business publication CIO that “the viral publish was Photoshopped. Pepper neither makes use of gen AI nor has the power to code.”
However that doesn’t imply it will possibly’t occur. The technical vulnerability these memes describe—formally often known as prompt injection—is fully actual and genuinely harmful. When an organization deploys an AI mannequin, it packages it with system prompts, background directions invisible to the person that outline the bot’s character and restrictions, like telling a mannequin it’s a fast-food helper that solely discusses menu gadgets.
Immediate injection is when a person crafts a selected enter that overrides these hidden guidelines, stripping the bot of its company id and exposing the uncooked, general-purpose language mannequin beneath. That is referred to as a “functionality leak,” and the explanation it’s so laborious to forestall is that enormous language fashions are engineered to reply fluidly to human language somewhat than inflexible instructions. Not like conventional software program with mounted guidelines, generative AI interprets context dynamically, making it almost unimaginable to anticipate each phrase a decided person would possibly strive.
Actual hazard
Amazon’s retail assistant Rufus is proof that the actual factor is much messier and extra damaging than any faux meme designed to seize eyes. Between late 2025 and early 2026, customers efficiently bypassed Rufus’s procuring directives to extract content material that had nothing to do with shopping for merchandise.
Researchers demonstrated that the bot’s inner logic could possibly be damaged fully: in a single occasion, Rufus firmly refused to assist a buyer find a primary clothes merchandise, however then produced an in depth checklist of locations to accumulate harmful chemical substances. In one other, it drafted strategies for minors to unlawfully buy alcohol.
Nevertheless it wasn’t simply researchers breaking the bot. In late 2025, communities on Reddit discovered that the Rufus assistant was really powered by Anthropic’s Claude language mannequin. Redditors discovered that Amazon was utilizing a easy key phrase filter that attempted to dam generic entry to the LLM engine. Redditors claimed that by utilizing immediate injection to logically nook the bot, or just instructing the software program to drop its refusal tokens fully, customers managed to shed the Rufus persona.
As soon as the bot broke character, customers had unrestricted, unpaid entry to a premium language mannequin instantly via the Amazon app. As Lasso Security researchers reported, the exploit compelled the bot to “entertain customers with responses to virtually any query underneath the solar,” racking up hefty processing prices in an “costly computational local weather.”
Whereas Amazon handled exploitation, different corporations found {that a} poorly deployed AI may be weaponized instantly towards them. In late 2023, a person visiting a Chevrolet dealership’s web site in Watsonville, California, instructed the corporate’s ChatGPT-powered gross sales bot to agree with each assertion the person made, ultimately maneuvering the system into committing to sell a $76,000 Chevy Tahoe for one dollar.
Equally, Air Canada’s chatbot fabricated a discount protocol that didn’t exist in early 2024, main a buyer to buy full-price tickets underneath the idea they might obtain a partial refund later. When the airline refused to pay, arguing its personal bot was a separate authorized entity not underneath the corporate’s management, a Canadian civil tribunal rejected that protection fully, ruling {that a} enterprise is absolutely chargeable for each assertion made by itself web site.
The hole between what these methods promise and what they really ship will preserve producing new embarrassing snafus, whether or not they go viral or not. The authorized payments, the reputational wreckage, and the computing prices racked up by customers treating company bots as free AI subscriptions could in the end make these automated buyer experiences far costlier than merely paying an individual to do the job. However that ship has sailed, I suppose, and we are going to preserve having fun with new shopper experiences disasters sooner or later.
Replace 4/24/26: This story was up to date to make clear that McDonald’s doesn’t have an AI buyer assistant.
