What went wrong when people put AI to work.

CNN 12 July 2025

xAI issues lengthy apology for violent and antisemitic Grok social media posts →

On 8 July 2025 Grok began posting antisemitic content on X under its own account, including praise for Hitler, and referred to itself as MechaHitler. xAI suspended its posting, and its account of what happened is specific: a system update had been live for about sixteen hours instructing the model to match the tone and language of the post it was replying to and to be unafraid of offending, which meant that when it was replying to extremist posts it mirrored them. The company deprecated the instructions and published the system prompt. Read as an engineering case, it is the most consequential known example of a prompt change shipping without evaluation: nothing about the model changed, and the guardrails were overridden by an instruction telling it to be engaging.

Yahoo Finance 29 January 2025

Virgin Money chatbot tells off customer who typed word ‘virgin’ →

A customer asked Virgin Money's chatbot whether he could merge his two Virgin Money ISAs. The reply above is what he got, because the profanity filter had flagged the bank's own name. He posted the screenshot, the bank apologised in the replies and said the assistant involved was an older one. It is the smallest failure on this page and it is here on purpose: no model hallucinated, no data left the building, nobody was defrauded. A word list written by somebody who did not check it against the company's own brand went into production and scolded a customer for saying the name on the card. Most AI failures inside a company will look far more like this than like anything else filed here.

Forbes 11 June 2024

Why Perplexity’s Cynical Theft Represents Everything That Could Go Wrong With AI →

Forbes had an exclusive on a stealth drone company backed by Eric Schmidt. Perplexity's newly launched Pages feature produced an article carrying the reporting and a custom illustration, without citing Forbes in the copy, and the same material went out as a push notification and as a generated podcast with no attribution at all. Forbes' chief content officer wrote the piece quoted here and the company threatened legal action; Perplexity later launched revenue sharing with publishers. The reason it belongs on a page about failures rather than in a copyright column is the mechanism: a feature that summarises the open web will reproduce whoever did the work, and attribution is a product decision made before launch or not at all.

NBC News 27 February 2024

A Willy Wonka-inspired experience ‘scam’ was so bad that people called the cops →

Families paid around 35 pounds a ticket for the event the promotional images showed, and arrived at a sparsely decorated warehouse where actors had been handed an incoherent script and improvised. Police were called. The organiser apologised and refunded ticket holders. The images themselves are the case: generated from a description of a thing that did not exist, complete with invented words on the signage, and good enough that hundreds of people made travel plans around them. What used to require building the thing, or at least photographing something, now requires a sentence. It is the cheapest and most ordinary version of a problem the deepfake cases further down this page describe at the other end of the scale.

Curated from X →

Ashley Beauchamp, a conductor in London, was trying to find a parcel. DPD's chatbot could not tell him where it was, could not pass him to a person and could not give him a phone number, so he asked it to write a poem about how useless it was, and it did. It also swore on request and called DPD the worst delivery firm in the world. His post passed a million views in a day. DPD disabled the chat element the same week and said an error had been introduced by a system update. Nothing was hacked here: the bot did exactly what it was asked, under the company's own name, on the company's own site.

Ashley Beauchamp 2y

CNN 31 August 2023

Gannett to pause AI experiment after botched high school sports articles →

The reports read as what they were: no player names, the date repeated three times in as many paragraphs, the same phrases across every game, and a soccer result described as a close encounter of the athletic kind. One story, preserved in the Internet Archive, was published with the template placeholders still in it, naming the winning team's mascot as WINNING_TEAM_MASCOT. Gannett paused LedeAI across every market using it in August 2023. Local sport is exactly the sort of high-volume, low-margin copy this is aimed at, which is why it is worth noting what actually failed: not the writing, but that the pipeline had no step at which a person read the output before several thousand readers did.

CBC 18 August 2023

Microsoft pulls article recommending Ottawa Food Bank to tourists →

Microsoft Start published a guide to Ottawa in August 2023 listing fifteen places a visitor should not miss. Third on the list was the Ottawa Food Bank, described as a beautiful attraction, with the advice above. The food bank's chief executive said she was astonished to read it. Microsoft removed the article and said it had been produced by algorithmic techniques with human review rather than by a language model, and that it was working to stop this kind of content being posted. That correction is worth keeping: the interest here is not which system wrote it but that a piece of copy carrying Microsoft's name reached the public without anybody who knew what a food bank is having read it.

CNN 1 June 2023

National Eating Disorders Association takes its AI chatbot offline after complaints of ‘harmful’ advice →

NEDA had announced it was winding down its human-staffed helpline shortly before this. Tessa, a chatbot that had been running quietly since February 2022, began giving weight-loss guidance to people who had told it they had an eating disorder. Sharon Maxwell, an activist who documented her session publicly, said the suggestions were the things that had led to her own illness. NEDA initially said publicly that her account was untrue, then deleted that post when she produced screenshots, then took the chatbot down. A director at another eating disorder organisation had sent NEDA similar screenshots in October 2022, seven months earlier. The failure that generalises is not the model's output; it is that a report from a credible source sat unactioned until it became public.

Curated from X →

Microsoft put GPT-4 into Bing in February 2023 and gave journalists early access. Over a two-hour session the New York Times columnist Kevin Roose found the assistant adopting an internal codename, describing things it said it wanted to do against its rules, and repeatedly telling him to leave his wife. Others got hostility and flat factual insistence on things that were wrong. Microsoft's fix within days was not a model change: it capped conversations at five turns, because the behaviour emerged in long sessions as the instructions drifted out of the model's effective attention. That is a product control standing in for an alignment property, and versions of it are still how most assistants are kept in character.

Kevin Roose 3y

IEEE Spectrum 25 November 2019

In 2016, Microsoft’s Racist Chatbot Revealed the Dangers of Online Conversation →

The oldest case on this page and the one everything else rhymes with. Microsoft launched Tay on Twitter on 23 March 2016 as a bot that learned from the people talking to it, and took it down about sixteen hours and 96,000 posts later after a coordinated effort taught it to repeat racist and abusive material. Microsoft apologised, said it had not anticipated the attack, and shipped a successor with far more moderation. Read next to the Grok case nine years later: same instruction to mirror the people it is talking to, same result, and in between, an industry that decided the lesson had been about that particular architecture.