One area we identified was our ability to interpret nonsensical queries and satirical content. Let's take a look at an example: "How many rocks should I eat?" Prior to these screenshots going viral, practically no one asked Google that question. [...] This is what is often called a "data void" or "information gap," where there's a limited amount of high quality content about a topic. [...] In other examples, we saw AI Overviews that featured sarcastic or troll-y content from discussion forums. Forums are often a great source of authentic, first-hand information, but in some cases can lead to less-than-helpful advice, like using glue to get cheese to stick to pizza.
Curated from blog.google · 30 May 2024 →
Google put AI Overviews above ordinary results for everyone in the United States on 14 May 2024. Within a week screenshots were circulating of it recommending glue in pizza sauce, sourced from an eleven-year-old Reddit joke, and eating rocks, sourced from an Onion piece a geology software firm had republished. Google's head of Search wrote this account, which is unusually specific about the mechanism: the system was not inventing so much as faithfully summarising the only thing on the web that answered the question. That is the failure mode worth taking away, because it survives better models. Google shipped restrictions on satire and user-generated content in Overviews, and limits on where they appear at all.
