Overview
- Users on X and Reddit published viral examples in late May showing Google’s upgraded AI Mode confidently give wrong letter counts and odd spellings, such as saying there are two “p”s in “Google.”
- Google confirmed counting within words is a known challenge for large language models and said engineering teams are working on targeted fixes while continuing to show traditional blue links and keeping AI Mode opt-in.
- Researchers explain the root cause is tokenization in transformer-based LLMs, which encode chunks of text as tokens rather than processing text as individual letters, so letter-by-letter counts are error-prone.
- The prominence of AI-generated Overviews in Search increases the risk that confident but incorrect answers will be seen without users clicking source links, raising fresh concerns about trust in routine searches.
- The spelling and counting failures echo earlier AI Overview missteps, and the episode highlights a broader trade-off as Google pushes conversational, LLM summaries ahead of users evaluating original web sources.