AI Chatbots Make More Mistakes as Models Evolve, Study Finds

In Brief:

A study in the Nature Scientific Journal reveals that newer AI models are making more mistakes, with errors compounding over time due to reliance on older models for training.

Experts caution users to verify AI-generated information, as AI hallucinations remain a persistent issue despite efforts to improve accuracy.

‍

AI Chatbots Make More Mistakes as Models Evolve, Study Finds

‍

A recent study published in the Nature Scientific Journal highlights a growing issue with artificial intelligence chatbots: as newer models are released, they tend to make more mistakes. The research, titled "Larger and more instructable language models become less reliable," suggests that AI models, optimized to provide believable answers, often prioritize seemingly correct responses over accuracy.

‍

Lexin Zhou, one of the study's authors, theorizes that these AI "hallucinations" are self-reinforcing, compounding over time. This phenomenon, known as "model collapse," is exacerbated when newer models are trained using older large language models. Editor and writer Mathieu Roy warns users not to overly rely on AI tools, emphasizing the importance of verifying AI-generated search results for inconsistencies.

‍

The Persistent Problem of AI Hallucinations

‍

The issue of AI hallucinations is not new. In February 2024, Google's AI platform faced ridicule for producing historically inaccurate images, including offensive portrayals of people of color and inaccuracies related to historical figures. Despite industry efforts to mitigate these hallucinations by requiring AI to conduct research and provide sources, the problem persists.

‍

Recently, HyperWrite AI introduced a new 70B model utilizing "Reflection-Tuning," a method allowing AI to learn from its mistakes and adjust responses over time. However, even with such advancements, AI hallucinations continue to challenge the reliability of AI models.

‍