Why is Artificial Intelligence hallucinating?

You might have come across AI hallucinations with the rise of Generative AI and Large Language Models (LLMs) or even experienced them firsthand. Let's dive in and explore what causes these hallucinations and how we can mitigate them.

• What are AI hallucinations?

• Why does it happen?

• The implications of AI hallucinations

• How to prevent AI hallucinations?

• Will we ever fully overcome hallucinations?

• Sources

AI hallucination occurs when an artificial intelligence system fabricates details or generates false information. This happens due to processing errors or misapplications of learned patterns that are not actually present in the input data. Those hallucinations can appear in various AI applications… Most often in:

• Image Recognition: Seeing objects that are not there.

• Language Models: Generating nonsensical or factually incorrect text that seems coherent.

An AI might invent historical events, create non-existent scientific theories, or misattribute quotes. These errors highlight the limitations of current AI technologies and emphasize the need for robust training datasets and algorithms. AI LLMs tools, like ChatGPT, predict strings of words based on patterns but lack the reasoning to verify factual accuracy.

Here are a few real life examples:

• A developer seeking coding package recommendations might receive both legitimate and malicious suggestions by a LLM. This has happened when businesses unknowingly used fake software packages suggested by AI.

• Some small and “invisible” modifications to an image, after a deliberated attack, might cause a Machine Learning model to misclassify a cat as "guacamole." These vulnerabilities can present serious risks for applications that rely on accurate identifications.

• During Google's last AI keynote at I/O, an ad for their new AI-enhanced search engine gave a problematic suggestion: when asked how to fix a jammed film camera, one recommendation was to "open the back door and gently remove the film"… Which could potentially ruin any photos that have been taken!

In Natural Language Processing (NLP), including LLMs, models are designed to always provide an answer. This means they'll respond even if they don't know the answer… making one up. To mitigate this, you can instruct the LLM to say it when unsure. Beyond NLP, hallucinations can occur when a model gives a confident but incorrect response, seemingly unsupported by its training data.

Overall, hallucinations can be because of several reasons, which include:

A model's accuracy depends on the quality of its training data. Incorrect, outdated, or insufficient data leads to poor results. If the model encounters unfamiliar idioms or slang, it may produce nonsense. High-quality and comprehensive data is essential for reliable AI.

It occurs when the model becomes too tailored to the training data, capturing noise or irrelevant details as meaningful patterns. This prevents the model from generalizing to new data, resulting in AI hallucinations and inaccurate outputs.

AI models without contextual understanding can generate outputs that are out of context or irrelevant.

An AI model that continuously learns from interactions can be manipulated to recommend false or harmful information.

AI hallucinations are a significant ethical concern. They mislead people, erode trust, perpetuate biases, and can cause real-world harm if taken at face value.

AI-generated hallucinations can spread false information, especially in news, education, and science. Imagine some news bots giving unverified info during an emergency, spreading falsehoods and undermining response efforts.

AI hallucinations pose, for example, safety risks in healthcare, transportation (autonomous vehicles) and cybersecurity. Misdiagnoses and faulty commands can endanger lives. A healthcare AI might wrongly identify a benign lesion as malignant, leading to unnecessary treatment.

AI-generated false narratives can damage reputations. When AI incorrectly attributes actions or statements to public figures or organizations, it can lead to backlash, legal challenges, and a long-term loss of trust.

AI hallucinations can result in inaccurate predictions and flawed data analysis. This could potentially misguide businesses using AI for decision-making, leading to wrong strategies, resource misallocation, and missed opportunities.

The most effective way to mitigate the impact of AI hallucinations is to prevent them from occurring. Here are some useful tips and techniques.

• Limiting the dataset to reliable and verified sources ⇒ prevent AI from learning misleading information.

• Pre-training on large and diverse datasets (books, articles, websites…) ⇒ helps the model to learn a wide range of facts and contexts, reducing inaccuracies.

• Adding the relevant information and context ⇒ AI needs to be "grounded" with relevant data and context in order to provide accurate responses.

Guiding AI responses is a way to ensure more consistency and accuracy. It can be done by:

• Defining the format of responses (data templates).

• Providing example of answers within prompts.

• Introducing constraints on the model's output, like response length or staying within known facts, etc.

• In the development process: Incorporating human feedback, as done in Reinforcement Learning with Human Feedback (RLHF), allows users to flag and correct errors or false information.

• When the model is live: Ensuring human validation/review of AI outputs is a crucial backstop to prevent hallucinations. Human oversight means most AI errors can be identified and corrected.

Rigorously testing your AI model before use and evaluating it continuously are vital to preventing hallucinations ⇒ allow for adjustments or retraining as data evolves.

It can be done by the development team but also researchers, subject experts and the red team…

Transparency in AI models is crucial, especially for decision-making processes. By making these processes transparent, we can identify and correct errors or biases that may lead to hallucinations, or allow users to spot them more easily.

• Adversarial training: Technique that strengthens models against adversarial attacks, which involve subtle changes to input data to deceive models. By incorporating adversarial examples during training, models learn to detect and resist such attacks more effectively.

• Process supervision: OpenAI's new strategy to combat hallucinations involves training AI models to reward themselves for each correct step of reasoning, rather than just the final correct answer. This approach, opposed to "outcome supervision", encourages models to follow a human-like chain of thought, potentially leading to more explainable AI.

• Give the model a specific role and tell it not to lie.

• Tell the model what you want and don’t want.

• Experiment with the temperature to determine whether the output is more random and creative or more factual.

Some AI researchers believe hallucinations are inevitable in large language models, as no system can be 100% accurate. That is why major AI players, including Google and OpenAI, advise users to check responses for accuracy.

Furthermore, Sam Altman, CEO of OpenAI, argued that in the context of generative AI, removing all inaccuracies would remove its "magic" that people love. He also suggested that these hallucinations are more of a feature than a bug.

To conclude, while AI hallucinations are generally undesirable, they can also offer intriguing use cases, allowing organizations to leverage their creative potential positively.

Find all the sources here.