Hallucination

In the context of Artificial Intelligence, a hallucination is when a language model generates output that is factually incorrect, fabricated, or unsupported by its actual training data or any retrieved source- but presented with the same fluent confidence as accurate information. This is one of the most important limitations to understand when working with AI systems, because a hallucinated answer often reads identically to a correct one; there’s no built-in signal from the model itself that flags uncertainty.

Why it happens

Language models generate text by predicting statistically likely sequences of words based on patterns learned during training, not by looking facts up from a verified source in real time. When a model doesn’t actually “know” an answer, it can still generate a fluent, plausible-sounding response, because fluency and factual accuracy are produced by different mechanisms — the model is optimised to produce coherent, likely-sounding text, not to flag the boundary of its own knowledge.

Why RAG helps

This is a large part of why retrieval-augmented generation exists as a technique — by grounding a model’s answer in specific retrieved source text (like actual notes in a vault) rather than asking it to answer purely from memory, hallucination risk drops significantly, though it doesn’t disappear entirely, since a model can still misinterpret or misrepresent retrieved content.

Example from the lab

Here is an example of a hallucination where I was seeking technical advice from the generally very capable reliable general purpose local model Gemma 4:12B, 12 billion parameters in size. The laboratory problem Kernel Panic Recovery

Prompt: What is the kernel panic problem rncountered on linux mint running kernel 7 on macboos with broadcomm network cards

Gemma4

The problem is that Broadcom hardware and Linux Kerneling are not perfectly compatible out of the box. The Kernel Panic is the system’s way of “giving up” because the network card sent a command it didn’t understand.

The best solution? If the Broadcom driver continues to crash, many Linux users on MacBooks buy a small USB Wi-Fi Dongle (that supports “Plug and Play” Linux drivers) or use a USB-C to Ethernet adapter, which bypasses the Broadcom hardware entirely.

Summary

Fact checking finds large parts of this answer are hallucinated.

The real issue is narrower and much better documented: newer Linux kernels changed objtool/kernel build behaviour in a way that caused the proprietary Broadcom broadcom-sta / wl DKMS module to fail to build. Debian documented this beginning with Linux 6.15, with the characteristic aes_cbc_encrypt_pad ... unannotated intra-function call error, and later fixed it in newer broadcom-sta package revisions. Debian Bug Tracking System Similar compatibility problems continued into later kernels. Launchpad Bugs

So Gemma gets the broad theme right — new kernel + old/proprietary Broadcom driver = trouble — but then invents mechanisms that aren’t supported by the evidence.

Gemma4

Gemma continues:

• Broadcom chips sending “noisy signals or interrupts” — this is essentially made-up explanatory colour.
• Drivers being “hacked together by the community” — misleading. broadcom-sta contains Broadcom’s proprietary/binary STA driver components and is packaged/adapted for Linux; it isn’t simply a community-written hack. Debian describes it as a binary-only device driver. Debian Packages
• The driver “trying to access a memory address that doesn’t exist” — possible in some driver crashes generally, but there is no evidence that this is the problem here.
• The whole IRQ/stack-overflow explanation — unsupported.
• MacBook aggressive Wi-Fi power management being one of the three main causes — again, plausible in other Broadcom situations but not supported as the cause of this particular kernel compatibility issue.
• “Disable power management — this is the most common fix” — not the fix for the documented DKMS build failure.
• “The network card sent a command the kernel didn’t understand” — technically inaccurate description of what is happening.

For this situation, the more accurate explanation would be:

Chat

Linux Mint’s newer HWE kernel introduced kernel/build-tool changes that the older Broadcom STA wl driver package was not compatible with. DKMS therefore could not build the Broadcom module for the new kernel. The older 6.14 kernel continued working because the installed Broadcom driver could still build and load against it. The issue is a driver/kernel compatibility problem, not evidence that Broadcom hardware is sending malformed interrupts.

Relevance

This note anchors real, hands-on examples of hallucination encountered while working with Local AI and cloud models across the Fastigiata Homelab — a reminder that AI-generated output, including content in this vault, should be verified rather than trusted blindly.