Hallucination

Hallucination — when a model states something with confidence that its sources do not support.

The version that costs you money

The famous examples are exotic — invented court cases, imaginary citations. The version that affects a local business is mundane and much more common: hours you don’t keep, a service area you don’t cover, a price you retired two years ago, a competitor credited with work that was yours.

The model is not lying. It is filling a gap, and gaps get filled from whatever is nearest — an old directory listing, a stale profile, a page you forgot you published.

The fix is boring

Publish the fact plainly, where it can be retrieved, and then make sure nothing anywhere else contradicts it. Most hallucinations about a real business are not invented from nothing. They are a stale record winning against a current one because the stale record was easier to find.


Related: grounding, entity resolution, NAP consistency.

← All terms