You Can't Delete a Hallucination

This site does not allow itself to be embedded. You can still read it on the original site — the toolbar below keeps your place in the directory.

A team's model kept 'hearing' a phrase in videos with no audio. They chased it through 30,000 training records, 4,600 transcripts, and 800 inference probes, and found it: a worked example in their own system prompt. They deleted it. The model just hallucinated a different phrase. The lesson is that the model didn't learn a confabulation. It learned to confabulate, and that lives in the…

Read on paddo.dev ↗