Ternary Truths and Hallucinations

So recently, this paper⁠ from OpenAI has been going the rounds, talking about hallucinations. I’ve only (barely) skimmed the paper, but the section which seems to be the most pertinent happily is the first one, and I’d strongly encourage you to read it first. “Ternary truths” is something I thought about for a while—I blogged about this way back in January. I didn’t end up writing anything back then, but I’m still gonna say I called it—I’m also going to expound a bit on my thoughts here, ‘cause I might as well now that it is the hot thing in AI (for the next 5 seconds, approximately). There isn’t a whole lot I can say that the paper can’t say better, and there’s not much point in my quoting or rephrasing the paper when you can read it yourself; as such, these are mostly my own thoughts. Read the paper if you want a more cogent, general, and forceful argument.

A bit on what I mean(t) before continuing. The paragraph from my above linked post, in case you didn’t hit the link, is repeated verbatim to save a request.

Models need to be more accurate to be useful. Ternary truths are key. I want to write about this soon, but I’m thinking of nullable bools as a means to express uncertainty for decisions–things aren’t always yes or no, sometimes you don’t know, and that should be made explicit.

This was in January, about five days after R1—so an eon ago in model-time. Things have changed a lot in this regard, at least when it comes to recalling/generating facts. Search tooling has obviated this concern to some non-trivial degree, as has the constant march of model improvement. Just in my little world of LLMs for statistical programming, anecdotally, models have gotten a lot better at not confabulating packages/methods/arguments/etc. See Simon Couch’s post⁠ on this too. As it stand, in my personal use, it seems as if hallucination incidence rates have decreased.

It is probably a good idea to define how I use “hallucination” before continuing—instead of coming up with such a definition, I will quote Mike Caulfield⁠, whose article I found through Simon Willison’s Weblog⁠.

[Hallucinations] initially described a very weird, mostly non-humanlike behavior where LLMs would make up things out of whole cloth that did not seem to exist as claims referenced any known source material or claims inferable from any known source material. Hallucinations as stuff made up out of nothing.

The key to understanding this definition is, as stated by Willison, that “the initial incorrect answers were not hallucinations: they correctly summarized online content that contained misinformation.” So, some people are wrong on the internet⁠ and sometimes a model (even with search tooling) will incorrectly infer that these incorrect statements are, in fact, the truth—and so you end up with an accurate summary of inaccurate information. To alleviate this, you need to restrict the data available to a model to a vetted and high quality corpus. I’d hazard that this is untenable for training large models.

Ternary truths ideally reduce the numbers of hallucinations to simple True/False questions by offering the model the opportunity to state that it doesn’t know the answer. Nullable Booleans are merely an implementation detail. The idea isn’t novel by any means; the idea of “ternary truths” is espoused by many people, if indirectly—the idea of admitting when you don’t know the answer to a question instead of faking an answer isn’t uncommon advice. Implementing such a scheme wasn’t something I had though much about—see section 4.3 in the paper. The “ternary” part is also easily expanded; the core idea is expressing uncertainty, and so any nullable sum Type would work.

The other axis of resolution that could be increased is vis-a-vis the uncertainty—here, null would work as a statement of uncertainty but that is, again, binary. The model is either uncertain or it isn’t. This may suffice on its own, or could provide most of the performance gains, but looking further into measuring and conveying measures of uncertainty would be interesting. Maybe a Likert-styled answer which bins uncertainty would work? If we are sticking with finite answer choices, I’m guessing some proper prompting and other scaffolding would allow for models to reply with a PMF over the answer choices which could be summarized by perplexity, SD, or something else? The other obvious limitation I noted immediately was that this approach doesn’t easily admit open ended responses. Perhaps foolishly, I hoped that RL considering uncertainty would bake in a cautious nature into the model which would extend to complete generation, in the relevant scenarios. Section 5 of the paper addresses these limitations and more.

Hallucinations are quite interesting, and reductions in hallucination rates have certainly encouraged me to further use LLms as their outputs become more useful. I look forward to seeing how this area develops as labs focus on axes besides model size as naive/raw compute scaling becomes increasing untenable (contra Bitter Lesson, slightly? That is something I need to write about—though Sutton might not have meant it this way, I always read The Bitter Lesson as more of Kuhnian perspective.)

I found this line from the paper to be quite funny, and certainly not the kind of language I’d expect in research material:

Humans learn the value of expressing uncertainty outside of school, in the school of hard knocks.