Confidence in Classification using LLMs and Conformal Sets

One of the common use cases for large language models is classifying text -- but data scientists often want to know the confidence that the text falls into the category. This blog post shows an example using a models log-probabilities to determine if text has obscene language, while setting the recall rate or estimating the precision of the classification.