HealthBench Does Not Evaluate Patient Safety

HealthBench is a recently released benchmark to evaluate large language models in healthcare. This blog post summarizes what HealthBench is, and overviews its positive and negative aspects, including a patient safety gap. What is HealthBench? HealthBench is a new healthcare LLM benchmark released by OpenAI in May 2025. It includes 5,000 health conversations between a [ ]