AI for Healthcare: Great benchmark scores but minimal impact

The paradox of AI in healthcare is that while benchmarks show soaring (and sometimes superhuman) performance, in the real world AI is not actually helping people much. This is partially because real-world healthcare is messy, AI/NLP people do weak evaluations, deploying AI systems in real-world healthcare is challenging, and impact is limited even when systems are deployed.