“Why Do You Still Have a Job?”

“Will there be doctors?” It’s the question I’ve been asked more than any other on my book tour. No specialty makes the stakes – or the folly of confident predictions – clearer than radiology, long considered the medicine specialty most vulnerable to AI replacement. What follows is an abridged version of my chapter on radiology from A Giant Leap.

At a 2016 conference in his hometown of Toronto, Geoffrey Hinton, who would later win the Nobel Prize for his work on neural networks, was asked about the future of radiology. “If you work as a radiologist, you’re like the coyote that’s already over the edge of the cliff but hasn’t yet looked down, so he doesn’t realize there’s no ground underneath him,” Hinton said. “I think we should stop training radiologists now. It’s just completely obvious that within five years, deep learning is going to do better than radiologists… we’ve got plenty of radiologists already.”

Diagnostic radiology has always been among the most popular specialties for medical students – it’s one of the so-called “ROAD (Radiology, Ophthalmology, Anesthesia, Dermatology) to Happiness” specialties – ones that combine high incomes with relatively humane hours. But Hinton’s statement was the medical equivalent of Warren Buffett saying he was shorting his radiology stock. Whereas radiology residency programs had always been massively competitive, in 2020, only 41% of residency spots in America were filled by graduates of U.S. medical schools, as students decided that a career in radiology was too risky a bet.

Then something funny happened. Medical students noticed that their radiology attendings’ salaries remained sky-high. (They average about $600,000 per year.) Moreover, rather than hearing tales of unemployed radiologists, students witnessed an explosion in radiology help-wanted ads. By 2024, the Hinton Effect had evaporated – the number of U.S. medical students choosing diagnostic radiology nearly doubled from 2020. If the field is in crisis, it’s because of a nationwide shortage, not a surfeit, of radiologists.

While Hinton’s miscue was humbling, he was far from alone in being off base regarding the future of radiology. In fact, if you’d asked me 15 years ago which would come first, my willingness to climb into the back seat of a driverless car or to have my X-ray read autonomously by AI, I would have chosen the X-ray. Yet today, I take a driverless Waymo in San Francisco several times each month. Meanwhile, my own hospital, based in the same tech-obsessed city, can’t hire radiologists fast enough.

When I began A Giant Leap, I thought that Hinton’s blunder was partly due to his underappreciation of the power of the radiology guild to protect its franchise. I no longer do. In fact, given the enormous increase in the number of scans to be read, I found that, for most radiologists, useful assistance from AI can’t come fast enough.

Hinton’s miscalculation has much to teach us about the complexity of medicine, the forces that will likely shape (and slow) healthcare’s AI revolution, and why predictions that AI will replace physicians any time soon should be served up with a generous dollop of skepticism.

I asked John Mongan, a UCSF radiologist and AI expert, why he still had a job a decade after Hinton’s famous prediction. “The people who were making those predictions understood computer vision but didn’t really understand radiology,” he said. “They were writing algorithms that could tell you that an image was a dog or a sailboat. And they thought that radiology was just doing that for medical stuff. But radiology is a lot more than that.” Added Mayo Clinic neuroradiologist Bradley Erickson, “I can take a 16-year-old and teach them to drive in a day, max. I don’t think I can take a 16-year-old and teach them to do neuroradiology in a day.”

They’re right. For one thing, the interpretation of an X-ray is often influenced by the patient’s history, which the radiologist might glean from reading the medical record or talking to the clinician caring for the patient. I remember when the first cases of AIDS began cropping up in the early 1980s. At the time, the diagnosis of AIDS was a death sentence, the most common fatal complication being a previously rare lung infection called Pneumocystis carinii pneumonia, or PCP (now called PJP, Pneumocystis jirovecii pneumonia). PCP could produce a chest X-ray that was floridly abnormal: the normally black air sacs looked like they had been smothered under an avalanche of white snow. But PCP’s radiologic appearance could also be exceedingly subtle, more like a little smudge on a glass windowpane. In these latter cases, my chest radiologist would often say, “If you tell me this is a straight 50-year-old man, I’d say the X-ray is normal. If it’s a 26-year-old gay man, I’d say it’s PCP.” In other words, the exact same X-ray appearance could mean very different things, depending on the clinical context.

Yet even this constraint – that the radiologist must interpret an image while considering the patient’s other medical information – seems surmountable given AI’s rapid advancement in “reading” the medical record. If the only reason Hinton got it wrong was AI’s inability in 2016 to incorporate the patient’s clinical situation, then we should see an AI revolution in radiology fairly soon.

But it turns out there are even more challenges to overcome on the path to useful and trustworthy AI in radiology.

The first is the dearth of labeled training sets. While it’s relatively easy to obtain thousands of doctor-patient conversations to train AI scribes or conversational agents, there are surprisingly few curated and well-labeled digital X-ray datasets available for training. Companies trying to build AI radiology tools have often been forced to undertake the expensive, labor-intensive task of purchasing large radiology datasets and labeling thousands of images themselves.

Another key consideration is radiologists’ need to review old films. In a complex case, the radiologist might need to review dozens of prior images, often from various imaging modalities (plain films, CT scans, MRIs, ultrasounds, etc.). To be truly constructive, a radiology AI tool would also need to review these past images, then compare those findings against the current image, assigning appropriate weights to differences in imaging type and varying time horizons.

Another reason for Hinton’s faux pas was his failure to appreciate that, to be useful in radiology, the AI tool needs to sync up with the radiologists’ workflow – an assembly line that’s moving at breakneck speed. The process involves first confirming that you’re looking at the correct patient’s images, reading the relevant clinical history, then looking-clicking-looking-clicking (toggling between the current and old images)… then looking again and, finally, dictating or typing the findings. It’s especially challenging for AI to blend seamlessly into this process when the company developing the AI isn’t the one that built the machines supporting the overall workflow.

An additional limitation is that today’s machine vision systems can generally master only one diagnosis at a time. NYU bone radiologist Miriam Bredella recalled seeing a demo of an AI system designed to detect arm fractures. It correctly diagnosed a fracture of the radius, a bone in the forearm, but missed one an inch away in the thinner but longer ulna. “The system wasn’t trained on that,” the company representative confessed.

This specificity means that the radiology AI products currently on the market are mostly one-trick ponies. For example, GE has integrated a pneumothorax (collapsed lung) detector developed at UCSF into its advanced chest X-ray scanners. Since a pneumothorax can rapidly become life-threatening, the AI flags X-rays that appear to show one, moving the image to the top of the queue for the human radiologist to review. Another commercially available AI tool detects whether feeding tubes and catheters are correctly positioned in the chest, saving the radiologist a few minutes of tedious measurement. Each one of these algorithms – the pneumothorax detector, the feeding tube placement detector, a pulmonary embolism detector – are sold separately, often by different companies, and each can run hospitals tens of thousands of dollars a year for subscriptions and operating costs. But human radiologists must look for all these things – as well as broken bones, signs of pneumonia, edema, cardiomegaly, pulmonary hypertension, and enlarged lymph nodes – simultaneously, which markedly limits the value of these single-disease detectors. (This may be changing – the FDA recently approved a multi-disease-detecting abdominal CT scan triage system developed by a company called Aidoc.)

It follows that the AI tools currently having the biggest impact in radiology are screening programs, particularly for cancer, where the goal is straightforward: identify a single disease, like spotting a nodule suspicious for lung cancer on a chest CT or a density that might be a breast cancer on a mammogram. This kind of AI-enabled screening can be a huge help if it allows the radiologist to quickly endorse a negative result and move on to review positive or uncertain cases.

A growing body of literature supports the use of AI in such screenings. Even here, though, our enthusiasm needs to be tempered by AI’s long history of overpromising and underdelivering. In the 1990s, computer-assisted mammography tools were widely adopted after the FDA endorsed them. Yet once the tools entered community practice, studies showed that their accuracy plummeted.

What caused the problem? In the real world of clinical practice, the doctor – who lives in fear of overlooking a cancer (missing a breast cancer is not only devastating to both patient and doctor; it’s also the most common cause of successful lawsuits against radiologists) – will likely recommend a biopsy after a positive reading by the AI, even if she would have judged the lesion to be benign in the absence of AI.

As more attention is paid to these human-AI interactions, things are improving. In a 2023 Swedish study, roughly 80,000 women were randomized to receive either standard mammography – in which two human radiologists look at the scan to be sure they’re not missing anything – or an AI-assisted reading. In the latter group, the AI reviewed the scan first. If the AI found the mammogram benign, a single radiologist signed off on the result. If there was anything suspicious, the scan entered the standard double-reading queue.

The results were impressive. The AI-assisted group identified 20% more cancers than the human radiologists operating without AI. The false positive rates were low in both groups. Notably, the overall workload of the radiologists decreased by 44% in the AI-reading arm, a savings of nearly 37,000 radiologist readings, allowing patients to receive results faster and saving the health system a small fortune. While it will be important to follow these women over time to see whether the early detection of cancers was clinically meaningful, AI-assisted radiology seems to be finally hitting its stride, at least in breast cancer screening.

Regulatory scrutiny is another hurdle for digital radiology. While it remains unclear whether an AI-based readmission predictor or diagnosis-suggester needs regulatory approval, there’s no doubt that a radiology AI tool falls under the FDA’s jurisdiction as a Software as a Medical Device (SaMD). Just as AI radiology tools typically address a single diagnosis, the current FDA approval process has tended to mirror that specificity (although that may be changing, as evidenced by the multi-indication Aidoc approval I mentioned earlier). This means that, until now, radiology AI companies have been forced to seek one approval for pulmonary embolism detection, then another for pneumonia detection. Each approval takes many months to obtain and may cost several million dollars per indication.

The real world of radiology – and medicine – is not that siloed; it’s messier and far more integrated. Take a patient with unexplained weight loss who gets a CT of the abdomen. There are literally hundreds of diagnoses that may be lurking, and we rely on radiologists to see them all. The need for each diagnosis to have its own bespoke algorithm and separate regulatory approval may be prudent at this stage of AI’s development, but will significantly hinder the utility and adoption of artificial intelligence in radiology.

Given the never-ending volume of images in their work queues and the waning fear of Hintonian job replacement, you might think that radiologists would be clamoring for AI to help. But most radiologists don’t find that today’s tools – with the notable exception of single-disease screeners – improve their efficiency very much. Penn radiologist Saurabh Jha likens the current AI-assisted programs to a backseat driver who incessantly and annoyingly points out road hazards. “That’s not helpful,” said Jha. “If you want to help me drive, then you take over the driving so that I can sit back and relax.”

A 2024 study supported this concern. Among 6,726 radiologists in China, those using AI experienced a burnout rate 20% higher than those who weren’t. The reasons were unclear – it’s possible that the AI created additional work by flagging more abnormalities to review, or that the AI, by taking the easy stuff off their plate, increased the radiologists’ cognitive load. Since it wasn’t a randomized trial, it’s also possible that the radiologists who were already burned out were the ones choosing to use AI. In any case, this is another thread in the tapestry of why replacing radiologists with AI is harder than it looks.

While the tools are clearly not good enough to replace radiologists today, the progress is unmistakable, and the potential is immense. I asked UCSF’s John Mongan what he’d tell his kids if they said they wanted to go into diagnostic radiology. “I think it’s a great field,” he said. “If you don’t like computers and you don’t want to learn anything about AI, you should really think twice about it. But if you are ready to embrace and be part of the revolution, there’s going to be plenty of work for human radiologists for the next several decades... I don’t think the radiology department is going to turn into a data center with two people.”

While I believe Mongan is right for the foreseeable future, the challenges that have slowed the march of AI in radiology now seem manageable, particularly with generative AI. These include the need to integrate multi-modal data (e.g., patient history and lab studies) with radiologic findings; to assess images not for single abnormalities but holistically; to have access to large, well-curated datasets of images; to ingest, interpret, and integrate the findings from prior imaging studies; and to weave AI seamlessly into radiologists’ workflows. In fact, a 2024 survey of 200 healthcare organizations and imaging groups found that more than half were using at least one radiology AI tool, up from 17% in 2018. The most common uses were to triage brain scans done to diagnose strokes, to screen for breast and lung cancer, and to draft radiology reports.

It’s hard to predict exactly how that will play out, but it does seem like AI is gaining a toehold in radiology and may even be nearing its tipping point. In the end, I suspect that AI will have a central role in radiology, and that we’ll ultimately need fewer radiologists than we do now (or perhaps the same number to interpret far more imaging studies). Perhaps Geoffrey Hinton’s infamous 2016 warning about training new radiologists will prove prescient after all – though the job losses seem destined to arrive on a timeline measured in several decades rather than the handful of years that he predicted. Like many prophets of technological disruption – particularly in healthcare – Hinton may have correctly identified the destination while drastically underestimating the length and complexity of the journey.