Medical evidence should not be too technical for doctors

I debated two key opinion leaders last week. I will tell you topic later; it doesn’t matter for what I want to argue now.

In both debates, my academic colleagues did not focus on the evidence underpinning the procedure. They mentioned evidence, but in only a guideline-sort of way. As if to say, it’s in the guidelines, it’s settled.

In one of the debates, my opponent cited the vast number of procedures that have been done since approval. The message: Mandrola, the majority of your colleagues love this procedure. You are an outlier? Are you smarter than the masses?

In the other debate, my opponent said he just knew the device worked. The studies I cited were old, and my analysis was technical. “John, most doctors can’t see the details in studies like you can.”

I’ve been stewing about these attitudes for a few days. I’ve heard them before, especially the one about doctors not being able to appraise studies.

This is embarrassing for our field. Of course doctors should be able to assess evidence. We are in the evidence-assessment business. When a patient seeks our help, we use our evidence-assessment skills to sort out the problem and the fix.

We should expect nothing less when it comes to assessing the evidence underpinning our therapies.

But that’s not what I see out there—in the hospital, clinic and speaking circuit.

What I see is that doctors simply accept what guideline writers put into those colored boxes. Which would be fine if guidelines were written by neutral Martians free of dualities of interests. But that’s not the case.

Let me show you some examples of dubious evidence, which a practicing doctor ought to know about and be able to use in clinic.

Example 1: Left atrial appendage occlusion proponents should be able to explain their enthusiasm for a procedure that had two failed regulatory trials against warfarin. PROTECT AF failed to pass FDA muster, and PREVAIL failed to meet noninferiority in its co-primary endpoint for stroke, embolism and CV death. Slide below.

I’ve been told that this is too nuanced for doctors. I disagree.

Example 4: Primary care clinicians don’t get off the hook. I would expect them to be able to explain the NORDICC Trial of screening colonoscopy. In this NEJM-published study, more than 84,000 people were randomized to be invited to screening or not invited. In the end, the risk of dying of colon cancer was 0.28% in the invited group vs 0.31% in the non-invited group and risk of dying was 11.0% in both arms. I do not think it is too technical for a primary care clinician ordering a colonoscopy to understand that the difference in colon cancer death in this most modern trial is 0.03%. And there was no survival advantage. (The response should not be that those who actually had the screening did better; if this is your answer, you fail critical appraisal 101. Because people who get the screen are different from those who did not.)

I will stop now. But my point is that doctors (and advanced practice professionals) are about to face a crisis. That is, when a patient comes to you with an AI-driven thread of questions regarding the evidentiary support of a proposed treatment, the answer better not be…it’s in the guidelines, or, I know it works, or, 500,000 procedures have already been done.

What will doctors be for, if we cannot be better at evidence appraisal than a large language model?

I propose that we reinvigorate the skill of critical appraisal.

If we can learn the biochemistry and organic chemistry, we can surely understand that putting efficacy and safety endpoints into a non-inferiority primary endpoint is an easy way to bias a trial.