0:00
-1:40
Post almost any study that is favorable to CRNA practice and the response is immediate:
“The AANA funded it.”
Sometimes it is even more direct:
“The AANA bought and paid for those results.”
That is usually where the analysis stops.
Nobody discusses the methodology. Nobody asks whether the outcome measures were appropriate. Nobody looks at the statistical analysis or whether the conclusions actually match the data. The AANA paid for it, so the results must be biased.
I see this constantly in anesthesia discussions, especially online. It happens on Reddit, in social-media debates, in advocacy materials, and anywhere else research involving CRNA practice is discussed.
Let me be clear: who paid for a study matters.
When a professional organization funds research that may advance its legislative or professional interests, that is a conflict of interest. It should be disclosed. It should make you look at the study a little sideways.
What it should not do is end the discussion before anyone reads the study.
Funding tells you where the scrutiny should begin. It is not where critical thinking should end.
This is the part that often gets “lost in the sauce” I think.
Research is expensive. Large database studies require access to data, researchers, statisticians, health-policy experts, and a considerable amount of time. Depending on the project, the cost may be substantial. We are talking hundreds of thousands to millions of dollars.
There is also very little commercial value (none?) in studying whether CRNA scope-of-practice laws affect anesthesia complications.
A drug company has an obvious reason to pay for research involving its medication. If the study supports the drug, it may pass FDA scrutiny and then the company may sell more of it.
A medical-device company has the same basic incentive. It pays for research because positive findings may help it sell the device.
That does not automatically mean those studies are invalid either. It means you need to understand the sponsor’s interest and then look closely at the design, the outcome selection, the analysis, the publication process, and whether negative findings were reported.
But there is no drug being sold when someone studies whether CRNAs can practice safely without physician medical direction. There is no proprietary device generating a return on investment. No one is making profit as a result of these studies.
Having said that, the questions are still important because hospitals, legislators, administrators, and regulators keep asking them:
Are CRNAs safe?
Does the data show physician supervision requirements improve outcomes?
Does expanded scope of practice increase anesthesia complications?
Does the anesthesia delivery model affect cost or access?
Those questions are extremely important to us. They are important to rural hospitals and facilities struggling to maintain anesthesia coverage. They are important whenever scope-of-practice legislation is debated.
But who has a financial reason to spend the money to study them?
Usually, it is going to be the professional organizations representing the people directly affected by those policies. If CRNAs and the AANA do not support this research, much of it will probably never be done.
That does not mean the AANA doesn’t have an interest in the outcome. Of course it does. The AANA advocates for CRNA practice, full scope of practice, reimbursement, and professional autonomy. That interest should be stated openly.
But professional interest is not identical to direct commercial profit. Different conflicts create different incentives. They should be identified and evaluated for what they actually are instead of being lumped together as though every funding relationship has the same meaning.
The truth is no one else cares to spend the money to study CRNAs except CRNAs.
In 2016, the American Society of Anesthesiologists published a two-page summary of research comparing anesthesia professionals.
Before getting into the individual studies, the document emphasized that four of the five studies it characterized as favorable to CRNAs had received AANA funding. It then highlighted AANA funding when criticizing the Dulisse and Cromwell study, Pine et al., Needleman and Minnick, and the Hogan cost-effectiveness analysis.
What stood out wasn’t that ASA disclosed the AANA funding. It was that they used it as one of the first reasons readers should doubt those studies, before discussing the methods or results.
That does not mean every criticism in the ASA document was wrong. Some of the criticisms are fair and should be considered.
Claims-based research has limitations. Risk adjustment is key. Billing modifiers are imperfect. A study should not claim more than its data support.
But look at how the funding gets used.
When the AANA funds a study, the funding source itself becomes one of the central criticisms.
When a study favors the ASA’s position, there is much less discussion of the professional, institutional, and economic interests involved in selecting that evidence and using it in advocacy.
Even the 2014 Cochrane review said funding can affect how research is published or interpreted. It also said that does not automatically invalidate the results.
And if we’re going to use that standard, it has to apply to the studies ASA likes too. Silber’s 2000 study is not a clean comparison of independent CRNA care versus physician anesthesiologist care. Its “undirected” category mixed together CRNA-only cases, cases supervised by non-anesthesiologist physicians, and unbilled cases. Its “directed” category also combined physician-anesthesiologist-only cases with medically directed CRNA cases. Even after adjustment, the study was still vulnerable to residual confounding, billing misclassification, differences in case complexity, and major differences between the hospitals where those patients received care.
The Memtsoudis study has a different problem. It looked at “unexpected disposition” after outpatient knee and shoulder procedures. That is not the same thing as measuring anesthesia-related mortality, major complications, or whether one type of anesthesia professional is safer than another. An unexpected admission can happen for all kinds of reasons, including pain, nausea, surgical issues, facility policy, social circumstances, or patient preference. That does not make the study worthless. It means the result should not be stretched into a broader claim than the outcome can support.
So does that make the study wrong? No. It just means don’t claim it says something it never studied.
The AANA does this too, by the way.
The Anesthesia Facts website is an advocacy-curated summary of studies that support CRNA safety, cost-effectiveness, access, and scope of practice. It is useful. It gives readers a place to find the research. But it is still an advocacy page.
It should not replace reading the original articles.
Neither should an ASA summary.
That is the standard I am arguing for. I do not care which association created the summary. Read the original study.
Take the 2016 study titled Scope of Practice Laws and Anesthesia Complications: No Measurable Impact of Certified Registered Nurse Anesthetist Expanded Scope of Practice on Anesthesia-Related Complications.
This is one of the studies that is regularly dismissed because it received AANA support.
The investigators examined millions of commercial insurance claims from inpatient, outpatient, and ambulatory surgery settings. They evaluated whether coded anesthesia-related complications varied according to state scope-of-practice laws and anesthesia delivery model.
They reported no measurable increase in anesthesia-related complications associated with the expanded scope-of-practice classifications or delivery models they studied.
Does the study have limitations?
Of course it does, they ALL do!
It is an observational study. It uses administrative claims data, which exist primarily for billing rather than clinical research. Claims data may not capture every complication or every relevant difference in patient condition. The researchers could only adjust for variables available in the database.
“There are three kinds of lies: lies, damned lies, and statistics.”
—popularized by Mark Twain
The billing modifiers also do not tell us exactly how every anesthesia practice operated.
QZ billing means the CRNA service was billed without physician medical direction. It does not necessarily mean there was no physician anesthesiologist anywhere in the facility or affiliated with the group. It does not tell us whether a physician was available for consultation or had some other involvement.
A QZ practice might be an entirely independent CRNA practice. It could also involve CRNAs and physician anesthesiologists working in the same facility in different rooms without billing under the medical-direction requirements. It could be highly autonomous or relatively restrictive.
What QZ does tell us is that the federal medical-direction billing conditions were not applied to those services. That is important to note. But it does not describe every clinical detail of the practice.
Those are fair criticisms of the study.
What is not fair is identifying those limitations and then pretending the study tells us nothing.
That means the conclusion has to be narrower than some CRNAs want it to be. It does not mean the finding is meaningless.
Within a very large commercial claims database, the researchers did not identify a measurable increase in coded anesthesia-related complications associated with the scope-of-practice and delivery models they evaluated.
That does not prove every CRNA and every physician anesthesiologist is identical. It does not prove every delivery model has exactly the same outcome under every possible circumstance. It does not establish that claims data capture every clinically important anesthesia event.
We should not claim that it does.
But the study also does not become invalid simply because the AANA helped pay for it.
This is where these arguments get ridiculous. Someone finds one limitation and suddenly the whole study has been “debunked.”
Every observational health-services study has limitations.
The real question is whether the limitation is serious enough to invalidate the finding, whether it may have biased the result, and how much it should narrow the conclusion.
Sometimes the study just cannot support the biggest claim people want to make from it.
That does not mean it tells us nothing. It may still give us useful evidence within a much narrower lane.
That is what actual critical appraisal looks like.
The 2018 care-team study by Sun and colleagues is a good example.
I recently wrote about this study because the ASA and anesthesiologist assistant organizations routinely use it as evidence that CRNAs and AAs are equivalent anesthesia providers.
That is not what the study examined.
The researchers compared physician-anesthesiologist-supervised care teams using CRNAs with physician-anesthesiologist-supervised teams using anesthesiologist assistants.
The outcomes included mortality, hospital length of stay, and spending in an older Medicare population.
The study did not compare independent CRNA practice with AA practice because AAs cannot practice independently of a physician anesthesiologist.
It did not evaluate independent crisis management.
It did not establish educational equivalence.
It did not examine whether CRNAs and AAs are interchangeable across all anesthesia environments.
It did not evaluate access to care, autonomous decision-making, or the full professional capabilities of the two groups.
It compared two types of workers functioning inside a physician-anesthesiologist-supervised anesthesia care team and found no significant difference in the broad hospital outcomes it selected.
That is what it studied.
Whether or not the investigators intended it as advocacy, it was a narrow comparison using broad hospital metrics that were not especially anesthesia-specific. The ASA, the AAAA, and many of their members have since turned it into a much broader claim that the study did not establish.
That is the problem.
I am not rejecting the study because of who supported it. I am looking at the methodology, the population, the outcomes, and the conclusions that can reasonably be drawn.
Here’s the point. A study can be perfectly legitimate for the question it actually asked and still get completely oversold afterward. That’s what I think happened here. The problem isn’t always the study itself. Sometimes it’s what people start claiming the study proves.
CRNAs are not immune to this problem.
We sometimes share a favorable title or abstract without reading the entire paper. We sometimes take a single observational study and present it as though it settles every question involving safety, independent practice, or anesthesia delivery models.
That is not good enough.
We cannot demand that everyone else read the research critically while we accept every study that helps our side without asking the same questions.
We should be willing to say:
This is what the study examined.
This is what it found.
These are its limitations.
This is what the evidence reasonably supports.
This is what it does not support.
Then we can explain why the finding still matters.
That is a much stronger position than pretending every study we like is perfect. None of them are.
Look, I don’t think it’s rocket science that the ASA’s interest in this argument is not just about academics.
There’s no way around it. Correcting scope of practice laws, removing antiquated supervision mandates, eliminating costly medical direction style practices absolutely directly affects physician anesthesiologist employment, demand, negotiating leverage, and at the bottom line, wallets.
Look, physician anesthesiologists are generally paid two to three times more than CRNAs in the same labor market. Any anesthesia model that does not require a physician anesthesiologist to participate in or direct the work of multiple CRNAs truly protects their positions and revenue. If the model were to change due to eliminations of barriers to practice for CRNAs and hospitals, along with changes in state laws or requirements that make them feel more comfortable, that would directly impact the bottom line of physician anesthesiologists. This is because the positions they rely on may decrease in avaliability.
That is a clear economic interest. It deserves disclosure and scrutiny just as AANA’s professional interest does.
CRNAs have interests too. Broader scope of practice may affect autonomy, employment options, compensation, and bargaining power.
This is not a situation where one side has a conflict and the other side somehow doesn’t.
The problem is that ASA frequently presents AANA’s interest as inherently disqualifying while paying far less attention to its own interest in maintaining physician-mandated delivery models.
Any reasonable observer can’t look at the way all of this is framed from the 30,000-foot view of the market, the actual preponderance of evidence pertaining to safety and outcomes, and believe this looks like a patient safety or academic disagreement. The more you hear the arguments, the more and more it starts to sound like trade protectionism. Which, by the way, is totally okay as long as you disclose that that is you’re lens in the first place.
At the end of the day, facilities should be able to choose the anesthesia model that works best for their patients, their workforce, their finances, and the laws in their state.
Research should inform that choice.
Fearmongering and selective descriptions of research should not.
There is literally no chance that legislators and administrators are going to sit down and do a statistical review of every anesthesia study handed to them.
I understand that.
But they should at least ask better questions.
What did the study actually compare?
Did it measure outcomes that relate to the policy being debated?
Were the patients and facilities comparable?
Did the sponsor participate in the design, analysis, publication decision, or messaging?
Did the researchers adjust for important differences?
Is the association’s summary broader than the original article?
Is this one study standing by itself, or does it fit with the rest of the evidence?
Most importantly:
Does the data actually support the ‘story’ being sold to me?
At the end of the day, I don’t really care whether the AANA paid for a study, the ASA paid for a study, or a drug company paid for a study.
I care whether the study actually says what people keep claiming it says.
Maybe we’ve been asking the wrong question all along.
Instead of asking, “Who paid for the study?”
Maybe we should start asking,
“Did you actually read it?”
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.