Well, it finally happened: On Thursday July 30th, a veterinary hospital in Oregon filed the first, to my knowledge, lawsuit over an alleged AI misdiagnosis. The suit was first reported by the local news outlet The Oregonian. However, that article is paywalled, and critical details of the case are missing from the first few paragraphs. For the full story, I suggest you read this similar article by The Independent.
LEGAL DISCLAIMER: Before we go further, I want to say that I have not reviewed the medical files nor the full text of the lawsuit (because neither are publicly available), nor do I claim to have inside knowledge about the situation. Everything that follows is based on journalists reporting on the case. All claims by the hospital below are alleged, and I cannot confirm OR refute them either way.
According to reports about the suit, the hospital used the Zoetis Imagyst AI system to review a cytology sample from a rapidly-growing neck mass in an 11-year-old dog. The AI detected mast cells and spindle cells, which can be seen with cancer (a mast cell tumor, or MCT for short). However, the device apparently concluded it was a benign inflammatory lesion.
The hospital then performed surgery to remove the mass. When they sent the chunk of tissue for a biopsy (histopathology), the Oregon State University Veterinary Diagnostic Lab diagnosed it as “an aggressive form of cancer”1.
Upon learning that news, the hospital determined another procedure was warranted:
Because the initial procedure failed to extract all the malignant tissue, the dog was required to undergo a second surgery and later died from post-operative complications, according to the lawsuit.
“The patient’s death resulted directly from the need for a second invasive surgical procedure, particularly within such a short amount of time from the first, which would not have been required if Imagyst had properly identified the tumor as cancerous as Defendant Zoetis represented that it would,” the plaintiffs state in the lawsuit.
Some vets reading this may say, “wait a minute, the dog died from surgical complications, isn’t that on the hospital??” What the lawsuit is really alleging is that Zoetis misrepresented the capabilities of the analyzer they had bought in 2024:
The filing alleges Zoetis falsely promoted the device as “the world’s most capable veterinary AI analyzer” capable of accurately detecting specific cancers, while omitting known operational limitations.
This could be riskier ground for the company. No diagnostic test is perfect, and human pathologists also make mistakes. However, if there are situations the analyzer was not trained on or couldn’t handle well, and the company did not disclose that, it could plausibly raise false advertising claims.
If that were the only complaint, I’m not sure the hospital would have a strong chance of prevailing in their lawsuit. One defense against false advertising is called “puffery,” essentially exaggeration that would not be expected to be taken seriously, such as “We have the world’s best slice of pizza” or “our customer service is unmatched.” That said, specific medical claims are more strictly scrutinized than otherwise harmless examples.
Yet, the hospital’s complaint goes even further:
The suit further alleges that Zoetis modified the initial AI diagnostic report after clinic staff reported the error. Following notification of the misdiagnosis, Zoetis requested a conference call in which company representatives apologized, admitted responsibility and acknowledged that similar errors had occurred with other clients using the Imagyst platform, the complaint states.
I am not a lawyer, but if these allegations are true, it seems like Zoetis could be in real trouble. While acknowledging that I don’t have all the details of how the report was modified2 or what was said, implying they were aware of systemic problems with the Imagyst AI while simultaneously selling it to vets as “the world’s most capable veterinary AI analyzer” is terrible optics at best, and potentially fraud at worst3.
That a case like this popped up does not surprise me in the slightest; I’ve actually been predicting this would happen for years. In previous roles where there was interest in developing automated AI diagnosis systems, I warned anyone who would listen that it would be much harder than they thought, and the risks were very high.
Mast cell tumors are a perfect example. You can have mast cells and/or spindle cells present in benign lesions, and telling the difference between that and malignancy can be challenging. Then there is the matter of high-grade vs low grade tumors: One of the first posts on All Science explored how weird and unusual poorly-granulated and atypical MCT can look:
Another example would be melanoma. Well-differentiated versions of that tumor are pretty easy to spot, even for beginners: there are lots of round cells packed to the gills with green-black melanin pigment. But there are variants like amelanotic or balloon cell melanomas that can mimic other types of cancer ranging from a carcinoma to sarcoma and beyond.
Computer vision AI is not magic: It is essentially a glorified pattern-matching system that learns to recognize the pixels in its image training set based on specific labels (so-called “ground truth”4). AI does not possess an understanding of anatomy or physiology, critical thinking to weigh conflicting data, knowledge of the medical literature, or a sense of ethics. AI cannot consult with colleagues, order additional tests to clarify ambiguous results, identify novel diseases that have not been reported yet, or understand how subtle aspects of the clinical history could change a diagnosis. AI is not able to explain its thought process, say “I’m sorry” when it makes a mistake, or be held legally accountable.
The most surprising thing to me about this case is that it has taken this long for a lawsuit to happen. Radiology AI systems have been in use for longer in veterinary medicine than those for cytology, and as a recent study in JAVMA concluded, “none appeared suitable for clinical use in their current form.”
One thing is clear: This lawsuit will almost certainly NOT be the last.
The most common response I’ve seen to this lawsuit from veterinarians on social media is horror and revulsion (I am in this camp). Bizarrely, a small minority seems to be on the side of the AI companies. I won’t call anyone specific out or quote people verbatim, but the general theme is:
“Well the vets should trust their own judgment and double check the slide.”
If that is our answer, it would require that most veterinarians are accurate and confident in their own cytology skills. Without putting anyone in clinical practice down—I was there and there are so many areas I am weak in!—that assumption seems tenuous at best. And why wouldn’t it be??
Vet school is packed with dozens of subjects, and cytopathology makes up only about 1/3 of a single, semester-long course at most schools. In some programs, a dedicated lab course to go over microscopic slides is an optional elective!
Once people graduate, any skills acquired quickly degrade if they are not often used. According to a 2008 JAVMA study by Christopher et al, the typical practitioner collects a median of 7 cytology samples a week, and sends roughly half to an external lab. Do we think reviewing ~3.5 cases a week in-house provides a strong fundamental base for confidence? Consider that when I was a staff pathologist at a corporate diagnostic lab, my minimum daily caseload was 40!
Here is a short, non-comprehensive list of things I have seen called “mast cells” by vet students and GPs:
Broken cells
Mitotic figures
Ultrasound gel lubricant
Broken cells
Granulated lymphocytes
Macrophages
Not sure if I mentioned yet, but broken cells
As if that were not enough, I’ve heard some veterinarians who questioned the AI readouts were told “the machine knows more than you” (even though there are ZERO published studies validating them!!)
The companies that make these devices might counter, “Hey, you can always upgrade to sending the case out for a pathologist to review it on top of the AI!”
Sure—for extra money.
It’s worth zooming out from the nitty gritty details of the lawsuit or technology to ask:
Because if the goal of AI is improved accuracy over human eyes, clearly we aren’t there yet, not by a long shot.
If the supposed purpose is to help busy practitioners who don’t have time to read their own cytology, but it is so unreliable that they’re expected to double-check their own samples anyway, how does that improve efficiency?
Finally, if the goal is to reduce veterinary costs for pet owners, but you are frequently pushed to upgrade to a path review, the AI step only adds expense.
At the risk of sounding cynical, it seems like the primary “problem” that AI “solves” is how to reduce the cost these diagnostic companies pay for specialist labor so they can boost their profit margins.
Maybe that’s good enough for them. But we don’t have to accept that for our practices or patients.
—Eric
This post is public so feel free to share it 👇
As always, this commentary represents my own personal opinion, and may not reflect the views of my current or previous employers.
It is not clear from the reporting whether they are simply lumping any type of mast cell tumor into the category of “cancer” and considering it aggressive, or that they are specifically claiming it was a high-grade form of MCT. Some kinds of MCT are low-grade and can be fully cured with simple surgery, whereas Kiupel “high-grade” or Patnaik grade III MCT (there are two different systems used to classify them) carry a much worse prognosis and requires more radical treatment. We simply do not have enough information to evaluate further.
If the company provided a human pathologist “overread” for a more accurate diagnosis, that is much less problematic than if they literally edited the original report to change the conclusions of the AI system itself and make it look less wrong in hindsight.
This is not my opinion: The word “fraud” is used in both news articles about the lawsuit, and the hospital reportedly “accuses New Jersey-based company Zoetis Inc. of fraud and violating Oregon’s Unlawful Trade Practices Act.”
Establishing “ground truth” is critical, and trickier than you might expect. How many images from how many patients were included? What variants of the lesions were included? Who did the data annotation: was it a pathologist, a GP veterinarian, or someone without a medical background? Was the ground truth just based on the cytology, or did it include histopathology or clinical follow-up to confirm? The specifics of these choices can have huge impacts on performance.
Comments
Nothing yet. Say the first thing.
Sign in to join the conversation.