In 2011, when the early AI engine IBM Watson won a three-game exhibition tournament of Jeopardy, besting two human contestants who were widely considered to be the greatest ever to play the game, it was considered a real breakthrough for AI and a harbinger of even better things to come. Up until that point, AI was best known for its performance in narrow, rule-based endeavors like playing chess. IBM Watson’s win on Jeopardy seemed to portend a future in which statistical, data-driven AI, not handcrafted expert systems, could tackle problems once thought uniquely human.
Instead, IBM Watson ultimately became known for something else: the extreme difficulty of taking AI from the planning to the implementation stage.
Within a few years of the Jeopardy win, IBM brought out Watson for Oncology, a decision-support system for cancer treatments. Early pilots with major cancer centers were announced and expectations ran high that Watson for Oncology would become a transformative clinical decision-support tool. It was not to be.
Reports soon surfaced that IBM Watson’s recommendations often conflicted with real-world oncology practice. For example, it would recommend treatments inappropriate for elderly or frail patients or therapies unavailable in a specific country or health system.
Clinicians who evaluated Watson for Oncology repeatedly cited workflow conflicts such as being forced to re-enter information already loaded into the EHR. Also, Watson’s recommendations were hard to defend by clinicians who had little transparency into the assumptions Watson relied on. As outputs conflicted with experience, trust evaporated.
What happened?
Over time, investigations identified various underlying problems:
While these problems were surprising at the time, they aren’t now. With the proverbial 20-20 hindsight, we can now see that IBM Watson was an early and influential example for how large AI projects would struggle to move from vision to sustained execution.
While Watson for Oncology is one of the best-known AI failures, a similar scenario unfolded with the ingestible sensor that Proteus developed for tracking medication adherence. Technically impressive and clinically validated, it stalled after approval. Over time it became clear that while Proteus had solved a measurement problem: “did the patient take the pill?”, it never addressed how to act on that data. For example, the system sat outside the EHR, requiring separate apps, integrations, security reviews, and support. It assumed that once adherence was visible, the healthcare system would naturally respond. Despite FDA clearance and partnerships with major pharma companies, Proteus declared bankruptcy in 2020.
Google experienced something similar with its screening solution for diabetic retinopathy. It demonstrated near–specialist-level accuracy in detecting diabetic retinopathy as early as 2016, but when it was introduced at scale into actual clinical settings problems arose. Its reliance on high-quality imaging hardware was one problem. It also struggled to integrate with other screening programs. Most critically, in many markets there was no clear reimbursement model for that kind of AI-only screening. In the end, it now stands as another example of how clinical validation alone is insufficient without operational, integration, and economic readiness.
It’s an old truism that “Success teaches us nothing; failure teaches us everything.” So, what are the learnings we can take away from these failures? Here are three.
1. Workflow misalignment
It’s not unusual for developers to start designing based on an idealized version of the proposed clinical use case. That’s fine, except when it’s not, i.e., when the focus on the idealized use case ends up obscuring the complex reality of actual care delivery. Medical care is delivered under intense time pressures. Norms and protocols differ from place to place. For these reasons it’s essential to embed workflow design early, treating clinicians, patients, and care teams as co-developers. The validation that moves a solution forward should be not just “can it work?” but “does it fit seamlessly into medical practice?”
2. Underestimated IT and data coupling
All three examples cited here assumed that integration with EHRs, imaging systems, mobile apps, or enterprise data would take place incrementally. In effect developers assumed they would have time to develop the necessary integrations that would support data quality or clinical workflow. Instead, IT architecture, interoperability and data stewardship have to be treated as first-order design constraints, developed in parallel with the clinical algorithm or sensor, not bolted on afterward.
3. Missing economic and reimbursement logic
All of these solutions lacked clear reimbursement pathways or operational ROI for providers. It’s essential to align regulatory strategy, reimbursement planning and business models from the outset: testing who pays, who benefits, and who bears operational risk before scaling.
The bottom line
As was seen at the outset with Watson for Oncology, the ultimate failure of these products did not come about because the initial idea was bad. Or the underlying algorithms couldn’t do the job. They failed because of a planning-execution gap created by siloed development.
The basic learning doesn’t change: Integrated product development that combines clinical workflow, IT architecture and regulatory and economic design from day one is essential to turning promising inventions into deployable medical products.
Why Suntra
Our three decades of experience in medical technology innovation have equipped us to help you align strategic vision, engineering precision, and innovation agility into a unified, risk-mitigated journey from concept to market.
Visit Suntra MedTech Solutions at https://SuntraMedTech.com/contact.
