Mayo Clinic's Secret Weapon Against AI Hallucinations: Reverse RAG in Action
The AI Hallucination Problem and Mayo Clinic's Challenge
Large language models (LLMs) are becoming increasingly sophisticated and powerful, yet they still suffer from the "hallucination" problem. In simple terms, they sometimes provide incorrect information or even fabricate facts. In the medical field especially, such errors can have fatal consequences.
Mayo Clinic, one of the top hospitals in the United States, introduced an innovative approach to solve this problem: Reverse RAG (Reverse Retrieval-Augmented Generation). This method works by having the model extract information and then linking every data point back to its original source. This has virtually eliminated data retrieval-based hallucinations in non-diagnostic use cases, enabling its application across clinical practice.
"With this approach of referencing source information, data extraction is no longer a problem."
- Matthew Callstrom, Strategic Medical Director and Chair of Radiology at Mayo Clinic
The Complexity of Medical Data and the First AI Use Case
Medical data is vast and complex, requiring significant time to process. Mayo Clinic first applied AI to writing discharge summaries. Discharge summaries are documents that include a summary of the patient's visit and follow-up care tips. Since this is essentially a data extraction and summarization task, it was well-suited for LLMs.
"In the first stage, we're not trying to make diagnoses. For example, we're not asking the model, 'What is the most appropriate next step for this patient right now?'"
- Matthew Callstrom
However, hallucination problems still occurred initially. For example, there were clearly unacceptable errors such as incorrectly displaying a patient's age. To address this, Mayo Clinic had to carefully build the model.
The Limitations of RAG and the Introduction of Reverse RAG
Traditional RAG is an important technology that helps LLMs retrieve information from specific data sources, but it has several limitations. For example, it can retrieve inappropriate or inaccurate data, fail to judge relevance to the human query, or generate output that doesn't match the requested format.
To solve this problem, Mayo Clinic introduced Reverse RAG. Specifically, they combined the CURE (Clustering Using Representatives) algorithm with LLMs and vector databases to double-check data retrieval.
- CURE Algorithm: A clustering technique that groups data by similarity or patterns. It can particularly detect "outliers" — items that don't match other data.
- Mayo Clinic split the summaries generated by the LLM into individual facts and matched them against original documents. A second LLM then scored how well these facts matched their sources.
"Every data point is referenced back to the original lab source data or imaging report. This system effectively resolves most retrieval-related hallucination issues by verifying that references are real and accurately retrieved."
- Matthew Callstrom
Expanding Mayo Clinic's AI Applications
This technology has also proven useful in integrating new patient records. External records come in various formats with vast amounts of data, making review and summarization time-consuming. AI automated this work, reducing a 90-minute task to 10 minutes.
"External medical records are like a spreadsheet. You can't tell what each cell contains, so you have to check them one by one."
- Matthew Callstrom
Mayo Clinic aims to expand this technology across the hospital to reduce physicians' administrative burden and allow more time for patient care.
"Our goal is to simplify the content processing workflow. How can we augment physicians' abilities and simplify their work?"
- Matthew Callstrom
AI's Potential to Solve More Complex Problems
Mayo Clinic also sees potential for applying AI to more complex problems. For example, they are collaborating with Cerebras Systems to build genomic models and predict the most suitable arthritis treatments for patients. They are also partnering with Microsoft to develop imaging encoders and imaging-based models.
- The first project focuses on chest X-rays, having already converted 1.5 million X-rays, with plans to process an additional 11 million in the next phase.
- AI can go beyond simply analyzing images to, for example, recommend endotracheal tube placement or predict the heart's blood-pumping capacity.
"Now we can begin thinking about predicting treatment responses on a broader scale."
- Matthew Callstrom
Personalized Medicine and the Future of AI
Mayo Clinic also sees AI's potential in fields like genomics and proteomics. AI can analyze a patient's DNA, create reference points with other patients, and suggest risk profiles or treatment pathways for complex diseases.
"Ultimately, what personalized medicine will deliver is this: 'You are similar to these patients. Therefore, treating you this way should yield the expected outcomes.'"
- Matthew Callstrom
However, diagnostic AI applications still require much research and validation. They must start with small datasets, gradually expand test groups, and compare results with existing treatments.
"We recognize that these models have remarkable potential to actually change patient care and diagnostic approaches."
- Matthew Callstrom
Key Terms
- AI Hallucination
- Reverse RAG
- CURE Algorithm
- Medical Data Automation
- Personalized Medicine
- Genomics
- Medical Imaging Analysis
Mayo Clinic's case is an excellent example of how AI can enhance reliability and efficiency in healthcare and realize patient-centered medicine. It will be exciting to see what further changes AI brings to medical innovation.