The U.S. government recently took steps to ensure that clinical decision support algorithms are safe for clinical use. The next and larger step will be teaching physicians how to use the algorithms ...
Preparing Physicians for the Clinical Algorithm Era
The U.S. government recently took steps to ensure that clinical decision support algorithms are safe for clinical use. The next and larger step will be teaching physicians how to use the algorithms ...
Medical Algorithms Are Failing Communities Of Color
Medical algorithms routinely make decisions about patients’ health care, yet they are rife with bias. Health equity must be built into the development and deployment of these health algorithms.
Performance of a large language model on the reasoning tasks of a physician
More than 65 years ago, complex clinical diagnostic reasoning cases were introduced as the gold standard for the evaluation of expert medical computing systems, a standard that has held ever since. In this study, we report the results of a physician ...
GraphDx: A Cost-Aware Knowledge-Enhanced Multi-Agent Framework for Sequential Diagnosis
Unlike static question answering, sequential diagnosis mirrors actual clinical workflows: physicians iteratively gather information through symptom inquiries and diagnostic tests, making decisions under uncertainty while balancing diagnostic accuracy against resource costs including monetary expense, patient discomfort, and time. This multi-turn process requires not merely knowledge recall, but systematic reasoning that coordinates information acquisition with hypothesis refinement.
Why Doctors Hate Their Computers
Digitization promises to make medical care easier and more efficient. But are screens coming between doctors and patients?
Business intelligence in the healthcare industry: The utilization of a data-driven approach to support clinical decision making
The pandemic has forced people to use digital technologies and accelerated the digitalization of many businesses. Using digital technologies generates a huge amount of data that are exploited by Business Intelligence (BI) to make decisions and improve the management of firms. This becomes particularly relevant in the healthcare sector where decisions are traditionally made on the physicians’ experience. Much work has been done on applying BI in the healthcare industry. Most of these studies were focused only on IT or medical aspects, while the usage of BI for improving the management of healthcare processes is an under-investigated field. This research aims at filling this gap by investigating whether a decision support system (DSS) model based on the exploitation of data through BI can outperform traditional experience-driven practices for managing processes in the healthcare domain. Focusing on the managing process of the therapeutic path of oncological patients, specifically BRCA-mutated women with breast cancer, a DSS model for benchmarking the costs of various treatment paths was developed in two versions: the first is experience-driven while the second is data-driven. We found that the data-driven version of the DSS model leads to a more accurate estimation of the costs that could potentially be prevented in the treatment of oncological patients, thus enabling significant cost savings. A more informed decision due to a more accurate cost estimation becomes crucial in a context where optimal treatment and unique clinical recommendations for patients are absent, thus permitting a substantial improvement of the decision making in the healthcare industry.
The Medium is the Message: How Non-Clinical Information Shapes Clinical Decisions in LLMs | Proceedings of the 2025 ACM Conference on Fairness, Accountability, and Transparency
There has been a growing interest in the HCI community to study Health, with particular focus in understanding healthcare practices and designing technologies to support and to enhance these practices. A majority of current health studies in HCI have ...
Doctor forced to apologise after AI makes 'scary' error about illegal drugs
Artificial Intelligence is listening to many people's medical appointments, but it is not always hearing things correctly.
UnitedHealth pushed employees to follow an algorithm to cut off Medicare patients' rehab care
UnitedHealth executives pressured clinical staff to follow an algorithm to cut off patients’ rehab care, leading to coverage decisions that may violate Medicare regulations.
Reliability of LLMs as medical assistants for the general public: a randomized preregistered study
Global healthcare providers are exploring the use of large language models (LLMs) to provide medical advice to the public. LLMs now achieve nearly perfect scores on medical licensing exams, but this does not necessarily translate to accurate performance in real-world settings. We tested whether LLMs can assist members of the public in identifying underlying conditions and choosing a course of action (disposition) in ten medical scenarios in a controlled study with 1,298 participants. Participants were randomly assigned to receive assistance from an LLM (GPT-4o, Llama 3, Command R+) or a source of their choice (control). Tested alone, LLMs complete the scenarios accurately, correctly identifying conditions in 94.9% of cases and disposition in 56.3% on average. However, participants using the same LLMs identified relevant conditions in fewer than 34.5% of cases and disposition in fewer than 44.2%, both no better than the control group. We identify user interactions as a challenge to the deployment of LLMs for medical advice. Standard benchmarks for medical knowledge and simulated patient interactions do not predict the failures we find with human participants. Moving forward, we recommend systematic human user testing to evaluate interactive capabilities before public deployments in healthcare.
Reliability of LLMs as medical assistants for the general public: a randomized preregistered study
Global healthcare providers are exploring the use of large language models (LLMs) to provide medical advice to the public. LLMs now achieve nearly perfect scores on medical licensing exams, but this does not necessarily translate to accurate performance in real-world settings. We tested whether LLMs can assist members of the public in identifying underlying conditions and choosing a course of action (disposition) in ten medical scenarios in a controlled study with 1,298 participants. Participants were randomly assigned to receive assistance from an LLM (GPT-4o, Llama 3, Command R+) or a source of their choice (control). Tested alone, LLMs complete the scenarios accurately, correctly identifying conditions in 94.9% of cases and disposition in 56.3% on average. However, participants using the same LLMs identified relevant conditions in fewer than 34.5% of cases and disposition in fewer than 44.2%, both no better than the control group. We identify user interactions as a challenge to the deployment of LLMs for medical advice. Standard benchmarks for medical knowledge and simulated patient interactions do not predict the failures we find with human participants. Moving forward, we recommend systematic human user testing to evaluate interactive capabilities before public deployments in healthcare.
Meta’s New AI Asked for My Raw Health Data—and Gave Me Terrible Advice
Meta’s Muse Spark model offers to analyze users’ health data, including lab results. Beyond the obvious privacy risks, it’s not a capable stand-in for a real doctor.
Implementation Science for AI Integration in Digital Health Systems
We systematically reviewed studies of implementation science frameworks used for healthcare AI deployment (2020-2026). Following PRISMA 2020, we searched MEDLINE, Embase, Web of Science, and Scopus and included 87 empirical studies. CFIR was most common (42.5%), followed by RE-AIM (28.7%) and EPIS (18.4%). The most frequent barriers were data infrastructure limitations (67.8%), clinician trust deficits (58.6%), and regulatory uncertainty (52.9%). Implementation success was associated with organizational readiness (r=0.64, p
ChatGPT Health and what AI can do for a broken system
Healthcare isn’t working for patients or doctors, but AI tools can help.
Chatbots Make Terrible Doctors, New Study Finds
Chatbots provided incorrect, conflicting medical advice, researchers found: “Despite all the hype, AI just isn't ready to take on the role of the physician.”
