Publication date: April 2026

Source: Artificial Intelligence in Medicine, Volume 174

Author(s): Farnaz Kheiri, Shahryar Rahnamayan, Masoud Makrehchi

arXiv:2601.05500v1 Announce Type: new Abstract: Benchmarking the relative capabilities of AI systems, including Large Language Models (LLMs) and Vision Models, typically ignores the impact of uncertainty in the underlying ground truth answers from experts. This ambiguity is particularly consequential in medicine where uncertainty is pervasive. In this paper, we introduce a probabilistic paradigm to theoretically explain how high certainty in ground truth answers is almost always necessary for even an expert to achieve high scores, whereas in datasets with high variation in ground truth answers there may be little difference between a random labeller and an expert. Therefore, ignoring uncertainty in ground truth evaluation data can result in the misleading conclusion that a non-expert has similar performance to that of an expert. Using the probabilistic paradigm, we thus bring forth the concepts of expected accuracy and expected F1 to estimate the score an expert human or system can achieve given ground truth answer variability. Our work leads to the recommendation that when establishing the capability of a system, results should be stratified by probability of the ground truth answer, typically measured by the agreement rate of ground truth experts. Stratification becomes critical when the overall performance drops below a threshold of 80%. Under stratified evaluation, performance comparison becomes more reliable in high certainty bins, mitigating the effect of the key confounding factor -- uncertainty.

arXiv:2601.05529v1 Announce Type: new Abstract: One mistake by an AI system in a safety-critical setting can cost lives. As Large Language Models (LLMs) become integral to robotics decision-making, the physical dimension of risk grows; a single wrong instruction can directly endanger human safety. This paper addresses the urgent need to systematically evaluate LLM performance in scenarios where even minor errors are catastrophic. Through a qualitative evaluation of a fire evacuation scenario, we identified critical failure cases in LLM-based decision-making. Based on these, we designed seven tasks for quantitative assessment, categorized into: Complete Information, Incomplete Information, and Safety-Oriented Spatial Reasoning (SOSR). Complete information tasks utilize ASCII maps to minimize interpretation ambiguity and isolate spatial reasoning from visual processing. Incomplete information tasks require models to infer missing context, testing for spatial continuity versus hallucinations. SOSR tasks use natural language to evaluate safe decision-making in life-threatening contexts. We benchmark various LLMs and Vision-Language Models (VLMs) across these tasks. Beyond aggregate performance, we analyze the implications of a 1% failure rate, highlighting how "rare" errors escalate into catastrophic outcomes. Results reveal serious vulnerabilities: several models achieved a 0% success rate in ASCII navigation, while in a simulated fire drill, models instructed robots to move toward hazardous areas instead of emergency exits. Our findings lead to a sobering conclusion: current LLMs are not ready for direct deployment in safety-critical systems. A 99% accuracy rate is dangerously misleading in robotics, as it implies one out of every hundred executions could result in catastrophic harm. We demonstrate that even state-of-the-art models cannot guarantee safety, and absolute reliance on them creates unacceptable risks.

arXiv:2601.05293v1 Announce Type: cross Abstract: Agentic AI marks an important transition from single-step generative models to systems capable of reasoning, planning, acting, and adapting over long-lasting tasks. By integrating memory, tool use, and iterative decision cycles, these systems enable continuous, autonomous workflows in real-world environments. This survey examines the implications of agentic AI for cybersecurity. On the defensive side, agentic capabilities enable continuous monitoring, autonomous incident response, adaptive threat hunting, and fraud detection at scale. Conversely, the same properties amplify adversarial power by accelerating reconnaissance, exploitation, coordination, and social-engineering attacks. These dual-use dynamics expose fundamental gaps in existing governance, assurance, and accountability mechanisms, which were largely designed for non-autonomous and short-lived AI systems. To address these challenges, we survey emerging threat models, security frameworks, and evaluation pipelines tailored to agentic systems, and analyze systemic risks including agent collusion, cascading failures, oversight evasion, and memory poisoning. Finally, we present three representative use-case implementations that illustrate how agentic AI behaves in practical cybersecurity workflows, and how design choices shape reliability, safety, and operational effectiveness.

arXiv:2505.10282v2 Announce Type: replace-cross Abstract: Clinical evidence underpins informed healthcare decisions, yet integrating it into real-time practice remains challenging due to intensive workloads, complex procedures, and time constraints. This study presents Quicker, an LLM-powered system that automates evidence synthesis and generates clinical recommendations following standard guideline development workflows. Quicker delivers an end-to-end pipeline from clinical questions to recommendations and supports customized decision-making through integrated tools and interactive interfaces. To evaluate how closely Quicker can reproduce guideline development processes, we constructed Q2CRBench-3, a benchmark derived from guideline development records for three diseases. Experiments show that Quicker produces precise question decomposition, expert-aligned retrieval, and near-comprehensive screening. Quicker assistance improved the accuracy of extracted study data, and its recommendations were more comprehensive and coherent than clinician-written ones. In system-level testing, Quicker working with one participant reduced recommendation development to 20-40 min. Overall, the findings demonstrate Quicker's potential to enhance the speed and reliability of evidence-based clinical decision-making.

arXiv:2507.22533v2 Announce Type: replace-cross Abstract: Large Language Models (LLMs) hold significant promise for improving clinical decision support and reducing physician burnout by synthesizing complex, longitudinal cancer Electronic Health Records (EHRs). However, their implementation in this critical field faces three primary challenges: the inability to effectively process the extensive length and fragmented nature of patient records for accurate temporal analysis; a heightened risk of clinical hallucination, as conventional grounding techniques such as Retrieval-Augmented Generation (RAG) do not adequately incorporate process-oriented clinical guidelines; and unreliable evaluation metrics that hinder the validation of AI systems in oncology. To address these issues, we propose CliCARE, a framework for Grounding Large Language Models in Clinical Guidelines for Decision Support over Longitudinal Cancer Electronic Health Records. The framework operates by transforming unstructured, longitudinal EHRs into patient-specific Temporal Knowledge Graphs (TKGs) to capture long-range dependencies, and then grounding the decision support process by aligning these real-world patient trajectories with a normative guideline knowledge graph. This approach provides oncologists with evidence-grounded decision support by generating a high-fidelity clinical summary and an actionable recommendation. We validated our framework using large-scale, longitudinal data from a private Chinese cancer dataset and the public English MIMIC-IV dataset. In these settings, CliCARE significantly outperforms baselines, including leading long-context LLMs and Knowledge Graph-enhanced RAG methods. The clinical validity of our results is supported by a robust evaluation protocol, which demonstrates a high correlation with assessments made by oncologists.

arXiv:2508.13201v2 Announce Type: replace-cross Abstract: The surge in multimodal single-cell omics data exposes limitations in traditional, manually defined analysis workflows. AI agents offer a paradigm shift, enabling adaptive planning, executable code generation, traceable decisions, and real-time knowledge fusion. However, the lack of a comprehensive benchmark critically hinders progress. We introduce a novel benchmarking evaluation system to rigorously assess agent capabilities in single-cell omics analysis. This system comprises: a unified platform compatible with diverse agent frameworks and LLMs; multidimensional metrics assessing cognitive program synthesis, collaboration, execution efficiency, bioinformatics knowledge integration, and task completion quality; and 50 diverse real-world single-cell omics analysis tasks spanning multi-omics, species, and sequencing technologies. Our evaluation reveals that Grok-3-beta achieves state-of-the-art performance among tested agent frameworks. Multi-agent frameworks significantly enhance collaboration and execution efficiency over single-agent approaches through specialized role division. Attribution analyses of agent capabilities identify that high-quality code generation is crucial for task success, and self-reflection has the most significant overall impact, followed by retrieval-augmented generation (RAG) and planning. This work highlights persistent challenges in code generation, long-context handling, and context-aware knowledge retrieval, providing a critical empirical foundation and best practices for developing robust AI agents in computational biology.

The U.S. Food and Drug Administration today announced it is sharing information about the agency’s flexible approach to overseeing chemistry, manufacturing and control (CMC) requirements for cell and gene therapies (CGT).

This follows an investigation by the Guardian that found Google AI Overviews offering misleading information in response to some health-related queries.

(MedPage Today) -- SAN FRANCISCO -- Almost 80% of patients with heavily pretreated advanced/metastatic pancreatic cancer achieved disease control with a novel KRAS G12D inhibitor, a small preliminary study showed. The early results showed that...

We are National Institutes of Health scientists and administrators with more than 50 years of collective civil service.

Or, more accurately, we were NIH scientists and administrators.

Read the rest…

© Adobe

(MedPage Today) -- At the Consumer Electronics Show in Las Vegas, FDA Commissioner Marty Makary, MD, MPH, announced that his agency issued two new guidance documents aimed at easing regulation of digital health products. Both guidance documents...

If you have ever taken a self-driving Uber through downtown LA, you might recognise the strange sense of uncertainty that settles in when there is no driver and no conversation, just a quiet car making assumptions about the world around it. The journey feels fine until the car misreads a shadow or slows abruptly for something harmless. In that moment you see the real issue with autonomy. It does not panic when it should, and that gap between confidence and judgement is where trust is either earned or lost. Much of today’s enterprise AI feels remarkably similar. It is competent without being confident, and efficient without being empathetic, which is why the deciding factor in every successful deployment is no longer computing power but trust.

The MLQ State of AI in Business 2025 [PDF] report puts a sharp number on this. 95% of early AI pilots fail to produce measurable ROI, not because the technology is weak but because it is mismatched to the problems organisations are trying to solve. The pattern repeats itself in industries. Leaders get uneasy when they can’t tell if the output is right, teams are unsure whether dashboards can be trusted, and customers quickly lose patience when an interaction feels automated rather than supported. Anyone who has been locked out of their bank account while the automated recovery system insists their answers are wrong knows how quickly confidence evaporates.

Klarna remains the most publicised example of large-scale automation in action. The company has now halved its workforce since 2022 and says internal AI systems are performing the work of 853 full-time roles, up from 700 earlier this year. Revenues have risen 108%, while average employee compensation has increased 60%, funded in part by those operational gains. Yet the picture is more complicated. Klarna still reported a 95 million dollar quarterly loss, and its CEO has warned that further staff reductions are likely. It shows that automation alone does not create stability. Without accountability and structure, the experience breaks down long before the AI does. As Jason Roos, CEO of CCaaS provider Cirrus, puts it, “Any transformation that unsettles confidence, inside or outside the business, carries a cost you cannot ignore. it can leave you worse off.”

We have already seen what happens when autonomy runs ahead of accountability. The UK’s Department for Work and Pensions used an algorithm that wrongly flagged around 200,000 housing-benefit claims as potentially fraudulent, even though the majority were legitimate. The problem wasn’t the technology. It was the absence of clear ownership over its decisions. When an automated system suspends the wrong account, rejects the wrong claim or creates unnecessary fear, the issue is never just “why did the model misfire?” It’s “who owns the outcome?” Without that answer, trust becomes fragile.

“The missing step is always readiness,” says Roos. “If the process, the data and the guardrails aren’t in place, autonomy doesn’t accelerate performance, it amplifies the weaknesses. Accountability has to come first. Start with the outcome, find where effort is being wasted, check your readiness and governance, and only then automate. Skip those steps and accountability disappears just as fast as the efficiency gains arrive.”

Part of the problem is an obsession with scale without the grounding that makes scale sustainable. Many organisations push toward autonomous agents that can act decisively, yet very few pause to consider what happens when those actions drift outside expected boundaries. The Edelman Trust Barometer [PDF] shows a steady decline in public trust in AI over the past five years, and a joint KPMG and University of Melbourne study found that workers prefer more human involvement in almost half the tasks examined. The findings reinforce a simple point. Trust rarely comes from pushing models harder. It comes from people taking the time to understand how decisions are made, and from governance that behaves less like a brake pedal and more like a steering wheel.

The same dynamics appear on the customer side. PwC’s trust research reveals a wide gulf between perception and reality. Most executives believe customers trust their organisation, while only a minority of customers agree. Other surveys show that transparency helps to close this gap, with large majorities of consumers wanting clear disclosure when AI is used in service experiences. Without that clarity, people do not feel reassured. They feel misled, and the relationship becomes strained. Companies that communicate openly about their AI use are not only protecting trust but also normalising the idea that technology and human support can co-exist.

Some of the confusion stems from the term “agentic AI” itself. Much of the market treats it as something unpredictable or self-directing, when in reality it is workflow automation with reasoning and recall. It is a structured way for systems to make modest decisions inside parameters designed by people. The deployments that scale safely all follow the same sequence. They start with the outcome they want to improve, then look at where unnecessary effort sits in the workflow, then assess whether their systems and teams are ready for autonomy, and only then choose the technology. Reversing that order does not speed anything up. It simply creates faster mistakes. As Roos says, AI should expand human judgement, not replace it.

All of this points toward a wider truth. Every wave of automation eventually becomes a social question rather than a purely technical one. Amazon built its dominance through operational consistency, but it also built a level of confidence that the parcel would arrive. When that confidence dips, customers move on. AI follows the same pattern. You can deploy sophisticated, self-correcting systems, but if the customer feels tricked or misled at any point, the trust breaks. Internally, the same pressures apply. The KPMG global study [PDF] highlights how quickly employees disengage when they do not understand how decisions are made or who is accountable for them. Without that clarity, adoption stalls.

As agentic systems take on more conversational roles, the emotional dimension becomes even more significant. Early reviews of autonomous chat interactions show that people now judge their experience not only by whether they were helped but also by whether the interaction felt attentive and respectful. A customer who feels dismissed rarely keeps the frustration to themselves. The emotional tone of AI is becoming a genuine operational factor, and systems that cannot meet that expectation risk becoming liabilities.

The difficult truth is that technology will continue to move faster than people’s instinctive comfort with it. Trust will always lag behind innovation. That is not an argument against progress. It is an argument for maturity. Every AI leader should be asking whether they would trust the system with their own data, whether they can explain its last decision in plain language, and who steps in when something goes wrong. If those answers are unclear, the organisation is not leading transformation. It is preparing an apology.

Roos puts it simply, “Agentic AI is not the concern. Unaccountable AI is.”

When trust goes, adoption goes, and the project that looked transformative becomes another entry in the 95% failure rate. Autonomy is not the enemy. Forgetting who is responsible is. The organisations that keep a human hand on the wheel will be the ones still in control when the self-driving hype eventually fades.

The post Autonomy without accountability: The real AI risk appeared first on AI News.

DNA doesn’t just sit still inside our cells — it folds, loops, and rearranges in ways that shape how genes behave. Researchers have now mapped this hidden architecture in unprecedented detail, showing how genome structure changes from cell to cell and over time. These insights reveal why many disease-linked mutations outside genes can still cause harm. The findings could speed up the discovery of genetic risks and inspire new ways to target diseases.

This Special Communication introduces a cross-disciplinary framework to integrate computer science principles with public health to address data inequities and ensure that digital and technological advances improve health outcomes for all.

Nature Medicine, Published online: 09 January 2026; doi:10.1038/s41591-025-04170-z

Analysis of a placebo-controlled trial of a BCMA-targeting CAR-T cell therapy in patients with myasthenia gravis shows that CAR-T cell infusion selectively remodels the systemic immune environment, with elimination of BCMA-high plasma cells and activated plasmacytoid dendritic cells and changes in the autoreactive B cell repertoire.

Nature Medicine, Published online: 09 January 2026; doi:10.1038/s41591-025-04171-y

In a randomized, double-blind, placebo-controlled trial comparing autologous mRNA-engineered BCMA-targeting CAR T cell therapy versus placebo in patients with generalized myasthenia gravis, a significantly higher percentage of patients exhibited a reduction in disease activity in the treatment arm than in the placebo arm.

arXiv:2601.04234v1 Announce Type: new Abstract: Artificial General Intelligence (AGI) may face a confrontation question: under what conditions would a rationally self-interested AGI choose to seize power or eliminate human control (a confrontation) rather than remain cooperative? We formalize this in a Markov decision process with a stochastic human-initiated shutdown event. Building on results on convergent instrumental incentives, we show that for almost all reward functions a misaligned agent has an incentive to avoid shutdown. We then derive closed-form thresholds for when confronting humans yields higher expected utility than compliant behavior, as a function of the discount factor $\gamma$, shutdown probability $p$, and confrontation cost $C$. For example, a far-sighted agent ($\gamma=0.99$) facing $p=0.01$ can have a strong takeover incentive unless $C$ is sufficiently large. We contrast this with aligned objectives that impose large negative utility for harming humans, which makes confrontation suboptimal. In a strategic 2-player model (human policymaker vs AGI), we prove that if the AGI's confrontation incentive satisfies $\Delta \ge 0$, no stable cooperative equilibrium exists: anticipating this, a rational human will shut down or preempt the system, leading to conflict. If $\Delta