Normal view
-
MedPageToday.com - medical news for physicians

-
FDA Moves to 'Get Out of the Way' of Wearables, AI in the Clinic
(MedPage Today) -- At the Consumer Electronics Show in Las Vegas, FDA Commissioner Marty Makary, MD, MPH, announced that his agency issued two new guidance documents aimed at easing regulation of digital health products. Both guidance documents...
-
AI News
-
Autonomy without accountability: The real AI risk
If you have ever taken a self-driving Uber through downtown LA, you might recognise the strange sense of uncertainty that settles in when there is no driver and no conversation, just a quiet car making assumptions about the world around it. The journey feels fine until the car misreads a shadow or slows abruptly for something harmless. In that moment you see the real issue with autonomy. It does not panic when it should, and that gap between confidence and judgement is where trust is either earn
Autonomy without accountability: The real AI risk
If you have ever taken a self-driving Uber through downtown LA, you might recognise the strange sense of uncertainty that settles in when there is no driver and no conversation, just a quiet car making assumptions about the world around it. The journey feels fine until the car misreads a shadow or slows abruptly for something harmless. In that moment you see the real issue with autonomy. It does not panic when it should, and that gap between confidence and judgement is where trust is either earned or lost. Much of todayβs enterprise AI feels remarkably similar. It is competent without being confident, and efficient without being empathetic, which is why the deciding factor in every successful deployment is no longer computing power but trust.
The MLQ State of AI in Business 2025 [PDF] report puts a sharp number on this. 95% of early AI pilots fail to produce measurable ROI, not because the technology is weak but because it is mismatched to the problems organisations are trying to solve. The pattern repeats itself in industries. Leaders get uneasy when they canβt tell if the output is right, teams are unsure whether dashboards can be trusted, and customers quickly lose patience when an interaction feels automated rather than supported. Anyone who has been locked out of their bank account while the automated recovery system insists their answers are wrong knows how quickly confidence evaporates.
Klarna remains the most publicised example of large-scale automation in action. The company has now halved its workforce since 2022 and says internal AI systems are performing the work of 853 full-time roles, up from 700 earlier this year. Revenues have risen 108%, while average employee compensation has increased 60%, funded in part by those operational gains. Yet the picture is more complicated. Klarna still reported a 95 million dollar quarterly loss, and its CEO has warned that further staff reductions are likely. It shows that automation alone does not create stability. Without accountability and structure, the experience breaks down long before the AI does. As Jason Roos, CEO of CCaaS provider Cirrus, puts it, βAny transformation that unsettles confidence, inside or outside the business, carries a cost you cannot ignore. it can leave you worse off.β
We have already seen what happens when autonomy runs ahead of accountability. The UKβs Department for Work and Pensions used an algorithm that wrongly flagged around 200,000 housing-benefit claims as potentially fraudulent, even though the majority were legitimate. The problem wasnβt the technology. It was the absence of clear ownership over its decisions. When an automated system suspends the wrong account, rejects the wrong claim or creates unnecessary fear, the issue is never just βwhy did the model misfire?β Itβs βwho owns the outcome?β Without that answer, trust becomes fragile.
βThe missing step is always readiness,β says Roos. βIf the process, the data and the guardrails arenβt in place, autonomy doesnβt accelerate performance, it amplifies the weaknesses. Accountability has to come first. Start with the outcome, find where effort is being wasted, check your readiness and governance, and only then automate. Skip those steps and accountability disappears just as fast as the efficiency gains arrive.β
Part of the problem is an obsession with scale without the grounding that makes scale sustainable. Many organisations push toward autonomous agents that can act decisively, yet very few pause to consider what happens when those actions drift outside expected boundaries. The Edelman Trust Barometer [PDF] shows a steady decline in public trust in AI over the past five years, and a joint KPMG and University of Melbourne study found that workers prefer more human involvement in almost half the tasks examined. The findings reinforce a simple point. Trust rarely comes from pushing models harder. It comes from people taking the time to understand how decisions are made, and from governance that behaves less like a brake pedal and more like a steering wheel.
The same dynamics appear on the customer side. PwCβs trust research reveals a wide gulf between perception and reality. Most executives believe customers trust their organisation, while only a minority of customers agree. Other surveys show that transparency helps to close this gap, with large majorities of consumers wanting clear disclosure when AI is used in service experiences. Without that clarity, people do not feel reassured. They feel misled, and the relationship becomes strained. Companies that communicate openly about their AI use are not only protecting trust but also normalising the idea that technology and human support can co-exist.
Some of the confusion stems from the term βagentic AIβ itself. Much of the market treats it as something unpredictable or self-directing, when in reality it is workflow automation with reasoning and recall. It is a structured way for systems to make modest decisions inside parameters designed by people. The deployments that scale safely all follow the same sequence. They start with the outcome they want to improve, then look at where unnecessary effort sits in the workflow, then assess whether their systems and teams are ready for autonomy, and only then choose the technology. Reversing that order does not speed anything up. It simply creates faster mistakes. As Roos says, AI should expand human judgement, not replace it.
All of this points toward a wider truth. Every wave of automation eventually becomes a social question rather than a purely technical one. Amazon built its dominance through operational consistency, but it also built a level of confidence that the parcel would arrive. When that confidence dips, customers move on. AI follows the same pattern. You can deploy sophisticated, self-correcting systems, but if the customer feels tricked or misled at any point, the trust breaks. Internally, the same pressures apply. The KPMG global study [PDF] highlights how quickly employees disengage when they do not understand how decisions are made or who is accountable for them. Without that clarity, adoption stalls.
As agentic systems take on more conversational roles, the emotional dimension becomes even more significant. Early reviews of autonomous chat interactions show that people now judge their experience not only by whether they were helped but also by whether the interaction felt attentive and respectful. A customer who feels dismissed rarely keeps the frustration to themselves. The emotional tone of AI is becoming a genuine operational factor, and systems that cannot meet that expectation risk becoming liabilities.
The difficult truth is that technology will continue to move faster than peopleβs instinctive comfort with it. Trust will always lag behind innovation. That is not an argument against progress. It is an argument for maturity. Every AI leader should be asking whether they would trust the system with their own data, whether they can explain its last decision in plain language, and who steps in when something goes wrong. If those answers are unclear, the organisation is not leading transformation. It is preparing an apology.
Roos puts it simply, βAgentic AI is not the concern. Unaccountable AI is.β
When trust goes, adoption goes, and the project that looked transformative becomes another entry in the 95% failure rate. Autonomy is not the enemy. Forgetting who is responsible is. The organisations that keep a human hand on the wheel will be the ones still in control when the self-driving hype eventually fades.
The post Autonomy without accountability: The real AI risk appeared first on AI News.
-
Latest Science News -- ScienceDaily
-
A hidden world inside DNA is finally revealed
DNA doesnβt just sit still inside our cells β it folds, loops, and rearranges in ways that shape how genes behave. Researchers have now mapped this hidden architecture in unprecedented detail, showing how genome structure changes from cell to cell and over time. These insights reveal why many disease-linked mutations outside genes can still cause harm. The findings could speed up the discovery of genetic risks and inspire new ways to target diseases.
A hidden world inside DNA is finally revealed
-
JAMA Health Forum New Online
-
Ten Core Concepts for Ensuring Data Equity in Public Health
This Special Communication introduces a cross-disciplinary framework to integrate computer science principles with public health to address data inequities and ensure that digital and technological advances improve health outcomes for all.
Ten Core Concepts for Ensuring Data Equity in Public Health
-
Nature Medicine
-
BCMA-directed mRNA CAR-T cell therapy for myasthenia gravis: exploratory biomarker analysis of a placebo-controlled phase 2b trial
Nature Medicine, Published online: 09 January 2026; doi:10.1038/s41591-025-04170-zAnalysis of a placebo-controlled trial of a BCMA-targeting CAR-T cell therapy in patients with myasthenia gravis shows that CAR-T cell infusion selectively remodels the systemic immune environment, with elimination of BCMA-high plasma cells and activated plasmacytoid dendritic cells and changes in the autoreactive B cell repertoire.
BCMA-directed mRNA CAR-T cell therapy for myasthenia gravis: exploratory biomarker analysis of a placebo-controlled phase 2b trial
Nature Medicine, Published online: 09 January 2026; doi:10.1038/s41591-025-04170-z
Analysis of a placebo-controlled trial of a BCMA-targeting CAR-T cell therapy in patients with myasthenia gravis shows that CAR-T cell infusion selectively remodels the systemic immune environment, with elimination of BCMA-high plasma cells and activated plasmacytoid dendritic cells and changes in the autoreactive B cell repertoire.-
Nature Medicine
-
BCMA-directed mRNA CAR T cell therapy for myasthenia gravis: a randomized, double-blind, placebo-controlled phase 2b trial
Nature Medicine, Published online: 09 January 2026; doi:10.1038/s41591-025-04171-yIn a randomized, double-blind, placebo-controlled trial comparing autologous mRNA-engineered BCMA-targeting CAR T cell therapy versus placebo in patients with generalized myasthenia gravis, a significantly higher percentage of patients exhibited a reduction in disease activity in the treatment arm than in the placebo arm.
BCMA-directed mRNA CAR T cell therapy for myasthenia gravis: a randomized, double-blind, placebo-controlled phase 2b trial
Nature Medicine, Published online: 09 January 2026; doi:10.1038/s41591-025-04171-y
In a randomized, double-blind, placebo-controlled trial comparing autologous mRNA-engineered BCMA-targeting CAR T cell therapy versus placebo in patients with generalized myasthenia gravis, a significantly higher percentage of patients exhibited a reduction in disease activity in the treatment arm than in the placebo arm.-
cs.AI, q-bio.NC updates on arXiv.org
-
Formal Analysis of AGI Decision-Theoretic Models and the Confrontation Question
arXiv:2601.04234v1 Announce Type: new Abstract: Artificial General Intelligence (AGI) may face a confrontation question: under what conditions would a rationally self-interested AGI choose to seize power or eliminate human control (a confrontation) rather than remain cooperative? We formalize this in a Markov decision process with a stochastic human-initiated shutdown event. Building on results on convergent instrumental incentives, we show that for almost all reward functions a misaligned agen
Formal Analysis of AGI Decision-Theoretic Models and the Confrontation Question
-
cs.AI, q-bio.NC updates on arXiv.org
-
Systems Explaining Systems: A Framework for Intelligence and Consciousness
arXiv:2601.04269v1 Announce Type: new Abstract: This paper proposes a conceptual framework in which intelligence and consciousness emerge from relational structure rather than from prediction or domain-specific mechanisms. Intelligence is defined as the capacity to form and integrate causal connections between signals, actions, and internal states. Through context enrichment, systems interpret incoming information using learned relational structure that provides essential context in an efficien
Systems Explaining Systems: A Framework for Intelligence and Consciousness
-
cs.AI, q-bio.NC updates on arXiv.org
-
An ASP-based Solution to the Medical Appointment Scheduling Problem
arXiv:2601.04274v1 Announce Type: new Abstract: This paper presents an Answer Set Programming (ASP)-based framework for medical appointment scheduling, aimed at improving efficiency, reducing administrative overhead, and enhancing patient-centered care. The framework personalizes scheduling for vulnerable populations by integrating Blueprint Personas. It ensures real-time availability updates, conflict-free assignments, and seamless interoperability with existing healthcare platforms by central
An ASP-based Solution to the Medical Appointment Scheduling Problem
-
cs.AI, q-bio.NC updates on arXiv.org
-
Sci-Reasoning: A Dataset Decoding AI Innovation Patterns
arXiv:2601.04577v1 Announce Type: new Abstract: While AI innovation accelerates rapidly, the intellectual process behind breakthroughs -- how researchers identify gaps, synthesize prior work, and generate insights -- remains poorly understood. The lack of structured data on scientific reasoning hinders systematic analysis and development of AI research agents. We introduce Sci-Reasoning, the first dataset capturing the intellectual synthesis behind high-quality AI research. Using community-vali
Sci-Reasoning: A Dataset Decoding AI Innovation Patterns
-
cs.AI, q-bio.NC updates on arXiv.org
-
Autonomous Agents on Blockchains: Standards, Execution Models, and Trust Boundaries
arXiv:2601.04583v1 Announce Type: new Abstract: Advances in large language models have enabled agentic AI systems that can reason, plan, and interact with external tools to execute multi-step workflows, while public blockchains have evolved into a programmable substrate for value transfer, access control, and verifiable state transitions. Their convergence introduces a high-stakes systems challenge: designing standard, interoperable, and secure interfaces that allow agents to observe on-chain s
Autonomous Agents on Blockchains: Standards, Execution Models, and Trust Boundaries
-
cs.AI, q-bio.NC updates on arXiv.org
-
ResMAS: Resilience Optimization in LLM-based Multi-agent Systems
arXiv:2601.04694v1 Announce Type: new Abstract: Large Language Model-based Multi-Agent Systems (LLM-based MAS), where multiple LLM agents collaborate to solve complex tasks, have shown impressive performance in many areas. However, MAS are typically distributed across different devices or environments, making them vulnerable to perturbations such as agent failures. While existing works have studied the adversarial attacks and corresponding defense strategies, they mainly focus on reactively det
ResMAS: Resilience Optimization in LLM-based Multi-agent Systems
-
cs.AI, q-bio.NC updates on arXiv.org
-
Beyond Monolithic Architectures: A Multi-Agent Search and Knowledge Optimization Framework for Agentic Search
arXiv:2601.04703v1 Announce Type: new Abstract: Agentic search has emerged as a promising paradigm for complex information seeking by enabling Large Language Models (LLMs) to interleave reasoning with tool use. However, prevailing systems rely on monolithic agents that suffer from structural bottlenecks, including unconstrained reasoning outputs that inflate trajectories, sparse outcome-level rewards that complicate credit assignment, and stochastic search noise that destabilizes learning. To a
Beyond Monolithic Architectures: A Multi-Agent Search and Knowledge Optimization Framework for Agentic Search
-
cs.AI, q-bio.NC updates on arXiv.org
-
The Artificial Intelligence Value Chain: A Critical Appraisal. [Spanish Version]
arXiv:2601.04218v1 Announce Type: cross Abstract: The artificial intelligence value chain is one of the main concepts underpinning the European legislation on the subject, especially the Artificial Intelligence Act. It is an economic concept that has become a legal one. i.e., a concept of legal governance, due to its continued use in policy documents and legal texts. This article (i) analyses its significance and function within the framework of the regulatory strategy established by recent EU
The Artificial Intelligence Value Chain: A Critical Appraisal. [Spanish Version]
-
cs.AI, q-bio.NC updates on arXiv.org
-
Balancing Usability and Compliance in AI Smart Devices: A Privacy-by-Design Audit of Google Home, Alexa, and Siri
arXiv:2601.04403v1 Announce Type: cross Abstract: This paper investigates the privacy and usability of AI-enabled smart devices commonly used by youth, focusing on Google Home Mini, Amazon Alexa, and Apple Siri. While these devices provide convenience and efficiency, they also raise privacy and transparency concerns due to their always-listening design and complex data management processes. The study proposes and applies a combined framework of Heuristic Evaluation, Personal Information Protect
Balancing Usability and Compliance in AI Smart Devices: A Privacy-by-Design Audit of Google Home, Alexa, and Siri
-
cs.AI, q-bio.NC updates on arXiv.org
-
Decision-Aware Trust Signal Alignment for SOC Alert Triage
arXiv:2601.04486v1 Announce Type: cross Abstract: Detection systems that utilize machine learning are progressively implemented at Security Operations Centers (SOCs) to help an analyst to filter through high volumes of security alerts. Practically, such systems tend to reveal probabilistic results or confidence scores which are ill-calibrated and hard to read when under pressure. Qualitative and survey based studies of SOC practice done before reveal that poor alert quality and alert overload g
Decision-Aware Trust Signal Alignment for SOC Alert Triage
-
cs.AI, q-bio.NC updates on arXiv.org
-
Surface-based Molecular Design with Multi-modal Flow Matching
arXiv:2601.04506v1 Announce Type: cross Abstract: Therapeutic peptides show promise in targeting previously undruggable binding sites, with recent advancements in deep generative models enabling full-atom peptide co-design for specific protein receptors. However, the critical role of molecular surfaces in protein-protein interactions (PPIs) has been underexplored. To bridge this gap, we propose an omni-design peptides generation paradigm, called SurfFlow, a novel surface-based generative algori
Surface-based Molecular Design with Multi-modal Flow Matching
-
cs.AI, q-bio.NC updates on arXiv.org
-
Self-MedRAG: a Self-Reflective Hybrid Retrieval-Augmented Generation Framework for Reliable Medical Question Answering
arXiv:2601.04531v1 Announce Type: cross Abstract: Large Language Models (LLMs) have demonstrated significant potential in medical Question Answering (QA), yet they remain prone to hallucinations and ungrounded reasoning, limiting their reliability in high-stakes clinical scenarios. While Retrieval-Augmented Generation (RAG) mitigates these issues by incorporating external knowledge, conventional single-shot retrieval often fails to resolve complex biomedical queries requiring multi-step inferen
Self-MedRAG: a Self-Reflective Hybrid Retrieval-Augmented Generation Framework for Reliable Medical Question Answering
-
cs.AI, q-bio.NC updates on arXiv.org
-
A Vision for Multisensory Intelligence: Sensing, Synergy, and Science
arXiv:2601.04563v1 Announce Type: cross Abstract: Our experience of the world is multisensory, spanning a synthesis of language, sight, sound, touch, taste, and smell. Yet, artificial intelligence has primarily advanced in digital modalities like text, vision, and audio. This paper outlines a research vision for multisensory artificial intelligence over the next decade. This new set of technologies can change how humans and AI experience and interact with one another, by connecting AI to the hu
A Vision for Multisensory Intelligence: Sensing, Synergy, and Science
-
cs.AI, q-bio.NC updates on arXiv.org
-
FedKDX: Federated Learning with Negative Knowledge Distillation for Enhanced Healthcare AI Systems
arXiv:2601.04587v1 Announce Type: cross Abstract: This paper introduces FedKDX, a federated learning framework that addresses limitations in healthcare AI through Negative Knowledge Distillation (NKD). Unlike existing approaches that focus solely on positive knowledge transfer, FedKDX captures both target and non-target information to improve model generalization in healthcare applications. The framework integrates multiple knowledge transfer techniques--including traditional knowledge distilla