The Denario project: Deep knowledge AI agents for scientific discovery

By: Francisco Villaescusa-Navarro · Boris Bolliet · Pablo Villanueva-Domingo · Adrian E. Bayer · Aidan Acquah · Chetana Amancharla · Almog Barzilay-Siegal · Pablo Bermejo · Camille Bilodeau · Pablo C\'ardenas Ram\'irez · Miles Cranmer · Urbano L. Fran\c{c}a · ChangHoon Hahn · Yan-Fei Jiang · Raul Jimenez · Jun-Young Lee · Antonio Lerario · Osman Mamun · Thomas Meier · Anupam A. Ojha · Pavlos Protopapas · Shimanto Roy · David N. Spergel · Pedro Taranc\'on-\'Alvarez · Ujjwal Tiwari · Matteo Viel · Digvijay Wadekar · Chi Wang · Bonny Y. Wang · Licong Xu · Yossi Yovel · Shuwen Yue · Wen-Han Zhou · Qiyao Zhu · Jiajun Zou · \'I\~nigo Zubeldia

3 November 2025 at 13:00

arXiv:2510.26887v1 Announce Type: new Abstract: We present Denario, an AI multi-agent system designed to serve as a scientific research assistant. Denario can perform many different tasks, such as generating ideas, checking the literature, developing research plans, writing and executing code, making plots, and drafting and reviewing a scientific paper. The system has a modular architecture, allowing it to handle specific tasks, such as generating an idea, or carrying out end-to-end scientific analysis using Cmbagent as a deep-research backend. In this work, we describe in detail Denario and its modules, and illustrate its capabilities by presenting multiple AI-generated papers generated by it in many different scientific disciplines such as astrophysics, biology, biophysics, biomedical informatics, chemistry, material science, mathematical physics, medicine, neuroscience and planetary science. Denario also excels at combining ideas from different disciplines, and we illustrate this by showing a paper that applies methods from quantum physics and machine learning to astrophysical data. We report the evaluations performed on these papers by domain experts, who provided both numerical scores and review-like feedback. We then highlight the strengths, weaknesses, and limitations of the current system. Finally, we discuss the ethical implications of AI-driven research and reflect on how such technology relates to the philosophy of science. We publicly release the code at https://github.com/AstroPilot-AI/Denario. A Denario demo can also be run directly on the web at https://huggingface.co/spaces/astropilot-ai/Denario, and the full app will be deployed on the cloud.

Adaptive Data Flywheel: Applying MAPE Control Loops to AI Agent Improvement

cs.AI, q-bio.NC updates on arXiv.org

By: Aaditya Shukla · Sidney Knowles · Meenakshi Madugula · Dave Farris · Ryan Angilly · Santiago Pombo · Anbang Xu · Lu An · Abhinav Balasubramanian · Tan Yu · Jiaxiang Ren · Rama Akkiraju

3 November 2025 at 13:00

arXiv:2510.27051v1 Announce Type: new Abstract: Enterprise AI agents must continuously adapt to maintain accuracy, reduce latency, and remain aligned with user needs. We present a practical implementation of a data flywheel in NVInfo AI, NVIDIA's Mixture-of-Experts (MoE) Knowledge Assistant serving over 30,000 employees. By operationalizing a MAPE-driven data flywheel, we built a closed-loop system that systematically addresses failures in retrieval-augmented generation (RAG) pipelines and enables continuous learning. Over a 3-month post-deployment period, we monitored feedback and collected 495 negative samples. Analysis revealed two major failure modes: routing errors (5.25\%) and query rephrasal errors (3.2\%). Using NVIDIA NeMo microservices, we implemented targeted improvements through fine-tuning. For routing, we replaced a Llama 3.1 70B model with a fine-tuned 8B variant, achieving 96\% accuracy, a 10x reduction in model size, and 70\% latency improvement. For query rephrasal, fine-tuning yielded a 3.7\% gain in accuracy and a 40\% latency reduction. Our approach demonstrates how human-in-the-loop (HITL) feedback, when structured within a data flywheel, transforms enterprise AI agents into self-improving systems. Key learnings include approaches to ensure agent robustness despite limited user feedback, navigating privacy constraints, and executing staged rollouts in production. This work offers a repeatable blueprint for building robust, adaptive enterprise AI agents capable of learning from real-world usage at scale.

Glia: A Human-Inspired AI for Automated Systems Design and Optimization

cs.AI, q-bio.NC updates on arXiv.org

By: Pouya Hamadanian · Pantea Karimi · Arash Nasr-Esfahany · Kimia Noorbakhsh · Joseph Chandler · Ali ParandehGheibi · Mohammad Alizadeh · Hari Balakrishnan

3 November 2025 at 13:00

arXiv:2510.27176v1 Announce Type: new Abstract: Can an AI autonomously design mechanisms for computer systems on par with the creativity and reasoning of human experts? We present Glia, an AI architecture for networked systems design that uses large language models (LLMs) in a human-inspired, multi-agent workflow. Each agent specializes in reasoning, experimentation, and analysis, collaborating through an evaluation framework that grounds abstract reasoning in empirical feedback. Unlike prior ML-for-systems methods that optimize black-box policies, Glia generates interpretable designs and exposes its reasoning process. When applied to a distributed GPU cluster for LLM inference, it produces new algorithms for request routing, scheduling, and auto-scaling that perform at human-expert levels in significantly less time, while yielding novel insights into workload behavior. Our results suggest that by combining reasoning LLMs with structured experimentation, an AI can produce creative and understandable designs for complex systems problems.

An In-depth Study of LLM Contributions to the Bin Packing Problem

cs.AI, q-bio.NC updates on arXiv.org

By: Julien Herrmann · Guillaume Pallez

3 November 2025 at 13:00

arXiv:2510.27353v1 Announce Type: new Abstract: Recent studies have suggested that Large Language Models (LLMs) could provide interesting ideas contributing to mathematical discovery. This claim was motivated by reports that LLM-based genetic algorithms produced heuristics offering new insights into the online bin packing problem under uniform and Weibull distributions. In this work, we reassess this claim through a detailed analysis of the heuristics produced by LLMs, examining both their behavior and interpretability. Despite being human-readable, these heuristics remain largely opaque even to domain experts. Building on this analysis, we propose a new class of algorithms tailored to these specific bin packing instances. The derived algorithms are significantly simpler, more efficient, more interpretable, and more generalizable, suggesting that the considered instances are themselves relatively simple. We then discuss the limitations of the claim regarding LLMs' contribution to this problem, which appears to rest on the mistaken assumption that the instances had previously been studied. Our findings instead emphasize the need for rigorous validation and contextualization when assessing the scientific value of LLM-generated outputs.

VeriMoA: A Mixture-of-Agents Framework for Spec-to-HDL Generation

cs.AI, q-bio.NC updates on arXiv.org

By: Heng Ping · Arijit Bhattacharjee · Peiyu Zhang · Shixuan Li · Wei Yang · Anzhe Cheng · Xiaole Zhang · Jesse Thomason · Ali Jannesari · Nesreen Ahmed · Paul Bogdan

3 November 2025 at 13:00

arXiv:2510.27617v1 Announce Type: new Abstract: Automation of Register Transfer Level (RTL) design can help developers meet increasing computational demands. Large Language Models (LLMs) show promise for Hardware Description Language (HDL) generation, but face challenges due to limited parametric knowledge and domain-specific constraints. While prompt engineering and fine-tuning have limitations in knowledge coverage and training costs, multi-agent architectures offer a training-free paradigm to enhance reasoning through collaborative generation. However, current multi-agent approaches suffer from two critical deficiencies: susceptibility to noise propagation and constrained reasoning space exploration. We propose VeriMoA, a training-free mixture-of-agents (MoA) framework with two synergistic innovations. First, a quality-guided caching mechanism to maintain all intermediate HDL outputs and enables quality-based ranking and selection across the entire generation process, encouraging knowledge accumulation over layers of reasoning. Second, a multi-path generation strategy that leverages C++ and Python as intermediate representations, decomposing specification-to-HDL translation into two-stage processes that exploit LLM fluency in high-resource languages while promoting solution diversity. Comprehensive experiments on VerilogEval 2.0 and RTLLM 2.0 benchmarks demonstrate that VeriMoA achieves 15--30% improvements in Pass@1 across diverse LLM backbones, especially enabling smaller models to match larger models and fine-tuned alternatives without requiring costly training.

Impact of clinical decision support systems (cdss) on clinical outcomes and healthcare delivery in low- and middle-income countries: protocol for a systematic review and meta-analysis

cs.AI, q-bio.NC updates on arXiv.org

By: Garima Jain · Anand Bodade · Sanghamitra Pati

3 November 2025 at 13:00

arXiv:2510.26812v1 Announce Type: cross Abstract: Clinical decision support systems (CDSS) are used to improve clinical and service outcomes, yet evidence from low- and middle-income countries (LMICs) is dispersed. This protocol outlines methods to quantify the impact of CDSS on patient and healthcare delivery outcomes in LMICs. We will include comparative quantitative designs (randomized trials, controlled before-after, interrupted time series, comparative cohorts) evaluating CDSS in World Bank-defined LMICs. Standalone qualitative studies are excluded; mixed-methods studies are eligible only if they report comparative quantitative outcomes, for which we will extract the quantitative component. Searches (from inception to 30 September 2024) will cover MEDLINE, Embase, CINAHL, CENTRAL, Web of Science, Global Health, Scopus, IEEE Xplore, LILACS, African Index Medicus, and IndMED, plus grey sources. Screening and extraction will be performed in duplicate. Risk of bias will be assessed with RoB 2 (randomized trials) and ROBINS-I (non-randomized). Random-effects meta-analysis will be performed where outcomes are conceptually or statistically comparable; otherwise, a structured narrative synthesis will be presented. Heterogeneity will be explored using relative and absolute metrics and a priori subgroups or meta-regression (condition area, care level, CDSS type, readiness proxies, study design).

Frame Semantic Patterns for Identifying Underreporting of Notifiable Events in Healthcare: The Case of Gender-Based Violence

cs.AI, q-bio.NC updates on arXiv.org

By: L\'ivia Dutra · Arthur Lorenzi · La\'is Berno · Franciany Campos · Karoline Biscardi · Kenneth Brown · Marcelo Viridiano · Frederico Belcavello · Ely Matos · Ol\'ivia Guaranha · Erik Santos · Sofia Reinach · Tiago Timponi Torrent

3 November 2025 at 13:00

arXiv:2510.26969v1 Announce Type: cross Abstract: We introduce a methodology for the identification of notifiable events in the domain of healthcare. The methodology harnesses semantic frames to define fine-grained patterns and search them in unstructured data, namely, open-text fields in e-medical records. We apply the methodology to the problem of underreporting of gender-based violence (GBV) in e-medical records produced during patients' visits to primary care units. A total of eight patterns are defined and searched on a corpus of 21 million sentences in Brazilian Portuguese extracted from e-SUS APS. The results are manually evaluated by linguists and the precision of each pattern measured. Our findings reveal that the methodology effectively identifies reports of violence with a precision of 0.726, confirming its robustness. Designed as a transparent, efficient, low-carbon, and language-agnostic pipeline, the approach can be easily adapted to other health surveillance contexts, contributing to the broader, ethical, and explainable use of NLP in public health systems.

MARIA: A Framework for Marginal Risk Assessment without Ground Truth in AI Systems

cs.AI, q-bio.NC updates on arXiv.org

By: Jieshan Chen · Suyu Ma · Qinghua Lu · Sung Une Lee · Liming Zhu

3 November 2025 at 13:00

arXiv:2510.27163v1 Announce Type: cross Abstract: Before deploying an AI system to replace an existing process, it must be compared with the incumbent to ensure improvement without added risk. Traditional evaluation relies on ground truth for both systems, but this is often unavailable due to delayed or unknowable outcomes, high costs, or incomplete data, especially for long-standing systems deemed safe by convention. The more practical solution is not to compute absolute risk but the difference between systems. We therefore propose a marginal risk assessment framework, that avoids dependence on ground truth or absolute risk. It emphasizes three kinds of relative evaluation methodology, including predictability, capability and interaction dominance. By shifting focus from absolute to relative evaluation, our approach equips software teams with actionable guidance: identifying where AI enhances outcomes, where it introduces new risks, and how to adopt such systems responsibly.

MemeArena: Automating Context-Aware Unbiased Evaluation of Harmfulness Understanding for Multimodal Large Language Models

cs.AI, q-bio.NC updates on arXiv.org

By: Zixin Chen · Hongzhan Lin · Kaixin Li · Ziyang Luo · Yayue Deng · Jing Ma

3 November 2025 at 13:00

arXiv:2510.27196v1 Announce Type: cross Abstract: The proliferation of memes on social media necessitates the capabilities of multimodal Large Language Models (mLLMs) to effectively understand multimodal harmfulness. Existing evaluation approaches predominantly focus on mLLMs' detection accuracy for binary classification tasks, which often fail to reflect the in-depth interpretive nuance of harmfulness across diverse contexts. In this paper, we propose MemeArena, an agent-based arena-style evaluation framework that provides a context-aware and unbiased assessment for mLLMs' understanding of multimodal harmfulness. Specifically, MemeArena simulates diverse interpretive contexts to formulate evaluation tasks that elicit perspective-specific analyses from mLLMs. By integrating varied viewpoints and reaching consensus among evaluators, it enables fair and unbiased comparisons of mLLMs' abilities to interpret multimodal harmfulness. Extensive experiments demonstrate that our framework effectively reduces the evaluation biases of judge agents, with judgment results closely aligning with human preferences, offering valuable insights into reliable and comprehensive mLLM evaluations in multimodal harmfulness understanding. Our code and data are publicly available at https://github.com/Lbotirx/MemeArena.

Can LLMs Help You at Work? A Sandbox for Evaluating LLM Agents in Enterprise Environments

cs.AI, q-bio.NC updates on arXiv.org

By: Harsh Vishwakarma · Ankush Agarwal · Ojas Patil · Chaitanya Devaguptapu · Mahesh Chandran

3 November 2025 at 13:00

arXiv:2510.27287v1 Announce Type: cross Abstract: Enterprise systems are crucial for enhancing productivity and decision-making among employees and customers. Integrating LLM based systems into enterprise systems enables intelligent automation, personalized experiences, and efficient information retrieval, driving operational efficiency and strategic growth. However, developing and evaluating such systems is challenging due to the inherent complexity of enterprise environments, where data is fragmented across multiple sources and governed by sophisticated access controls. We present EnterpriseBench, a comprehensive benchmark that simulates enterprise settings, featuring 500 diverse tasks across software engineering, HR, finance, and administrative domains. Our benchmark uniquely captures key enterprise characteristics including data source fragmentation, access control hierarchies, and cross-functional workflows. Additionally, we provide a novel data generation pipeline that creates internally consistent enterprise tasks from organizational metadata. Experiments with state-of-the-art LLM agents demonstrate that even the most capable models achieve only 41.8% task completion, highlighting significant opportunities for improvement in enterprise-focused AI systems.

Best Practices for Biorisk Evaluations on Open-Weight Bio-Foundation Models

cs.AI, q-bio.NC updates on arXiv.org

By: Boyi Wei, Zora Che, Nathaniel Li, Udari Madhushani Sehwag, Jasper G\"otting, Samira Nedungadi, Julian Michael, Summer Yue, Dan Hendrycks, Peter Henderson, Zifan Wang, Seth Donoughe, Mantas Mazeika

3 November 2025 at 13:00

arXiv:2510.27629v1 Announce Type: cross Abstract: Open-weight bio-foundation models present a dual-use dilemma. While holding great promise for accelerating scientific research and drug development, they could also enable bad actors to develop more deadly bioweapons. To mitigate the risk posed by these models, current approaches focus on filtering biohazardous data during pre-training. However, the effectiveness of such an approach remains unclear, particularly against determined actors who might fine-tune these models for malicious use. To address this gap, we propose \eval, a framework to evaluate the robustness of procedures that are intended to reduce the dual-use capabilities of bio-foundation models. \eval assesses models' virus understanding through three lenses, including sequence modeling, mutational effects prediction, and virulence prediction. Our results show that current filtering practices may not be particularly effective: Excluded knowledge can be rapidly recovered in some cases via fine-tuning, and exhibits broader generalizability in sequence modeling. Furthermore, dual-use signals may already reside in the pretrained representations, and can be elicited via simple linear probing. These findings highlight the challenges of data filtering as a standalone procedure, underscoring the need for further research into robust safety and security strategies for open-weight bio-foundation models.

A Systematic Literature Review of Spatio-Temporal Graph Neural Network Models for Time Series Forecasting and Classification

cs.AI, q-bio.NC updates on arXiv.org

By: Flavio Corradini · Flavio Gerosa · Marco Gori · Carlo Lucheroni · Marco Piangerelli · Martina Zannotti

3 November 2025 at 13:00

arXiv:2410.22377v3 Announce Type: replace-cross Abstract: In recent years, spatio-temporal graph neural networks (GNNs) have attracted considerable interest in the field of time series analysis, due to their ability to capture, at once, dependencies among variables and across time points. The objective of this systematic literature review is hence to provide a comprehensive overview of the various modeling approaches and application domains of GNNs for time series classification and forecasting. A database search was conducted, and 366 papers were selected for a detailed examination of the current state-of-the-art in the field. This examination is intended to offer to the reader a comprehensive review of proposed models, links to related source code, available datasets, benchmark models, and fitting results. All this information is hoped to assist researchers in their studies. To the best of our knowledge, this is the first and broadest systematic literature review presenting a detailed comparison of results from current spatio-temporal GNN models applied to different domains. In its final part, this review discusses current limitations and challenges in the application of spatio-temporal GNNs, such as comparability, reproducibility, explainability, poor information capacity, and scalability. This paper is complemented by a GitHub repository at https://github.com/FlaGer99/SLR-Spatio-Temporal-GNN.git providing additional interactive tools to further explore the presented findings.

Artificial Empathy: AI based Mental Health

cs.AI, q-bio.NC updates on arXiv.org

By: Aditya Naik · Jovi Thomas · Teja Sree Mandava · Himavanth Reddy Vemula

3 November 2025 at 13:00

arXiv:2506.00081v2 Announce Type: replace-cross Abstract: Many people suffer from mental health problems but not everyone seeks professional help or has access to mental health care. AI chatbots have increasingly become a go-to for individuals who either have mental disorders or simply want someone to talk to. This paper presents a study on participants who have previously used chatbots and a scenario-based testing of large language model (LLM) chatbots. Our findings indicate that AI chatbots were primarily utilized as a "Five minute therapist" or as a non-judgmental companion. Participants appreciated the anonymity and lack of judgment from chatbots. However, there were concerns about privacy and the security of sensitive information. The scenario-based testing of LLM chatbots highlighted additional issues. Some chatbots were consistently reassuring, used emojis and names to add a personal touch, and were quick to suggest seeking professional help. However, there were limitations such as inconsistent tone, occasional inappropriate responses (e.g., casual or romantic), and a lack of crisis sensitivity, particularly in recognizing red flag language and escalating responses appropriately. These findings can inform both the technology and mental health care industries on how to better utilize AI chatbots to support individuals during challenging emotional periods.

Deep Learning-based Prediction of Clinical Trial Enrollment with Uncertainty Estimates

cs.AI, q-bio.NC updates on arXiv.org

By: Tien Huu Do · Antoine Masquelier · Nae Eoun Lee · Jonathan Crowther

3 November 2025 at 13:00

arXiv:2507.23607v2 Announce Type: replace-cross Abstract: Clinical trials are a systematic endeavor to assess the safety and efficacy of new drugs or treatments. Conducting such trials typically demands significant financial investment and meticulous planning, highlighting the need for accurate predictions of trial outcomes. Accurately predicting patient enrollment, a key factor in trial success, is one of the primary challenges during the planning phase. In this work, we propose a novel deep learning-based method to address this critical challenge. Our method, implemented as a neural network model, leverages pre-trained language models (PLMs) to capture the complexities and nuances of clinical documents, transforming them into expressive representations. These representations are then combined with encoded tabular features via an attention mechanism. To account for uncertainties in enrollment prediction, we enhance the model with a probabilistic layer based on the Gamma distribution, which enables range estimation. We apply the proposed model to predict clinical trial duration, assuming site-level enrollment follows a Poisson-Gamma process. We carry out extensive experiments on real-world clinical trial data, and show that the proposed method can effectively predict the number of patients enrolled at a number of sites for a given clinical trial, outperforming established baseline models.

A Process Mining-Based System For The Analysis and Prediction of Software Development Workflows

cs.AI, q-bio.NC updates on arXiv.org

By: Ant\'ia Dorado · Iv\'an Folgueira · Sof\'ia Mart\'in · Gonzalo Mart\'in · \'Alvaro Porto · Alejandro Ramos · John Wallace

3 November 2025 at 13:00

arXiv:2510.25935v2 Announce Type: replace-cross Abstract: CodeSight is an end-to-end system designed to anticipate deadline compliance in software development workflows. It captures development and deployment data directly from GitHub, transforming it into process mining logs for detailed analysis. From these logs, the system generates metrics and dashboards that provide actionable insights into PR activity patterns and workflow efficiency. Building on this structured representation, CodeSight employs an LSTM model that predicts remaining PR resolution times based on sequential activity traces and static features, enabling early identification of potential deadline breaches. In tests, the system demonstrates high precision and F1 scores in predicting deadline compliance, illustrating the value of integrating process mining with machine learning for proactive software project management.

On the limitation of evaluating machine unlearning using only a single training seed

cs.AI, q-bio.NC updates on arXiv.org

By: Jamie Lanyon · Axel Finke · Petros Andreou · Georgina Cosma

3 November 2025 at 13:00

arXiv:2510.26714v2 Announce Type: replace-cross Abstract: Machine unlearning (MU) aims to remove the influence of certain data points from a trained model without costly retraining. Most practical MU algorithms are only approximate and their performance can only be assessed empirically. Care must therefore be taken to make empirical comparisons as representative as possible. A common practice is to run the MU algorithm multiple times independently starting from the same trained model. In this work, we demonstrate that this practice can give highly non-representative results because -- even for the same architecture and same dataset -- some MU methods can be highly sensitive to the choice of random number seed used for model training. We therefore recommend that empirical comparisons of MU algorithms should also reflect the variability across different model training seeds.

International expert consensus on the clinical integration of circulating tumor cells in solid tumors

31 October 2025 at 18:00

Eur J Cancer. 2025 Dec 9;231:116050. doi: 10.1016/j.ejca.2025.116050. Epub 2025 Oct 20.

ABSTRACT

BACKGROUND: Circulating tumor cells (CTCs) are a versatile biomarker in solid tumors. Extensive research supports their clinical relevance and led to regulatory approval in breast, prostate, and colorectal cancers. However, clinical adoption remains limited mainly due to the lack of consensus and standardized technologies. Additionally, CTC research lacks unified direction. To address these gaps, an international expert panel was established to assess the current and future clinical utility of CTCs.

METHODS: A panel of 11 CTC experts identified key areas of controversy, informing a structured survey distributed to 55 international multidisciplinary experts. Consensus was predefined as ≥ 70 % agreement. Areas without consensus were discussed in a virtual meeting, leading to final statements on the clinical integration of CTCs.

RESULTS: Thirty-seven experts completed the survey. Consensus was reached on the clinical utility of CTCs for prognosis and treatment monitoring in metastatic breast (BC) and prostate (PC) cancers, including AR-V7 testing in metastatic castration-resistant PC for therapy selection. In other tumors, CTCs remain investigational. Experts agreed that while clinical utility is not yet established in early-stage disease, CTCs show promise in early BC, especially combined with cell-free DNA (cfDNA) for minimal residual disease detection. CellSearch® is currently the only platform with high-level evidence for clinical use, though emerging technologies are promising. Key challenges include improving detection sensitivity/specificity, standardizing workflows, generating robust data, and clinician education. Experts emphasized shifting from enumeration to phenotypic and molecular characterization, particularly for treatment guidance, and highlighted the complementary role of CTCs and cfDNA, advocating for integrated liquid biopsy approaches.

CONCLUSIONS: This consensus offers practical guidance for clinical integration of CTCs and outlines strategic research priorities to unlock their full potential in precision oncology.

PMID:41172567 | DOI:10.1016/j.ejca.2025.116050

Reading view