❌

Reading view

↗️

Towards trustworthy agentic AI: a comprehensive survey of safety, robustness, privacy, and system security

✇cs.AI, q-bio.NC updates on arXiv.org
By: Jinhu Qi · Muzhi Li · Jiahong Liu · Yuqin Shu · Dianzhi Yu · Shicheng Ma · Wenqian Cui · Yiyang Zhao · Yiyi Chen · Ruoxi Jiang · Irwin King · Zenglin Xu
arXiv:2605.23989v1 Announce Type: new Abstract: Agentic AI systems -- Large Language Models (LLMs) augmented with planning, tool use, memory, and long-horizon interactions -- can execute complex tasks autonomously, but their multi-step trajectories introduce new failure modes that challenge trustworthiness. This survey provides a focused examination of trustworthy agentic AI through two core dimensions that are critical for high-risk deployments: Safety and Robustness, and Privacy and System Security. For each dimension, we clarify key concepts, identify where risks emerge along the agent workflow, and summarize stage-targeted mitigation strategies. Other trustworthiness aspects (value alignment, transparency, fairness, and accountability) are discussed as relevant context rather than parallel chapters. To support consistent comparison and deployment decisions, we consolidate evaluation into a unified metrics-and-benchmarks hub, emphasizing both outcome and process signals (e.g., constraint violations, trace completeness, and adversarial success rates) and offering scenario-to-metric guidance for release gating. We conclude by outlining open challenges such as self-evolving agents, runtime monitoring and verification, privacy-preserving personalization, and the trust-utility trade-off, and present a case study of real-world security failures in open-source agentic systems. Our goal is to serve as a practical reference for researchers and practitioners building trustworthy agentic systems in high-stakes environments.
  •  
  • ↗️
↗️

GlobalDentBench: A Multinational Benchmark for Evaluating LLM Clinical Reasoning in Dentistry with Expert Calibration

✇cs.AI, q-bio.NC updates on arXiv.org
By: Junjie Zhao · Jingyi Liang · Zhenyang Cai · Jiaming Zhang · Zhenwei Wen · Shuzhi Deng · Wenjing Yi · Chunfeng Luo · Hexian Zhang · Junying Chen · Tianrui Liu · Zhuhui Bai · Zixu Zhang · Pradeep Singh · Xiang Liu · Jianquan Li · Nhan L Tran · Falk Schwendicke · Zuolin Jin · Lijian Jin · Liangyi Chen · Wei-fa Yang · Benyou Wang · Junwen Wang · Shan Jiang
arXiv:2605.24636v2 Announce Type: new Abstract: While large language models (LLMs) hold transformative potential for medicine, their reasoning robustness and safety in real-world clinical scenarios remain critically underexplored, particularly in dentistry. Here we introduce GlobalDentBench, the first multinational dental benchmark, featuring a taxonomy that encompasses 14 dental specialties across 88 countries and regions spanning six continents. The benchmark comprises 8,978 expert-validated questions across three formats (multiple-choice, short-answer, and case-based questions) and assesses three progressive reasoning levels: knowledge recall (L1), routine reasoning (L2), and individualized reasoning (L3). To ensure data quality, the automated construction framework was calibrated by six senior dentists, achieving expert agreement rates of 99.98% for multiple-choice and short-answer questions and 96.78% for the more complex case-based questions. Evaluation of 12 frontier LLMs on GlobalDentBench revealed a sharp, stepwise performance degradation with increasing reasoning complexity. Specifically, accuracy plummeted from 81.34% on multiple-choice to 64.53% on short-answer and 22.34% on case-based questions, while declining markedly from 74.01% at L1 to 55.64% at L2 and 35.71% at L3. More critically, risk analysis of real-world dental cases demonstrated an alarming overall unsafe rate of 31.01% in LLM-generated clinical recommendations, with 4.51% posing risks of irreversible patient harm and risks particularly pronounced in specialties such as orthodontics. These findings expose fundamental limitations in the medical reasoning and safety of current LLMs. Consequently, GlobalDentBench provides a scalable foundation for trustworthy clinical AI evaluation, underscoring the urgent need for rigorous validation before the safe deployment of these models in healthcare.
  •  
  • ↗️
↗️

HoloFair: Unified T2I Fairness Evaluation and Fair-GRPO Debiasing

✇cs.AI, q-bio.NC updates on arXiv.org
By: Ruyi Chen · Lu Zhou · Xiaogang Xu · Chiyu Zhang · Jiafei Wu · Liming Fang
arXiv:2605.24687v1 Announce Type: cross Abstract: Text-to-Image (T2I) models have made significant strides in visual realism and semantic consistency, yet they often perpetuate and amplify societal biases. Existing evaluation methods typically address only single-dimensional biases, lacking perspectives to uncover model biases at social-related deeper semantic levels. We introduce HoloFair, a comprehensive benchmark framework for multidimensional demographic bias analysis. Built upon our large-scale fairness-oriented dataset and the SpaFreq (Spatial-Frequency) attribute classifier, this framework proposes the Multi-attribute, Group-wise Bias Index (MGBI) metric, designed to assess both intrinsic diversity and conditional biases. Beyond evaluation, we further introduce Fair-GRPO, a reinforcement-learning-based debiasing method that alters the distribution of generative models through a designed multi-objective reward function. E.g., experiments on the SD3.5-Medium model demonstrate that Fair-GRPO significantly improves multidimensional fairness while maintaining high image quality. We also analyze potential reward hacking phenomena and provide corresponding mitigation strategies. Code and dataset are available at https://github.com/1059684669/HoloFair
  •  
  • ↗️
↗️

Cuproptosis causes meiotic metaphase I arrest by disrupting mitochondrial functions in oocytes

✇Cell Death Discovery nature.com science feeds
By: You-Hui Lu · Can Wang · Lei-Ning Chen · Li-Tao Yi · Ke Xu · Xu-Feng Li · Shu-Chen Liu · Xiao-Yi Chen · Yi-Xiao Li · Qing-Yuan Sun · Qiong Wang · Tie-Gang Meng

Cell Death Discovery, Published online: 23 May 2026; doi:10.1038/s41420-026-03168-x

Cuproptosis causes meiotic metaphase I arrest by disrupting mitochondrial functions in oocytes
  •  
  • ↗️
↗️

Integrative multi-omics analysis identifies stromal-immune crosstalk as a determinant of immunotherapy efficacy and establishes a prognostic signature in gastric cancer

✇Omics in Gastric
By: Fan Zhang · Qingqing Zhang · Shuai Shao · Xiaobo Li · Yi Cheng · Xu Cao · Xiaotang Yu · Zhengming Gao

Comput Biol Chem. 2026 Apr 23;124(Pt 1):109095. doi: 10.1016/j.compbiolchem.2026.109095. Online ahead of print.

ABSTRACT

Immune checkpoint inhibitors like pembrolizumab exhibit variable efficacy in metastatic gastric cancer (GC). This study aimed to identify molecular drivers of pembrolizumab response, explore mechanisms of immune checkpoint inhibitors (ICIs) efficacy, and develop a prognostic signature. Transcriptomic analysis of pembrolizumab-treated GC (TIGER database) identified 165 response-associated differentially expressed genes (DEGs). Functional annotation and single-cell RNA sequencing (scRNA-seq) data from the Gene Expression Omnibus (GEO) revealed that responder-upregulated genes (R-DEGs) were enriched in immune activation pathways and mainly localized to CD8 + T/NK cells. In contrast, non-responder-upregulated genes (D-DEGs) were linked to extracellular matrix (ECM) remodeling and mainly expressed in fibroblasts/endothelial cells. CellChat analysis demonstrated that key DEGs mediate immune-stromal crosstalk via MHC-I and collagen/laminin signaling. A prognostic signature (Lasso-StepCox[forward] Riskscore; LSR: APOD, APOH, BATF2, GJA1, MAGED1, SLC5A1, SLCO2A1, VWF, VCAN) was derived and validated in four independent GC cohorts from the GEO and Cancer Genome Atlas (TCGA) database. Multi-omics analyses showed that LSR-high tumors exhibited aggressive clinicopathological features, increased stromal components, reduced cytotoxic immune infiltration, diminished tumor mutational burden (TMB), and poorer prognosis. Immunohistochemistry (IHC) and spatial transcriptomics in GC showed that stromal VWF/VCAN expression correlates with reduced CD8⁺ T cell granzyme B expression, suggesting T cell dysfunction. High VWF expression in GC predicted poor survival, and a combined VWF/VCAN score showed enhanced prognostic stratification. This study highlights stromal-immune crosstalk as a driver of pembrolizumab resistance and provides a signature as a clinical tool for prognosis and personalized therapy in metastatic GC.

PMID:42068630 | DOI:10.1016/j.compbiolchem.2026.109095

  •  
  • ↗️
↗️

Integrin β3 deficiency unleashes spontaneous pulmonary inflammation by promoting B cell hyperactivation via the CD40-CD40L axis

✇Omics In Lung
By: Shuang Liu · Kaiwen Wang · Jiayi Chen · Yan Shen · Zheng Ruan · Kankan Wang · Xiaodong Xi · Jianhua Mao

Front Immunol. 2026 Mar 24;17:1796926. doi: 10.3389/fimmu.2026.1796926. eCollection 2026.

ABSTRACT

BACKGROUND: Pulmonary immune homeostasis requires tight control of adaptive responses. Integrin β3 is a well-known mediator of cell adhesion and platelet function. However, its role in adaptive immunity, especially in B cell responses, remains unclear.

METHODS: We defined the pulmonary phenotype of constitutive β3-deficient (β3-/-) mice by histopathology. We performed integrated transcriptomic and proteomic profiling of lung tissue to map the molecular signature of spontaneous pulmonary inflammation. We further probed the underlying mechanisms with additional histology and functional assays and tested for biological significance using transcriptomics data from auto-immune disease patients.

RESULTS: β3-/- mice developed spontaneous pulmonary inflammation marked by B cell activation and in situ immune-complex deposition within alveoli. Multi-omics integration implicated the CD40-CD40 Ligand (CD40L) axis as a central driver of this pathology. Mechanistically, loss of β3 enhanced CD40L-CD40 engagement on B cells, resulting in NF-κB pathway hyperactivation. Consistent with our murine data, reduced ITGB3 expression in patients with autoimmune disease correlated with transcriptional signatures of B cell activation and inflammation.

CONCLUSIONS: These results reframe integrin β3 as a threshold regulator of B cell activation. The β3-CD40L-CD40 axis therefore represents a potential therapeutic target for B cell-mediated autoimmune diseases.

PMID:41953039 | PMC:PMC13055533 | DOI:10.3389/fimmu.2026.1796926

  •  
  • ↗️
↗️

Integrin β3 deficiency unleashes spontaneous pulmonary inflammation by promoting B cell hyperactivation via the CD40-CD40L axis

✇(Multiomics OR Omics) AND (Lung OR gastric OR Hepatocellular)
By: Shuang Liu · Kaiwen Wang · Jiayi Chen · Yan Shen · Zheng Ruan · Kankan Wang · Xiaodong Xi · Jianhua Mao

Front Immunol. 2026 Mar 24;17:1796926. doi: 10.3389/fimmu.2026.1796926. eCollection 2026.

ABSTRACT

BACKGROUND: Pulmonary immune homeostasis requires tight control of adaptive responses. Integrin β3 is a well-known mediator of cell adhesion and platelet function. However, its role in adaptive immunity, especially in B cell responses, remains unclear.

METHODS: We defined the pulmonary phenotype of constitutive β3-deficient (β3-/-) mice by histopathology. We performed integrated transcriptomic and proteomic profiling of lung tissue to map the molecular signature of spontaneous pulmonary inflammation. We further probed the underlying mechanisms with additional histology and functional assays and tested for biological significance using transcriptomics data from auto-immune disease patients.

RESULTS: β3-/- mice developed spontaneous pulmonary inflammation marked by B cell activation and in situ immune-complex deposition within alveoli. Multi-omics integration implicated the CD40-CD40 Ligand (CD40L) axis as a central driver of this pathology. Mechanistically, loss of β3 enhanced CD40L-CD40 engagement on B cells, resulting in NF-κB pathway hyperactivation. Consistent with our murine data, reduced ITGB3 expression in patients with autoimmune disease correlated with transcriptional signatures of B cell activation and inflammation.

CONCLUSIONS: These results reframe integrin β3 as a threshold regulator of B cell activation. The β3-CD40L-CD40 axis therefore represents a potential therapeutic target for B cell-mediated autoimmune diseases.

PMID:41953039 | PMC:PMC13055533 | DOI:10.3389/fimmu.2026.1796926

  •  
  • ↗️
↗️

Countering Catastrophic Forgetting of Large Language Models for Better Instruction Following via Weight-Space Model Merging

✇cs.AI, q-bio.NC updates on arXiv.org
By: Mengxian Lyu · Cheng Peng · Ziyi Chen · Mengyuan Zhang · Jieting Li Lu · Yonghui Wu
arXiv:2604.01538v1 Announce Type: cross Abstract: Large language models have been adopted in the medical domain for clinical documentation to reduce clinician burden. However, studies have reported that LLMs often "forget" a significant amount of instruction-following ability when fine-tuned using a task-specific medical dataset, a critical challenge in adopting general-purpose LLMs for clinical applications. This study presents a model merging framework to efficiently adapt general-purpose LLMs to the medical domain by countering this forgetting issue. By merging a clinical foundation model (GatorTronLlama) with a general instruct model (Llama-3.1-8B-Instruct) via interpolation-based merge methods, we seek to derive a domain-adapted model with strong performance on clinical tasks while retaining instruction-following ability. Comprehensive evaluation across medical benchmarks and five clinical generation tasks (e.g., radiology and discharge summarization) shows that merged models can effectively mitigate catastrophic forgetting, preserve clinical domain expertise, and retain instruction-following ability. In addition, our model merging strategies demonstrate training efficiency, achieving performance on par with fully fine-tuned baselines under severely constrained supervision (e.g., 64-shot vs. 256-shot). Consequently, weight-space merging constitutes a highly scalable solution for adapting open-source LLMs to clinical applications, facilitating broader deployment in resource-constrained healthcare environments.
  •  
  • ↗️
↗️

Bridging Large-Model Reasoning and Real-Time Control via Agentic Fast-Slow Planning

✇cs.AI, q-bio.NC updates on arXiv.org
By: Jiayi Chen · Shuai Wang · Guangxu Zhu · Chengzhong Xu
arXiv:2604.01681v1 Announce Type: cross Abstract: Large foundation models enable powerful reasoning for autonomous systems, but mapping semantic intent to reliable real-time control remains challenging. Existing approaches either (i) let Large Language Models (LLMs) generate trajectories directly - brittle, hard to verify, and latency-prone - or (ii) adjust Model Predictive Control (MPC) objectives online - mixing slow deliberation with fast control and blurring interfaces. We propose Agentic Fast-Slow Planning, a hierarchical framework that decouples perception, reasoning, planning, and control across natural timescales. The framework contains two bridges. Perception2Decision compresses scenes into ego-centric topologies using an on-vehicle Vision-Language Model (VLM) detector, then maps them to symbolic driving directives in the cloud with an LLM decision maker - reducing bandwidth and delay while preserving interpretability. Decision2Trajectory converts directives into executable paths: Semantic-Guided A* embeds language-derived soft costs into classical search to bias solutions toward feasible trajectories, while an Agentic Refinement Module adapts planner hyperparameters using feedback and memory. Finally, MPC tracks the trajectories in real time, with optional cloud-guided references for difficult cases. Experiments in CARLA show that Agentic Fast-Slow Planning improves robustness under perturbations, reducing lateral deviation by up to 45% and completion time by over 12% compared to pure MPC and an A*-guided MPC baseline. Code is available at https://github.com/cjychenjiayi/icra2026_AFSP.
  •  
  • ↗️
↗️

DIAL: Decoupling Intent and Action via Latent World Modeling for End-to-End VLA

✇cs.AI, q-bio.NC updates on arXiv.org
By: Yi Chen · Yuying Ge · Hui Zhou · Mingyu Ding · Yixiao Ge · Xihui Liu
arXiv:2603.29844v1 Announce Type: cross Abstract: The development of Vision-Language-Action (VLA) models has been significantly accelerated by pre-trained Vision-Language Models (VLMs). However, most existing end-to-end VLAs treat the VLM primarily as a multimodal encoder, directly mapping vision-language features to low-level actions. This paradigm underutilizes the VLM's potential in high-level decision making and introduces training instability, frequently degrading its rich semantic representations. To address these limitations, we introduce DIAL, a framework bridging high-level decision making and low-level motor execution through a differentiable latent intent bottleneck. Specifically, a VLM-based System-2 performs latent world modeling by synthesizing latent visual foresight within the VLM's native feature space; this foresight explicitly encodes intent and serves as the structural bottleneck. A lightweight System-1 policy then decodes this predicted intent together with the current observation into precise robot actions via latent inverse dynamics. To ensure optimization stability, we employ a two-stage training paradigm: a decoupled warmup phase where System-2 learns to predict latent futures while System-1 learns motor control under ground-truth future guidance within a unified feature space, followed by seamless end-to-end joint optimization. This enables action-aware gradients to refine the VLM backbone in a controlled manner, preserving pre-trained knowledge. Extensive experiments on the RoboCasa GR1 Tabletop benchmark show that DIAL establishes a new state-of-the-art, achieving superior performance with 10x fewer demonstrations than prior methods. Furthermore, by leveraging heterogeneous human demonstrations, DIAL learns physically grounded manipulation priors and exhibits robust zero-shot generalization to unseen objects and novel configurations during real-world deployment on a humanoid robot.
  •  
  • ↗️
↗️

Hepatotoxicity Prediction and Multi-omics Reveal Mitochondrial and Lipid Metabolic Dysregulation in PM<sub>2.5</sub>-Induced Liver Fibrosis

✇Omics in Hepatocellular
By: Hiu-Lok Ngan · Jialing Zhang · Yi Chen · Yuanyuan Song · Zenghua Qi · Zhu Yang · Hong Yan · Zongwei Cai

Environ Health (Wash). 2025 Nov 14;4(3):513-521. doi: 10.1021/envhealth.5c00401. eCollection 2026 Mar 20.

ABSTRACT

Prolonged exposure to fine particulate matter (PM2.5) has been linked to chronic liver injury and cancer. However, an alternative risk assessment method to prospective longitudinal studies of exposome-metabolome interactions for liver inflammation-associated hepatocellular carcinoma (HCC) is lacking. This study investigates the risk of long-term real-world PM2.5 exposure in hepatocarcinogenesis through machine learning techniques. Shotgun mass spectrometry (MS) imaging data were acquired from mouse models across a continuum of fibrosis, cirrhosis, and HCC for training a multiclass classification model to identify "No Risk", "Cancer Risk", and "Cancer". Direct infusion-MS data from PM2.5-exposed mouse livers were analyzed to classify risk. By integrating data-driven and knowledge-based approaches, 14 disease progression biomarkers were identified for modeling. Our results suggest that chronic real-world PM2.5 exposure can induce liver fibrosis, presenting cancer risk. Incorporating metabolomics, lipidomics, and transcriptomics, we propose PM2.5 exposure induces mitochondrial dysfunction, activates AMPK signaling, and increases ceramide accumulation, potentially mediating insulin resistance that contributes to nonalcoholic fatty liver disease and HCC progression. This work represents a significant advancement in assessing hepatotoxicity of environmental toxicants by reducing reliance on traditional animal testing methods. It also underscores the potential of emerging technologies in transforming our understanding of PM2.5 exposure, paving the way for targeted interventions.

PMID:41883379 | PMC:PMC13010293 | DOI:10.1021/envhealth.5c00401

  •  
  • ↗️
↗️

AOI: Turning Failed Trajectories into Training Signals for Autonomous Cloud Diagnosis

✇cs.AI, q-bio.NC updates on arXiv.org
By: Pei Yang · Wanyi Chen · Yuxi Zheng · Xueqian Li · Xiang Li · Haoqin Tu · Jie Xiao · Yifan Pang · Bill Shi · Lynn Ai · Eric Yang
arXiv:2603.03378v1 Announce Type: cross Abstract: Large language model (LLM) agents offer a promising data-driven approach to automating Site Reliability Engineering (SRE), yet their enterprise deployment is constrained by three challenges: restricted access to proprietary data, unsafe action execution under permission-governed environments, and the inability of closed systems to improve from failures. We present AOI (Autonomous Operations Intelligence), a trainable multi-agent framework formulating automated operations as a structured trajectory learning problem under security constraints. Our approach integrates three key components. First, a trainable diagnostic system applies Group Relative Policy Optimization (GRPO) to distill expert-level knowledge into locally deployed open-source models, enabling preference-based learning without exposing sensitive data. Second, a read-write separated execution architecture decomposes operational trajectories into observation, reasoning, and action phases, allowing safe learning while preventing unauthorized state mutation. Third, a Failure Trajectory Closed-Loop Evolver mines unsuccessful trajectories and converts them into corrective supervision signals, enabling continual data augmentation. Evaluated on the AIOpsLab benchmark, our contributions yield cumulative gains. (1) The AOI runtime alone achieves 66.3% best@5 success on all 86 tasks, outperforming the prior state-of-the-art (41.9%) by 24.4 points. (2) Adding Observer GRPO training, a locally deployed 14B model reaches 42.9% avg@1 on 63 held-out tasks with unseen fault types, surpassing Claude Sonnet 4.5. (3) The Evolver converts 37 failed trajectories into diagnostic guidance, improving end-to-end avg@5 by 4.8 points while reducing variance by 35%.
  •  
  • ↗️
↗️

Chat-Based Support Alone May Not Be Enough: Comparing Conversational and Embedded LLM Feedback for Mathematical Proof Learning

✇cs.AI, q-bio.NC updates on arXiv.org
By: Eason Chen · Sophia Judicke · Kayla Beigh · Xinyi Tang · Isabel Wang · Nina Yuan · Zimo Xiao · Chuangji Li · Shizhuo Li · Reed Luttmer · Shreya Singh · Maria Yampolsky · Naman Parikh · Yvonne Zhao · Meiyi Chen · Scarlett Huang · Anishka Mohanty · Gregory Johnson · John Mackey · Jionghao Lin · Ken Koedinger
arXiv:2602.18807v1 Announce Type: cross Abstract: We evaluate GPTutor, an LLM-powered tutoring system for an undergraduate discrete mathematics course. It integrates two LLM-supported tools: a structured proof-review tool that provides embedded feedback on students' written proof attempts, and a chatbot for math questions. In a staggered-access study with 148 students, earlier access was associated with higher homework performance during the interval when only the experimental group could use the system, while we did not observe this performance increase transfer to exam scores. Usage logs show that students with lower self-efficacy and prior exam performance used both components more frequently. Session-level behavioral labels, produced by human coding and scaled using an automated classifier, characterize how students engaged with the chatbot (e.g., answer-seeking or help-seeking). In models controlling for prior performance and self-efficacy, higher chatbot usage and answer-seeking behavior were negatively associated with subsequent midterm performance, whereas proof-review usage showed no detectable independent association. Together, the findings suggest that chatbot-based support alone may not reliably support transfer to independent assessment of math proof-learning outcomes, whereas work-anchored, structured feedback appears less associated with reduced learning.
  •  
  • ↗️
↗️

To Reason or Not to: Selective Chain-of-Thought in Medical Question Answering

✇cs.AI, q-bio.NC updates on arXiv.org
By: Zaifu Zhan · Min Zeng · Shuang Zhou · Yiran Song · Xiaoyi Chen · Yu Hou · Yifan Wu · Yang Ruan · Rui Zhang
arXiv:2602.20130v1 Announce Type: cross Abstract: Objective: To improve the efficiency of medical question answering (MedQA) with large language models (LLMs) by avoiding unnecessary reasoning while maintaining accuracy. Methods: We propose Selective Chain-of-Thought (Selective CoT), an inference-time strategy that first predicts whether a question requires reasoning and generates a rationale only when needed. Two open-source LLMs (Llama-3.1-8B and Qwen-2.5-7B) were evaluated on four biomedical QA benchmarks-HeadQA, MedQA-USMLE, MedMCQA, and PubMedQA. Metrics included accuracy, total generated tokens, and inference time. Results: Selective CoT reduced inference time by 13-45% and token usage by 8-47% with minimal accuracy loss ($\leq$4\%). In some model-task pairs, it achieved both higher accuracy and greater efficiency than standard CoT. Compared with fixed-length CoT, Selective CoT reached similar or superior accuracy at substantially lower computational cost. Discussion: Selective CoT dynamically balances reasoning depth and efficiency by invoking explicit reasoning only when beneficial, reducing redundancy on recall-type questions while preserving interpretability. Conclusion: Selective CoT provides a simple, model-agnostic, and cost-effective approach for medical QA, aligning reasoning effort with question complexity to enhance real-world deployability of LLM-based clinical systems.
  •  
  • ↗️
↗️

Emotion-LLaMAv2 and MMEVerse: A New Framework and Benchmark for Multimodal Emotion Understanding

✇cs.AI, q-bio.NC updates on arXiv.org
By: Xiaojiang Peng · Jingyi Chen · Zebang Cheng · Bao Peng · Fengyi Wu · Yifei Dong · Shuyuan Tu · Qiyu Hu · Huiting Huang · Yuxiang Lin · Jun-Yan He · Kai Wang · Zheng Lian · Zhi-Qi Cheng
arXiv:2601.16449v2 Announce Type: replace-cross Abstract: Understanding human emotions from multimodal signals poses a significant challenge in affective computing and human-robot interaction. While multimodal large language models (MLLMs) have excelled in general vision-language tasks, their capabilities in emotional reasoning remain limited. The field currently suffers from a scarcity of large-scale datasets with high-quality, descriptive emotion annotations and lacks standardized benchmarks for evaluation. Our preliminary framework, Emotion-LLaMA, pioneered instruction-tuned multimodal learning for emotion reasoning but was restricted by explicit face detectors, implicit fusion strategies, and low-quality training data with limited scale. To address these limitations, we present Emotion-LLaMAv2 and the MMEVerse benchmark, establishing an end-to-end pipeline together with a standardized evaluation setting for emotion recognition and reasoning. Emotion-LLaMAv2 introduces three key advances. First, an end-to-end multiview encoder eliminates external face detection and captures nuanced emotional cues via richer spatial and temporal multiview tokens. Second, a Conv Attention pre-fusion module is designed to enable simultaneous local and global multimodal feature interactions external to the LLM backbone. Third, a perception-to-cognition curriculum instruction tuning scheme within the LLaMA2 backbone unifies emotion recognition and free-form emotion reasoning. To support large-scale training and reproducible evaluation, MMEVerse aggregates twelve publicly available emotion datasets, including IEMOCAP, MELD, DFEW, and MAFW, into a unified multimodal instruction format. The data are re-annotated via a multi-agent pipeline involving Qwen2 Audio, Qwen2.5 VL, and GPT 4o, producing 130k training clips and 36k testing clips across 18 evaluation benchmarks.
  •  
  • ↗️
↗️

ALOE: Action-Level Off-Policy Evaluation for Vision-Language-Action Model Post-Training

✇cs.AI, q-bio.NC updates on arXiv.org
By: Rushuai Yang · Hecheng Wang · Chiming Liu · Xiaohan Yan · Yunlong Wang · Xuan Du · Shuoyu Yue · Yongcheng Liu · Chuheng Zhang · Lizhe Qi · Yi Chen · Wei Shan · Maoqing Yao
arXiv:2602.12691v2 Announce Type: replace-cross Abstract: We study how to improve large foundation vision-language-action (VLA) systems through online reinforcement learning (RL) in real-world settings. Central to this process is the value function, which provides learning signals to guide VLA learning from experience. In practice, the value function is estimated from trajectory fragments collected from different data sources, including historical policies and intermittent human interventions. Estimating the value function of current behavior quality from the mixture data is inherently an off-policy evaluation problem. However, prior work often adopts conservative on-policy estimation for stability, which avoids direct evaluation of the current high-capacity policy and limits learning effectiveness. In this paper, we propose ALOE, an action-level off-policy evaluation framework for VLA post-training. ALOE applies chunking-based temporal-difference bootstrapping to evaluate individual action sequences instead of predicting final task outcomes. This design improves effective credit assignment to critical action chunks under sparse rewards and supports stable policy improvement. We evaluate our method on three real-world manipulation tasks, including smartphone packing as a high-precision task, laundry folding as a long-horizon deformable-object task, and bimanual pick-and-place involving multi-object perception. Across all tasks, ALOE improves learning efficiency without compromising execution speed, showing that off-policy RL can be reintroduced in a reliable manner for real-world VLA post-training. Videos and additional materials are available at our project website.
  •  
  • ↗️
There are no more articles
❌