❌

Normal view

Received — 26 March 2026 ⏭

ImplicitRM: Unbiased Reward Modeling from Implicit Preference Data for LLM alignment

✇cs.AI, q-bio.NC updates on arXiv.org
By: Hao Wang · Haocheng Yang · Licheng Pan · Lei Shen · Xiaoxi Li · Yinuo Wang · Zhichao Chen · Yuan Lu · Haoxuan Li · Zhouchen Lin
25 March 2026 at 12:00
arXiv:2603.23184v1 Announce Type: cross Abstract: Reward modeling represents a long-standing challenge in reinforcement learning from human feedback (RLHF) for aligning language models. Current reward modeling is heavily contingent upon experimental feedback data with high collection costs. In this work, we study \textit{implicit reward modeling} -- learning reward models from implicit human feedback (e.g., clicks and copies) -- as a cost-effective alternative. We identify two fundamental challenges in implicit reward modeling: (1) Implicit preference data lacks definitive negative samples, which makes standard positive-negative classification methods inapplicable; (2) Implicit preference data suffers from user preference bias, where different responses have different propensities to elicit user feedback actions, which exacerbates the difficulty of distinguishing definitive negative samples. To address these challenges, we propose ImplicitRM, which aims to learn unbiased reward models from implicit preference data. ImplicitRM stratifies training samples into four latent groups via a stratification model. Building on this, it derives a learning objective through likelihood maximization, which we prove is theoretically unbiased, effectively resolving both challenges. Experiments demonstrate that ImplicitRM learns accurate reward models across implicit preference datasets. Code is available on our project website.

Hierarchical Long Video Understanding with Audiovisual Entity Cohesion and Agentic Search

✇cs.AI, q-bio.NC updates on arXiv.org
By: Xinlei Yin · Xiulian Peng · Xiao Li · Zhiwei Xiong · Yan Lu
25 March 2026 at 12:00
arXiv:2601.13719v2 Announce Type: replace-cross Abstract: Long video understanding presents significant challenges for vision-language models due to extremely long context windows. Existing solutions relying on naive chunking strategies with retrieval-augmented generation, typically suffer from information fragmentation and a loss of global coherence. We present HAVEN, a unified framework for long-video understanding that enables coherent and comprehensive reasoning by integrating audiovisual entity cohesion and hierarchical video indexing with agentic search. First, we preserve semantic consistency by integrating entity-level representations across visual and auditory streams, while organizing content into a structured hierarchy spanning global summary, scene, segment, and entity levels. Then we employ an agentic search mechanism to enable dynamic retrieval and reasoning across these layers, facilitating coherent narrative reconstruction and fine-grained entity tracking. Extensive experiments demonstrate that our method achieves good temporal coherence, entity consistency, and retrieval efficiency, establishing a new state-of-the-art with an overall accuracy of 84.1% on LVBench. Notably, it achieves outstanding performance in the challenging reasoning category, reaching 80.1%. These results highlight the effectiveness of structured, multimodal reasoning for comprehensive and context-consistent understanding of long-form videos.

When Models Judge Themselves: Unsupervised Self-Evolution for Multimodal Reasoning

✇cs.AI, q-bio.NC updates on arXiv.org
By: Zhengxian Wu · Kai Shi · Chuanrui Zhang · Zirui Liao · Jun Yang · Ni Yang · Qiuying Peng · Luyuan Zhang · Hangrui Xu · Tianhuang Su · Zhenyu Yang · Haonan Lu · Haoqian Wang
25 March 2026 at 12:00
arXiv:2603.21289v2 Announce Type: replace-cross Abstract: Recent progress in multimodal large language models has led to strong performance on reasoning tasks, but these improvements largely rely on high-quality annotated data or teacher-model distillation, both of which are costly and difficult to scale. To address this, we propose an unsupervised self-evolution training framework for multimodal reasoning that achieves stable performance improvements without using human-annotated answers or external reward models. For each input, we sample multiple reasoning trajectories and jointly model their within group structure. We use the Actor's self-consistency signal as a training prior, and introduce a bounded Judge based modulation to continuously reweight trajectories of different quality. We further model the modulated scores as a group level distribution and convert absolute scores into relative advantages within each group, enabling more robust policy updates. Trained with Group Relative Policy Optimization (GRPO) on unlabeled data, our method consistently improves reasoning performance and generalization on five mathematical reasoning benchmarks, offering a scalable path toward self-evolving multimodal models. The code are available at https://github.com/OPPO-Mente-Lab/LLM-Self-Judge.

SIRT3 deacetylates STEAP4 to modulate cuproptosis sensitivity via mitochondrial metabolic reprogramming in HBV-related HCC

✇Omics in Hepatocellular
By: Ze-Bang Du · Xin-Mou Wu · Jia-Ming Lei · Yu-Xin Cai · Jie He · Bo Qian · Xin-Xin He · Wen-Hao Han · Yuan Lu · Xiao-Gang Xia · Han-Ying Zheng · Dong-Bei Guo · You-Liang Yao · Wen-Gang Li · Yu-Chun Lin · Zhong-Ning Lin
17 March 2026 at 18:00

Cell Death Differ. 2026 Mar 16. doi: 10.1038/s41418-026-01713-w. Online ahead of print.

ABSTRACT

Hepatitis B virus (HBV) infection remains a leading etiological driver of hepatocellular carcinoma (HCC). Cuproptosis is a recently defined copper-dependent form of regulated cell death that selectively eliminates mitochondria-dependent cells; whether HBV rewires this vulnerability remains unknown. Here we unveil a novel HBV X protein (HBx)-driven mechanism of cuproptosis evasion. Integrative analysis of clinical specimens, HBx-transgenic (HBx-Tg) mice, and multi-omics datasets revealed marked downregulation of STEAP4 (six-transmembrane epithelial antigen of prostate 4), a metalloreductase essential for cuproptosis sensitivity, in HBV-positive HCC. Mechanistically, HBx attenuates sirtuin 3 (SIRT3), impairing deacetylation of STEAP4 at lysine 404 and abolishing its mitochondrial targeting. Consequently, cells switch from the tricarboxylic acid (TCA) cycle respiration to glycolysis, reducing sensitivity to the copper ionophore elesclomol (ES). Restoring STEAP4 expression or pharmacological activation of SIRT3 with honokiol (HKL) re-instated mitochondrial STEAP4 localization and re-sensitized HBV-related HCC cells to cuproptosis; combination with ES produced synergistic tumor suppression in vitro and in orthotopic models. Collectively, our findings establish the SIRT3-STEAP4 axis as a novel regulator of cuproptosis resistance in HBV-related HCC. HBx-mediated repression of SIRT3 disrupts STEAP4 deacetylation and mitochondrial targeting, fostering metabolic reprogramming and evasion of copper-induced cell death. The results provide a pre-clinical rationale for copper-directed combination strategies in HBV-associated HCC.

PMID:41840161 | DOI:10.1038/s41418-026-01713-w

There are no more articles
❌