❌

Normal view

Xpertbench: Expert Level Tasks with Rubrics-Based Evaluation

arXiv:2604.02368v3 Announce Type: replace Abstract: As Large Language Models (LLMs) exhibit plateauing performance on conventional benchmarks, a pivotal challenge persists: evaluating their proficiency in complex, open-ended tasks characterizing genuine expert-level cognition. Existing frameworks suffer from narrow domain coverage, reliance on generalist tasks, or self-evaluation biases. To bridge this gap, we present XpertBench, a high-fidelity benchmark engineered to assess LLMs across authentic professional domains. XpertBench consists of 1,346 meticulously curated tasks across 80 categories, spanning finance, healthcare, legal services, education, and dual-track research (STEM and Humanities). These tasks are derived from over 1,000 submissions by domain experts--including researchers from elite institutions and practitioners with extensive clinical or industrial experience--ensuring superior ecological validity. Each task uses detailed rubrics with mostly 15-40 weighted checkpoints to assess professional rigor. To facilitate scalable yet human-aligned assessment, we introduce ShotJudge, a novel evaluation paradigm that employs LLM judges calibrated with expert few-shot exemplars to mitigate self-rewarding biases. Our empirical evaluation of state-of-the-art LLMs reveals a pronounced performance ceiling: even leading models achieve a peak success rate of only ~66%, with a mean score around 55%. Models also exhibit domain-specific divergence, showing non-overlapping strengths in quantitative reasoning versus linguistic synthesis.. These findings underscore a significant "expert-gap" in current AI systems and establish XpertBench as a critical instrument for navigating the transition from general-purpose assistants to specialized professional collaborators.

Automatic In-Domain Exemplar Construction and LLM-Based Refinement of Multi-LLM Expansions for Query Expansion

arXiv:2602.08917v2 Announce Type: replace-cross Abstract: Query expansion with large language models is promising but often relies on hand-crafted prompts, manually chosen exemplars, or a single LLM, making it non-scalable and sensitive to domain shift. We present an automated, domain-adaptive QE framework that builds in-domain exemplar pools by harvesting pseudo-relevant passages using a BM25-MonoT5 pipeline. A training-free cluster-based strategy selects diverse demonstrations, yielding strong and stable in-context QE without supervision. To further exploit model complementarity, we introduce a two-LLM ensemble in which two heterogeneous LLMs independently generate expansions and a refinement LLM consolidates them into one coherent expansion. Across TREC DL20, DBPedia, and SciFact, the refined ensemble delivers consistent and statistically significant gains over BM25, Rocchio, zero-shot, and fixed few-shot baselines. The framework offers a reproducible testbed for exemplar selection and multi-LLM generation, and a practical, label-free solution for real-world QE.

Integrated Network Toxicology and Metabolomics Elucidate Mechanisms of Carbosulfan-Induced Respiratory Toxicity in Rats

Int J Mol Sci. 2026 Feb 25;27(5):2170. doi: 10.3390/ijms27052170.

ABSTRACT

Carbosulfan is a widely used carbamate insecticide, yet its mechanisms of respiratory toxicity remain poorly understood. This study integrated network toxicology, untargeted metabolomics, and molecular docking to systematically investigate the potential mechanisms of carbosulfan-induced respiratory toxicity in male Sprague Dawley rats. Rats were administered a single oral dose of carbosulfan (125 or 250 mg/kg) and assessed after 12 h. Exposure resulted in significant pathological lung damage, characterized by disrupted alveolar architecture, inflammatory cell infiltration, and increased serum levels of the pro-inflammatory cytokines IL-6, IL-1β, and TNF-α. Network toxicology analysis identified 51 potential targets associated with respiratory toxicity, with core targets including SRC, EGFR, PTGS2, CXCL8, CYP3A4, and NR3C1. Enriched pathways were primarily related to neuroactive ligand-receptor interaction, VEGF signaling, and arachidonic acid metabolism. Untargeted metabolomics revealed significant metabolic perturbations in pathways central to antioxidant defense and energy homeostasis, including glutathione metabolism, the tricarboxylic acid cycle, and arginine biosynthesis. Molecular docking confirmed stable in silico binding affinities between carbosulfan and the predicted core targets. Integrative analysis suggests that carbosulfan exposure is associated with respiratory damage, potentially through interconnected mechanisms involving oxidative stress, inflammation, and disruption of cell signaling and metabolic enzyme systems. However, given the acute high-dose nature of the model and the interpretative integration of multi-omics data, these findings should be considered hypothesis-generating. This study provides a novel system-level perspective on carbosulfan-induced respiratory toxicity and highlights key pathways and targets for future validation in chronic exposure models.

PMID:41828400 | PMC:PMC12984169 | DOI:10.3390/ijms27052170

Integrated Network Toxicology and Metabolomics Elucidate Mechanisms of Carbosulfan-Induced Respiratory Toxicity in Rats

Int J Mol Sci. 2026 Feb 25;27(5):2170. doi: 10.3390/ijms27052170.

ABSTRACT

Carbosulfan is a widely used carbamate insecticide, yet its mechanisms of respiratory toxicity remain poorly understood. This study integrated network toxicology, untargeted metabolomics, and molecular docking to systematically investigate the potential mechanisms of carbosulfan-induced respiratory toxicity in male Sprague Dawley rats. Rats were administered a single oral dose of carbosulfan (125 or 250 mg/kg) and assessed after 12 h. Exposure resulted in significant pathological lung damage, characterized by disrupted alveolar architecture, inflammatory cell infiltration, and increased serum levels of the pro-inflammatory cytokines IL-6, IL-1β, and TNF-α. Network toxicology analysis identified 51 potential targets associated with respiratory toxicity, with core targets including SRC, EGFR, PTGS2, CXCL8, CYP3A4, and NR3C1. Enriched pathways were primarily related to neuroactive ligand-receptor interaction, VEGF signaling, and arachidonic acid metabolism. Untargeted metabolomics revealed significant metabolic perturbations in pathways central to antioxidant defense and energy homeostasis, including glutathione metabolism, the tricarboxylic acid cycle, and arginine biosynthesis. Molecular docking confirmed stable in silico binding affinities between carbosulfan and the predicted core targets. Integrative analysis suggests that carbosulfan exposure is associated with respiratory damage, potentially through interconnected mechanisms involving oxidative stress, inflammation, and disruption of cell signaling and metabolic enzyme systems. However, given the acute high-dose nature of the model and the interpretative integration of multi-omics data, these findings should be considered hypothesis-generating. This study provides a novel system-level perspective on carbosulfan-induced respiratory toxicity and highlights key pathways and targets for future validation in chronic exposure models.

PMID:41828400 | PMC:PMC12984169 | DOI:10.3390/ijms27052170

MAPK14/SLC7A11/GPX4 axis dysregulation drives podocyte ferroptosis via mediating glycerophospholipid metabolism

Cell Death Discovery, Published online: 11 March 2026; doi:10.1038/s41420-026-02990-7

MAPK14/SLC7A11/GPX4 axis dysregulation drives podocyte ferroptosis via mediating glycerophospholipid metabolism

VITA: Vision-to-Action Flow Matching Policy

arXiv:2507.13231v4 Announce Type: replace-cross Abstract: Conventional flow matching and diffusion-based policies sample via iterative denoising from standard noise distributions (e.g., Gaussian), and require conditioning modules to repeatedly incorporate visual information during the generative process, incurring substantial time and memory overhead. To reduce the complexity, we develop VITA, VIsion-To-Action policy, a noise-free and conditioning-free flow matching policy learning framework that directly flows from visual representations to latent actions. Since the source of the flow is visually grounded, VITA eliminates the need for visual conditioning during generation. As expected, bridging vision and action is challenging, because actions are lower-dimensional, less structured, and sparser than visual representations; moreover, flow matching requires the source and target to have the same dimensionality. To overcome this, we introduce an action autoencoder that maps raw actions into a structured latent space aligned with visual latents, trained jointly with flow matching. To further prevent latent action space collapse during end-to-end training, we propose flow latent decoding, which anchors the latent generation process by backpropagating the action reconstruction loss through the flow matching ODE (ordinary differential equation) solving steps. We evaluate VITA on 9 simulation and 5 real-world tasks from ALOHA and Robomimic. VITA achieves 1.5x-2x faster inference compared to conventional methods with conditioning modules, while outperforming or matching state-of-the-art policies. Project page: https://ucd-dare.github.io/VITA/.

A Very Big Video Reasoning Suite

arXiv:2602.20159v1 Announce Type: cross Abstract: Rapid progress in video models has largely focused on visual quality, leaving their reasoning capabilities underexplored. Video reasoning grounds intelligence in spatiotemporally consistent visual environments that go beyond what text can naturally capture, enabling intuitive reasoning over spatiotemporal structure such as continuity, interaction, and causality. However, systematically studying video reasoning and its scaling behavior is hindered by the lack of large-scale training data. To address this gap, we introduce the Very Big Video Reasoning (VBVR) Dataset, an unprecedentedly large-scale resource spanning 200 curated reasoning tasks following a principled taxonomy and over one million video clips, approximately three orders of magnitude larger than existing datasets. We further present VBVR-Bench, a verifiable evaluation framework that moves beyond model-based judging by incorporating rule-based, human-aligned scorers, enabling reproducible and interpretable diagnosis of video reasoning capabilities. Leveraging the VBVR suite, we conduct one of the first large-scale scaling studies of video reasoning and observe early signs of emergent generalization to unseen reasoning tasks. Together, VBVR lays a foundation for the next stage of research in generalizable video reasoning. The data, benchmark toolkit, and models are publicly available at https://video-reason.com/ .
❌