❌

Normal view

Low-Cost Labels, Reliable Choices: Rollout-Calibrated Hyper-Heuristics for Job Shop Scheduling

arXiv:2605.23957v1 Announce Type: new Abstract: Learning-assisted hyper-heuristics can select among dispatching rules while preserving the feasibility and interpretability of constructive Job Shop Scheduling Problem (JSSP) heuristics. Their main computational cost lies in label generation rather than model fitting, since each supervised label usually requires rolling out candidate rules from a partial schedule. We study this label-cost problem together with a reliability problem: a learned selector should not switch away from a strong default rule unless the predicted gain is credible. The proposed selector uses regret-normalized rollout labels, a contextual KNN uncertainty estimate, and a gate that acts only when the predicted improvement exceeds an uncertainty-adjusted margin. We also vary rollout depth and breadth to measure the cost-quality trade-off. On synthetic JSSP instances, the gated selector achieves the lowest mean RPD among learned selectors, remains close to the best fixed dispatching rule, and reduces Random-HH mean RPD by more than an order of magnitude.

A Signal-Language Foundation Model for Broad-Spectrum Cardiovascular Assessment from Routine Electrocardiography

arXiv:2605.25446v1 Announce Type: new Abstract: Electrocardiography (ECG) is central to cardiovascular care, but conventional AI models are often restricted to common arrhythmias and may generalize poorly across populations or clinically subtle diseases. We developed ECG Contrastive Language-Image Pre-training (ECGCLIP), a signal-language contrastive learning framework that aligns ECG waveforms with expert diagnostic reports. ECGCLIP was pre-trained on 2,837,962 ECG studies from 1,324,856 patients and evaluated on a held-out internal test set plus nine independent external cohorts comprising about 1.5 million ECGs. Evaluation covered 89 downstream tasks, including 45 ECG diagnoses, 39 echocardiographic targets, and 5 rare cardiac diseases, using PRAUC as the primary metric. ECGCLIP consistently improved performance over random initialization and Merl-R18 baselines. On the internal test set, ECGCLIP-R34 achieved strong performance for atrial fibrillation (PRAUC 0.900) and ST-segment elevation myocardial infarction (PRAUC 0.383), with robust generalization across all external cohorts. It also improved low-prevalence and diagnostically elusive diseases, including Ebstein anomaly, constrictive pericarditis, dextrocardia, and cardiac amyloidosis, with internal PRAUC values of 0.253, 0.175, 0.121, and 0.201, respectively. ECGCLIP was data efficient, matching or exceeding full-dataset baseline performance with only 10% of training data. Feature visualization and saliency analysis suggested clinically meaningful representations aligned with established electrocardiographic criteria. These findings indicate that large-scale ECG-report contrastive pre-training can expand routine ECG interpretation beyond common arrhythmias toward broad cardiovascular assessment and opportunistic screening of echocardiographic and rare conditions.

CausaLab: A Scalable Environment for Interactive Causal Discovery Toward AI Scientists

arXiv:2605.26029v1 Announce Type: new Abstract: We introduce CausaLab, a scalable environment for evaluating interactive causal discovery by LLM agents. Unlike prior evaluations, CausaLab evaluates both whether an agent can solve a problem using causal evidence and whether its answer is supported by a correct hypothesis about the underlying causal mechanism. Each episode places an agent in a synthetic laboratory: it receives prior measurement records, intervenes on a manipulator crystal, and predicts the resonance frequency of a held-out reactor crystal governed by the same mechanism. The hidden data-generating process is a randomly sampled structural causal model (SCM), so success requires recovering both a causal graph and structural equations rather than recalling prior knowledge. CausaLab also includes a domain-specific language that records the agent's evolving SCM hypothesis, making trajectories inspectable and comparable with ground truth. Experiments show a persistent gap between prediction and mechanism recovery: in the purely observational 6-node setting, GPT-5.2-high reaches 92% task accuracy but only 0.471 all-edge $F_1$. This observation further motivates our exploration of different interaction strategies: Mixed observation--intervention strategies improve structural fidelity: in the mixed 6-node setting, GPT-5.2-high achieves 80% on both task accuracy and all-edge $F_1$. Yet even strong agents struggle to design informative interventions, as pure intervention strategies perform poorly on both task accuracy and all-edge $F_1$. We identify premature stopping as a major weakness of agents, and show that asking the model to verify the consistency between its hypothesis and past data can help mitigate this issue. CausaLab therefore separates predictive success from causal understanding and exposes current LLM agents' limits as experimental causal reasoners.

Towards a Universal Causal Reasoner

arXiv:2605.24873v1 Announce Type: cross Abstract: Despite the importance of causal reasoning, training LLMs to reason causally remains underexplored. Existing data efforts mostly focus on benchmarking LLMs on specific aspects of causality, making them less suitable for training generalizable causal reasoners. To address this, we propose UniCo, a data generation framework that both (1) addresses 18 causal query types across Pearl's Causal Ladder and (2) translates natively symbolic examples into code and natural language forms to simulate real-world use cases where causal terms are not explicitly specified. To ensure data quality, UniCo grounds answers with exact causal inference and filters cases with reasoning shortcuts. Upon supervised finetuning with 66.6K UniCo-generated instances, Qwen3-4B, Qwen3-8B and Olmo-3-7B-Instruct achieve an average of 22.9% improvements across all 18 in-distribution query types, and 8.1% over state-of-the-art causal data generation frameworks on 7 established causal benchmarks outside the training distribution. More importantly, in real-world medical understanding, legal decision, and tabular reasoning, UniCo-trained models consistently display more faithful reasoning traces, outperforming the base models by an average of 20.2% in faithfulness metrics. These suggest that causality-centered training not only strengthens causal reasoning, but also equips LLMs with a causal mindset in general reasoning tasks.

TGFormer: Towards Temporal Graph Transformer with Auto-Correlation Mechanism

arXiv:2605.24971v1 Announce Type: cross Abstract: The growing interest in Temporal Graph Neural Networks (TGNNs) stems from their ability to model complex dynamics and deliver superior performance. However, TGNNs encounter fundamental challenges in capturing long-term dependencies and identifying periodic patterns. To address these limitations, we propose TGFormer, a novel Transformer architecture specifically designed for temporal graphs. Our model redefines temporal graph learning by establishing a trajectory framework that aligns with time series analysis principles. This approach allows TGFormer to derive node representations through systematic analysis of historical interactions, enabling granular examination of node relationships across sequential timestamps. Building upon stochastic process theory, we develop an auto-correlation mechanism that systematically uncovers periodic dependencies in node interactions. This innovation empowers TGFormer to perform dependency discovery and representation aggregation at sub-interaction levels, demonstrating superior efficiency and accuracy compared to conventional attention mechanisms. Experimental validation across six public benchmarks confirms the effectiveness of our approach, with TGFormer at most achieving 9.35\% precision improvement compared to state-of-the-art approaches.

AgentArk: Distilling Multi-Agent Intelligence into a Single LLM Agent

arXiv:2602.03955v3 Announce Type: replace Abstract: While large language model (LLM) multi-agent systems achieve superior reasoning performance through iterative debate, practical deployment is limited by their high computational cost and error propagation. This paper proposes AgentArk, a novel framework to distill multi-agent dynamics into the weights of a single model, effectively transforming explicit test-time interactions into implicit model capabilities. This equips a single agent with the intelligence of multi-agent systems while remaining computationally efficient. Specifically, we investigate three hierarchical distillation strategies across various models, tasks, scaling, and scenarios: reasoning-enhanced fine-tuning; trajectory-based augmentation; and process-aware distillation. By shifting the burden of computation from inference to training, the distilled models preserve the efficiency of one agent while exhibiting strong reasoning and self-correction performance of multiple agents. They further demonstrate enhanced robustness and generalization across diverse reasoning tasks. We hope this work can shed light on future research on efficient and robust multi-agent development. Our code is at https://github.com/AIFrontierLab/AgentArk.

Safety in Embodied AI: A Survey of Risks, Attacks, and Defenses

arXiv:2605.02900v2 Announce Type: replace-cross Abstract: Embodied Artificial Intelligence (Embodied AI) integrates perception, cognition, planning, and interaction into agents that operate in open-world, safety-critical environments. As these systems gain autonomy and enter domains such as transportation, healthcare, and industrial or assistive robotics, ensuring their safety becomes both technically challenging and socially indispensable. Unlike digital AI systems, embodied agents must act under uncertain sensing, incomplete knowledge, and dynamic human-robot interactions, where failures can directly lead to physical harm. This survey provides a comprehensive and structured review of safety research in embodied AI, examining attacks and defenses across the full embodied pipeline, from perception and cognition to planning, action and interaction, and agentic system. We introduce a multi-level taxonomy that unifies fragmented lines of work and connects embodied-specific safety findings with broader advances in vision, language, and multimodal foundation models. Our review synthesizes insights from over 500 papers spanning adversarial, backdoor, jailbreak, and hardware-level attacks; attack detection, safe training and robust inference; and risk-aware human-agent interaction. This analysis reveals several overlooked challenges, including the fragility of multimodal perception fusion, the instability of planning under jailbreak attacks, and the trustworthiness of human-agent interaction in open-ended scenarios. By organizing the field into a coherent framework and identifying critical research gaps, this survey provides a roadmap for building embodied agents that are not only capable and autonomous but also safe, robust, and reliable in real-world deployment.

Integrative multi-omics and network perturbation analysis in human airway organoids reveals product-specific toxicity profiles of heated tobacco products

Ecotoxicol Environ Saf. 2026 May 25;319:120306. doi: 10.1016/j.ecoenv.2026.120306. Online ahead of print.

ABSTRACT

The respiratory toxicity of heated tobacco products (HTPs) remains incompletely characterized, and traditional models often fail to capture human-specific responses. Here, we established a human pluripotent stem cell (hPSC)-derived airway organoid (AO) platform and systematically compared the toxicological profiles of two HTP aerosols using an integrated framework encompassing conventional cytotoxicity assays, lineage-specific analysis, network perturbation modeling and multi-omics profiling. Both HTPs induced time- and concentration-dependent cytotoxicity, oxidative stress, DNA damage, and apoptosis in AOs. Exposure also triggered epithelial chemokine response characterized by elevated IL-8, MCP-1, MIP-1β, GM-CSF, and RANTES, with concomitant suppression of IP-10, indicating epithelial-derived inflammatory alarm signals. Lineage-specific transcriptional changes revealed mucociliary dysfunction characterized by goblet cell hyperplasia (MUC5AC upregulation) and ciliated cell impairment (FOXJ1 downregulation), key features of airway remodeling in chronic respiratory diseases. To delineate underlying mechanisms, we employed Network Perturbation Amplitude (NPA) analysis, which uncovered qualitatively distinct toxicity architectures: HTP-1 exhibited higher overall toxicity and elicited broad-spectrum network perturbations involving cell stress, proliferation, and immune regulation, correlating with greater apoptotic induction; HTP-2 triggered focused activation of damage-sensing pathways, consistent with its earlier membrane disruption and more pronounced genotoxicity. Multi-omics analysis further linked these mechanistic perturbations to human disease-relevant pathways, with HTP-1 showing stronger enrichment for COPD-associated expression patterns and HTP-2 for lung cancer-related signatures, suggesting the acute molecular response to each product exhibits similarity to specific pulmonary disease-associated molecular signatures. These findings establish human-derived airway organoids as a sensitive, human-relevant platform within the New Approach Methodologies‌ (NAMs) framework for qualitative comparison and mechanistic interrogation of product-specific toxicity.

PMID:42184653 | DOI:10.1016/j.ecoenv.2026.120306

Integrative multi-omics and network perturbation analysis in human airway organoids reveals product-specific toxicity profiles of heated tobacco products

Ecotoxicol Environ Saf. 2026 May 25;319:120306. doi: 10.1016/j.ecoenv.2026.120306. Online ahead of print.

ABSTRACT

The respiratory toxicity of heated tobacco products (HTPs) remains incompletely characterized, and traditional models often fail to capture human-specific responses. Here, we established a human pluripotent stem cell (hPSC)-derived airway organoid (AO) platform and systematically compared the toxicological profiles of two HTP aerosols using an integrated framework encompassing conventional cytotoxicity assays, lineage-specific analysis, network perturbation modeling and multi-omics profiling. Both HTPs induced time- and concentration-dependent cytotoxicity, oxidative stress, DNA damage, and apoptosis in AOs. Exposure also triggered epithelial chemokine response characterized by elevated IL-8, MCP-1, MIP-1β, GM-CSF, and RANTES, with concomitant suppression of IP-10, indicating epithelial-derived inflammatory alarm signals. Lineage-specific transcriptional changes revealed mucociliary dysfunction characterized by goblet cell hyperplasia (MUC5AC upregulation) and ciliated cell impairment (FOXJ1 downregulation), key features of airway remodeling in chronic respiratory diseases. To delineate underlying mechanisms, we employed Network Perturbation Amplitude (NPA) analysis, which uncovered qualitatively distinct toxicity architectures: HTP-1 exhibited higher overall toxicity and elicited broad-spectrum network perturbations involving cell stress, proliferation, and immune regulation, correlating with greater apoptotic induction; HTP-2 triggered focused activation of damage-sensing pathways, consistent with its earlier membrane disruption and more pronounced genotoxicity. Multi-omics analysis further linked these mechanistic perturbations to human disease-relevant pathways, with HTP-1 showing stronger enrichment for COPD-associated expression patterns and HTP-2 for lung cancer-related signatures, suggesting the acute molecular response to each product exhibits similarity to specific pulmonary disease-associated molecular signatures. These findings establish human-derived airway organoids as a sensitive, human-relevant platform within the New Approach Methodologies‌ (NAMs) framework for qualitative comparison and mechanistic interrogation of product-specific toxicity.

PMID:42184653 | DOI:10.1016/j.ecoenv.2026.120306

Cuproptosis causes meiotic metaphase I arrest by disrupting mitochondrial functions in oocytes

Cell Death Discovery, Published online: 23 May 2026; doi:10.1038/s41420-026-03168-x

Cuproptosis causes meiotic metaphase I arrest by disrupting mitochondrial functions in oocytes

Multi-omics integration identifies ribosome biogenesis-active macrophage subpopulation and its key gene GNL2 in driving liver hepatocellular carcinoma progression and mechanisms

Cancer Cell Int. 2026 May 14. doi: 10.1186/s12935-026-04330-2. Online ahead of print.

ABSTRACT

BACKGROUND: Liver hepatocellular carcinoma (LIHC) is a common malignancy, yet the core genes driving its progression and potential therapeutic targets remain insufficiently explored. Ribosome biogenesis (RB) is a critical biological process linked to various cancers; however, its systematic role in LIHC remains unclear.

METHODS: This study integrated LIHC single-cell RNA-Seq, bulk RNA-Seq, and spatial transcriptomic data with ribosome biogenesis-related gene sets to construct a single-cell atlas of LIHC. Weighted Gene Co-expression Network Analysis (WGCNA) was employed to characterize myeloid cell subsets. Furthermore, an LIHC prognostic risk model based on RB-related genes was developed using 117 machine-learning algorithm combinations. Key findings were subsequently corroborated through experimental validation and clinical sample analysis.

RESULTS: We identified a distinct macrophage subpopulation with high ribosome biogenesis activity, termed ribosome biogenesis-active macrophages (RAMs). These cells exhibited strong communication with inflammatory macrophages, potentially mediated by MIF-related receptor-ligand interactions. We further constructed an 8-gene prognostic model (PA2G4, GNL2, PWP1, DDX49, NOC4L, GDI2, CST7, and RCL1), which showed good predictive performance. Drug sensitivity analysis suggested that the high-risk group may be more responsive to several agents, including docetaxel. Among these genes, GNL2 was selected for further investigation. Elevated GNL2 expression was associated with increased stemness features in myeloid cells. Molecular docking analysis identified several candidate compounds with potential binding affinity to GNL2. Functionally, GNL2 knockdown in macrophages reduced TGF-β and TNF-α expression and was associated with decreased proliferation, migration, and invasion of LIHC cells.

CONCLUSION: We identified a highly active ribosome biogenesis-macrophage subpopulation (RAM), and constructed a robust risk model to aid in the diagnosis, prognosis, and treatment of LIHC. GNL2 is associated with increased expression of TGF-β and TNF-α and may contribute to LIHC progression.

PMID:42135716 | DOI:10.1186/s12935-026-04330-2

A Model Can Help Itself: Reward-Free Self-Training for LLM Reasoning

arXiv:2510.18814v2 Announce Type: replace-cross Abstract: Can language models improve their reasoning performance without external rewards, using only their own sampled responses for training? We show that they can. We propose Self-evolving Post-Training (SePT), a simple post-training method that alternates between self-generation and training on self-generated responses. It repeatedly samples questions, uses the model itself to generate low-temperature responses, and then finetunes the model on the self-generated data. In this self-training loop, we use an online data refresh mechanism, where each new batch is generated by the most recently updated model. Across six math reasoning benchmarks, SePT improves a strong no-training baseline, defined as the untuned base model evaluated at its best swept decoding temperature, on several tested models. In some settings, SePT can even approach the performance of Reinforcement Learning with Verifiable Rewards (RLVR). Additional ablations demonstrate the importance of online data refresh and temperature decoupling. Overall, our results identify a practical regime in which reasoning can be improved using self-generated supervision alone. Our code is available at https://github.com/ElementQi/SePT.

EvoSkills: Self-Evolving Agent Skills via Co-Evolutionary Verification

arXiv:2604.01687v1 Announce Type: new Abstract: Anthropic proposes the concept of skills for LLM agents to tackle multi-step professional tasks that simple tool invocations cannot address. A tool is a single, self-contained function, whereas a skill is a structured bundle of interdependent multi-file artifacts. Currently, skill generation is not only label-intensive due to manual authoring, but also may suffer from human--machine cognitive misalignment, which can lead to degraded agent performance, as evidenced by evaluations on SkillsBench. Therefore, we aim to enable agents to autonomously generate skills. However, existing self-evolving methods designed for tools cannot be directly applied to skills due to their increased complexity. To address these issues, we propose EvoSkills, a self-evolving skills framework that enables agents to autonomously construct complex, multi-file skill packages. Specifically, EvoSkills couples a Skill Generator that iteratively refines skills with a Surrogate Verifier that co-evolves to provide informative and actionable feedback without access to ground-truth test content. On SkillsBench, EvoSkills achieves the highest pass rate among five baselines on both Claude Code and Codex, and also exhibits strong generalization capabilities to six additional LLMs.

Moir\'e Video Authentication: A Physical Signature Against AI Video Generation

arXiv:2604.01654v1 Announce Type: cross Abstract: Recent advances in video generation have made AI-synthesized content increasingly difficult to distinguish from real footage. We propose a physics-based authentication signature that real cameras produce naturally, but that generative models cannot faithfully reproduce. Our approach exploits the Moir\'e effect: the interference fringes formed when a camera views a compact two-layer grating structure. We derive the Moir\'e motion invariant, showing that fringe phase and grating image displacement are linearly coupled by optical geometry, independent of viewing distance and grating structure. A verifier extracts both signals from video and tests their correlation. We validate the invariant on both real-captured and AI-generated videos from multiple state-of-the-art generators, and find that real and AI-generated videos produce significantly different correlation signatures, suggesting a robust means of differentiating them. Our work demonstrates that deterministic optical phenomena can serve as physically grounded, verifiable signatures against AI-generated video.

Editing strigolactone hormone receptor for robust antiviral silencing in rice

Precise genome editing of the rice strigolactone receptor DWARF14 confers robust, transgene-free antiviral resistance by blocking viral suppression of endogenous RNA silencing, offering a promising strategy for durable disease protection without a yield penalty.

Mean Masked Autoencoder with Flow-Mixing for Encrypted Traffic Classification

arXiv:2603.29537v1 Announce Type: cross Abstract: Network traffic classification using self-supervised pre-training models based on Masked Autoencoders (MAE) has demonstrated a huge potential. However, existing methods are confined to isolated byte-level reconstruction of individual flows, lacking adequate perception of the multi-granularity contextual relationship in traffic. To address this limitation, we propose Mean MAE (MMAE), a teacher-student MAE paradigm with flow mixing strategy for building encrypted traffic pre-training model. MMAE employs a self-distillation mechanism for teacher-student interaction, where the teacher provides unmasked flow-level semantic supervision to advance the student from local byte reconstruction to multi-granularity comprehension. To break the information bottleneck in individual flows, we introduce a dynamic Flow Mixing (FlowMix) strategy to replace traditional random masking mechanism. By constructing challenging cross-flow mixed samples with interferences, it compels the model to learn discriminative representations from distorted tokens. Furthermore, we design a Packet-importance aware Mask Predictor (PMP) equipped with an attention bias mechanism that leverages packet-level side-channel statistics to dynamically mask tokens with high semantic density. Numerous experiments on a number of datasets covering encrypted applications, malware, and attack traffic demonstrate that MMAE achieves state-of-the-art performance. The code is available at https://github.com/lx6c78/MMAE

Two-step clinical care pathway to predict MASLD-related advanced fibrosis and long-term outcomes in type 2 diabetes

Gut. 2026 Feb 9;75(3):576-587. doi: 10.1136/gutjnl-2025-337506.

ABSTRACT

BACKGROUND: Current guidelines recommend a two-step approach for risk stratification of metabolic dysfunction-associated steatotic liver disease (MASLD), starting with Fibrosis-4 index (FIB-4) followed by liver stiffness measurement (LSM) using vibration-controlled transient elastography (VCTE).

OBJECTIVE: To evaluate this approach for predicting advanced fibrosis and liver-related events (LREs) in patients with type 2 diabetes (T2D).

DESIGN: A prospective liver biopsy cohort of T2D patients with histologically confirmed MASLD from seven centres in China was used to assess diagnostic performance for advanced fibrosis. The international VCTE-Prognosis cohort, including T2D patients with MASLD who underwent VCTE at 16 centres in the USA, Europe and Asia, with longitudinal follow-up, was used to assess LREs, defined as hepatic decompensation or hepatocellular carcinoma.

RESULTS: 4781 participants were included. In the liver biopsy cohort (n=352; 22.2% with advanced fibrosis), applying LSM thresholds of <8 kPa and >12 kPa after FIB-4 classified patients into 63.4% low-risk, 9.4% intermediate-risk and 27.3% high-risk, with a correct classification rate of 71%. In the VCTE-Prognosis cohort (n=4429; median follow-up 51.3 (IQR 27.4-70.7) months), 140 (3.2%) patients developed LREs (110 (2.5%) with hepatic decompensation and 59 (1.3%) with hepatocellular carcinoma). The two-step approach classified 72.6%, 6.8% and 20.6% of patients into low-risk, intermediate-risk and high-risk groups, with corresponding 5-year cumulative LRE incidences of 0.7%, 0.9% and 11.8%. Refining classification of intermediate FIB-4 patients using LSM <10 kPa (low-risk) and >15 kPa (high-risk) reduced the intermediate-risk group to 5.6% while preserving predictive accuracy.

CONCLUSION: The non-invasive two-step approach of FIB-4 followed by LSM effectively stratifies MASLD-related advanced fibrosis and LREs risk in T2D. Applying LSM cut-offs of 10 and 15 kPa further optimises risk stratification for future LREs.

PMID:41911049 | DOI:10.1136/gutjnl-2025-337506

MCLR: Improving Conditional Modeling in Visual Generative Models via Inter-Class Likelihood-Ratio Maximization and Establishing the Equivalence between Classifier-Free Guidance and Alignment Objectives

arXiv:2603.22364v1 Announce Type: cross Abstract: Diffusion models have achieved state-of-the-art performance in generative modeling, but their success often relies heavily on classifier-free guidance (CFG), an inference-time heuristic that modifies the sampling trajectory. From a theoretical perspective, diffusion models trained with standard denoising score matching (DSM) are expected to recover the target data distribution, raising the question of why inference-time guidance is necessary in practice. In this work, we ask whether the DSM training objective can be modified in a principled manner such that standard reverse-time sampling, without inference-time guidance, yields effects comparable to CFG. We identify insufficient inter-class separation as a key limitation of standard diffusion models. To address this, we propose MCLR, a principled alignment objective that explicitly maximizes inter-class likelihood-ratios during training. Models fine-tuned with MCLR exhibit CFG-like improvements under standard sampling, achieving comparable qualitative and quantitative gains without requiring inference-time guidance. Beyond empirical benefits, we provide a theoretical result showing that the CFG-guided score is exactly the optimal solution to a weighted MCLR objective. This establishes a formal equivalence between classifier-free guidance and alignment-based objectives, offering a mechanistic interpretation of CFG.

Injecting Falsehoods: Adversarial Man-in-the-Middle Attacks Undermining Factual Recall in LLMs

arXiv:2511.05919v3 Announce Type: replace-cross Abstract: LLMs are now an integral part of information retrieval. As such, their role as question answering chatbots raises significant concerns due to their shown vulnerability to adversarial man-in-the-middle (MitM) attacks. Here, we propose the first principled attack evaluation on LLM factual memory under prompt injection via Xmera, our novel, theory-grounded MitM framework. By perturbing the input given to "victim" LLMs in three closed-book and fact-based QA settings, we undermine the correctness of the responses and assess the uncertainty of their generation process. Surprisingly, trivial instruction-based attacks report the highest success rate (up to ~85.3%) while simultaneously having a high uncertainty for incorrectly answered questions. To provide a simple defense mechanism against Xmera, we train Random Forest classifiers on the response uncertainty levels to distinguish between attacked and unattacked queries (average AUC of up to ~94.8%). We believe that signaling users to be cautious about the answers they receive from black-box and potentially corrupt LLMs is a first checkpoint toward user cyberspace safety.
❌