❌

Normal view

SeqMoE: Toward Full-Load Performance via Predictive and Graph-Compatible MoE Offloading

14 September 2026 at 12:00
arXiv:2609.12978v1 Announce Type: cross Abstract: Mixture-of-Experts (MoE) creates a structural advantage for offloading: only a small fraction of activated experts need to reside in device memory, and if they can be loaded in time for computation, offloading can in principle approach full-load performance, where all model weights reside in device memory. Yet translating MoE's structural advantage into practical offloading gains remains challenging. We propose SeqMoE to bridge this gap. To maximize expert hits, we build predictive memory management: (i) Sequence-to-sequence prediction. We are the first to recast expert activation prediction as sequence modeling, enabling accurate multi-step, multi-layer forecasts that provide a long and reliable window for downstream decisions. (ii) Joint prefetch scheduling. We formulate prefetch scheduling as Job Sequencing with Deadlines to maximize expected expert hits and improve bandwidth efficiency. (iii) Forecast-driven caching. Leveraging the recursive nature of sequence modeling, we introduce a probabilistic Belady policy for future-aware eviction. To eliminate execution bottleneck, we develop (iv) Graph-compatible offloading runtime. We derive general runtime principles encompassing compute-transparent expert placement and synchronization-free orchestration disciplines for end-to-end graph capture. With 45% expert residency, SeqMoE averages a 96.97% hit rate and 80.22% of full-load performance, advancing the state of the art in MoE offloading.

Performance of Conventional EEG Biomarkers Across Different Clinical Phases of Major Depressive Disorder: A Comprehensive Evaluation

arXiv:2603.03864v1 Announce Type: new Abstract: While EEG features differentiate Major Depressive Disorder (MDD) from healthy controls (HC), their clinical utility as biomarkers depends on a monotonic trajectory across the disease spectrum, from the acute (AC) phase to the maintenance (MA) phase and finally to the healthy baseline. However, the progression of the MA phase remains poorly understood in traditional marker analysis. Analyzing EEG data from 74 individuals (24 AC, 23 MA, and 27 HC), this study provides a comprehensive evaluation of classic ERP and resting-state indices across AC, MA, and HC groups. Our results demonstrate that almost no conventional metrics strictly satisfy the criterion of monotonic progression, likely due to profound inter-individual heterogeneity. These findings highlight the inherent limitations of group-level feature extraction and provide critical insights for developing future paradigms and algorithms to identify neurobiological markers with genuine clinical utility.

TrustMH-Bench: A Comprehensive Benchmark for Evaluating the Trustworthiness of Large Language Models in Mental Health

arXiv:2603.03047v1 Announce Type: cross Abstract: While Large Language Models (LLMs) demonstrate significant potential in providing accessible mental health support, their practical deployment raises critical trustworthiness concerns due to the domains high-stakes and safety-sensitive nature. Existing evaluation paradigms for general-purpose LLMs fail to capture mental health-specific requirements, highlighting an urgent need to prioritize and enhance their trustworthiness. To address this, we propose TrustMH-Bench, a holistic framework designed to systematically quantify the trustworthiness of mental health LLMs. By establishing a deep mapping from domain-specific norms to quantitative evaluation metrics, TrustMH-Bench evaluates models across eight core pillars: Reliability, Crisis Identification and Escalation, Safety, Fairness, Privacy, Robustness, Anti-sycophancy, and Ethics. We conduct extensive experiments across six general-purpose LLMs and six specialized mental health models. Experimental results indicate that the evaluated models underperform across various trustworthiness dimensions in mental health scenarios, revealing significant deficiencies. Notably, even generally powerful models (e.g., GPT-5.1) fail to maintain consistently high performance across all dimensions. Consequently, systematically improving the trustworthiness of LLMs has become a critical task. Our data and code are released.
❌