❌

Reading view

SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligence

arXiv:2512.22334v1 Announce Type: new Abstract: We introduce SciEvalKit, a unified benchmarking toolkit designed to evaluate AI models for science across a broad range of scientific disciplines and task capabilities. Unlike general-purpose evaluation platforms, SciEvalKit focuses on the core competencies of scientific intelligence, including Scientific Multimodal Perception, Scientific Multimodal Reasoning, Scientific Multimodal Understanding, Scientific Symbolic Reasoning, Scientific Code Generation, Science Hypothesis Generation and Scientific Knowledge Understanding. It supports six major scientific domains, spanning from physics and chemistry to astronomy and materials science. SciEvalKit builds a foundation of expert-grade scientific benchmarks, curated from real-world, domain-specific datasets, ensuring that tasks reflect authentic scientific challenges. The toolkit features a flexible, extensible evaluation pipeline that enables batch evaluation across models and datasets, supports custom model and dataset integration, and provides transparent, reproducible, and comparable results. By bridging capability-based evaluation and disciplinary diversity, SciEvalKit offers a standardized yet customizable infrastructure to benchmark the next generation of scientific foundation models and intelligent agents. The toolkit is open-sourced and actively maintained to foster community-driven development and progress in AI4Science.
  •  

Integrative spatial analysis reveals tumor heterogeneity and immune colony niche related to clinical outcomes in small cell lung cancer

Cancer Cell. 2025 Feb 14:S1535-6108(25)00030-3. doi: 10.1016/j.ccell.2025.01.012. Online ahead of print.

ABSTRACT

Recent advances have shed light on the molecular heterogeneity of small cell lung cancer (SCLC), yet the spatial organizations and cellular interactions in tumor immune microenvironment remain to be elucidated. Here, we employ co-detection by indexing (CODEX) and multi-omics profiling to delineate the spatial landscape for 165 SCLC patients, generating 267 high-dimensional images encompassing over 9.3 million cells. Integrating CODEX and genomic data reveals a multi-positive tumor cell neighborhood within ASCL1+ (SCLC-A) subtype, characterized by high SLFN11 expression and associated with poor prognosis. We further develop a cell colony detection algorithm (ColonyMap) and reveal a spatially assembled immune niche consisting of antitumoral macrophages, CD8+ T cells and natural killer T cells (MT2) which highly correlates with superior survival and predicts improving immunotherapy response in an independent cohort. This study serves as a valuable resource to study SCLC spatial heterogeneity and offers insights into potential patient stratification and personalized treatments.

PMID:39983726 | DOI:10.1016/j.ccell.2025.01.012

  •  
❌