SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligence

By: Yiheng Wang · Yixin Chen · Shuo Li · Yifan Zhou · Bo Liu · Hengjian Gao · Jiakang Yuan · Jia Bu · Wanghan Xu · Yuhao Zhou · Xiangyu Zhao · Zhiwang Zhou · Fengxiang Wang · Haodong Duan · Songyang Zhang · Jun Yao · Han Deng · Yizhou Wang · Jiabei Xiao · Jiaqi Liu · Encheng Su · Yujie Liu · Weida Wang · Junchi Yao · Shenghe Zheng · Haoran Sun · Runmin Ma · Xiangchao Yan · Bo Zhang · Dongzhan Zhou · Shufei Zhang · Peng Ye · Xiaosong Wang · Shixiang Tang · Wenlong Zhang · Lei Bai

30 December 2025 at 13:00

arXiv:2512.22334v1 Announce Type: new Abstract: We introduce SciEvalKit, a unified benchmarking toolkit designed to evaluate AI models for science across a broad range of scientific disciplines and task capabilities. Unlike general-purpose evaluation platforms, SciEvalKit focuses on the core competencies of scientific intelligence, including Scientific Multimodal Perception, Scientific Multimodal Reasoning, Scientific Multimodal Understanding, Scientific Symbolic Reasoning, Scientific Code Generation, Science Hypothesis Generation and Scientific Knowledge Understanding. It supports six major scientific domains, spanning from physics and chemistry to astronomy and materials science. SciEvalKit builds a foundation of expert-grade scientific benchmarks, curated from real-world, domain-specific datasets, ensuring that tasks reflect authentic scientific challenges. The toolkit features a flexible, extensible evaluation pipeline that enables batch evaluation across models and datasets, supports custom model and dataset integration, and provides transparent, reproducible, and comparable results. By bridging capability-based evaluation and disciplinary diversity, SciEvalKit offers a standardized yet customizable infrastructure to benchmark the next generation of scientific foundation models and intelligent agents. The toolkit is open-sourced and actively maintained to foster community-driven development and progress in AI4Science.

Systems pharmacology approaches decipher the anti-cancer efficacy of ethnopharmacological agents in hepatocellular carcinoma

Omics in Hepatocellular

By: Tingting Hao · Xiaonan Yang · Weibin Wang · Dawei Li · Yanqi Liu · Die Jiang · Ya Ning · Qinghua Kong · Hongmei Li · Ru Zhang · Zhu Qiao · Xing Jiang · Xiaolei Zhou · Dandan Mo · Wenqing Su · Wendong He · Xuzhen Li · Baozheng Chen · Qiang-Qiang Zhu · Xianghua Xia · Yang Dong

17 December 2025 at 19:00

Sci Rep. 2025 Dec 17;15(1):43996. doi: 10.1038/s41598-025-27744-w.

ABSTRACT

Hepatocellular carcinoma (HCC) poses a significant global health burden with limited therapeutic efficacy. Chinese herbal medicines (CHMs) offer multi-target potential, yet their systematic screening and mechanistic elucidation remain challenging. We established a high-throughput multi-omics platform integrating transcriptomics, proteomics, and deep learning (autoencoder and multiple kernel learning) to screen 187 medicinal plants. Five CHMs candidates were identified and shown to modulate hub genes (e.g., AKR1B10, HMGCR, THBS1) and key pathways (TNF/IL-17/MAPK, apoptosis, ferroptosis). Proteomic validation and functional assays confirmed their roles in suppressing proliferation, migration, and inducing apoptosis in HCC cells. This study provides a robust, data-driven pipeline for natural anti-HCC drug discovery, linking specific hub genes to CHM efficacy and offering novel insights into precision ethnopharmacology.

PMID:41408124 | DOI:10.1038/s41598-025-27744-w

ClinicalTrialsHub: Bridging Registries and Literature for Comprehensive Clinical Trial Access

cs.AI, q-bio.NC updates on arXiv.org

By: Jiwoo Park · Ruoqi Liu · Avani Jagdale · Andrew Srisuwananukorn · Jing Zhao · Lang Li · Ping Zhang · Sachin Kumar

10 December 2025 at 13:00

arXiv:2512.08193v1 Announce Type: cross Abstract: We present ClinicalTrialsHub, an interactive search-focused platform that consolidates all data from ClinicalTrials.gov and augments it by automatically extracting and structuring trial-relevant information from PubMed research articles. Our system effectively increases access to structured clinical trial data by 83.8% compared to relying on ClinicalTrials.gov alone, with potential to make access easier for patients, clinicians, researchers, and policymakers, advancing evidence-based medicine. ClinicalTrialsHub uses large language models such as GPT-5.1 and Gemini-3-Pro to enhance accessibility. The platform automatically parses full-text research articles to extract structured trial information, translates user queries into structured database searches, and provides an attributed question-answering system that generates evidence-grounded answers linked to specific source sentences. We demonstrate its utility through a user study involving clinicians, clinical researchers, and PhD students of pharmaceutical sciences and nursing, and a systematic automatic evaluation of its information extraction and question answering capabilities.

Genomic Next-Token Predictors are In-Context Learners

cs.AI, q-bio.NC updates on arXiv.org

By: Nathan Breslow · Aayush Mishra · Mahler Revsine · Michael C. Schatz · Anqi Liu · Daniel Khashabi

24 November 2025 at 13:00

arXiv:2511.12797v2 Announce Type: replace-cross Abstract: In-context learning (ICL) -- the capacity of a model to infer and apply abstract patterns from examples provided within its input -- has been extensively studied in large language models trained for next-token prediction on human text. In fact, prior work often attributes this emergent behavior to distinctive statistical properties in human language. This raises a fundamental question: can ICL arise organically in other sequence domains purely through large-scale predictive training? To explore this, we turn to genomic sequences, an alternative symbolic domain rich in statistical structure. Specifically, we study the Evo2 genomic model, trained predominantly on next-nucleotide (A/T/C/G) prediction, at a scale comparable to mid-sized LLMs. We develop a controlled experimental framework comprising symbolic reasoning tasks instantiated in both linguistic and genomic forms, enabling direct comparison of ICL across genomic and linguistic models. Our results show that genomic models, like their linguistic counterparts, exhibit log-linear gains in pattern induction as the number of in-context demonstrations increases. To the best of our knowledge, this is the first evidence of organically emergent ICL in genomic sequences, supporting the hypothesis that ICL arises as a consequence of large-scale predictive modeling over rich data. These findings extend emergent meta-learning beyond language, pointing toward a unified, modality-agnostic view of in-context learning.

Stereo-seq V2: Spatial mapping of total RNA on FFPE sections with high resolution

Cell

By: Yu Zhao · Young Li · Ying He · Junqi Wu · Yi Liu · Xinxing Li · Zhaoxun Li · Qiaomei Yuan · Jialuo Li · Xinya Zhang · Xiaole Hu · Guifang Wu · Ying Qiu · Jie Yuan · Xin Huang · Mengru Xu · Mei Yang · Jing Guo · Tingting Cao · Qiuling Chen · Dandan Li · Jiajun Zhang · Xiaoxi Zhou · Xia Jiang · Fan Zhu · Xuan Dong · Rong Xiang · Hailin Pan · Lei Han · Ziqing Deng · Haohao Deng · Yan Zhang · Maoyuan Liu · Qin Wu · Gaoyang Wang · Jie Zhai · Weihong Tan · Xiawei Liu · Zehao Wang · Shanshan Li · Tianbi Duan · Longqi Liu · Ao Chen · Haiying Liu · Chang Chen · Sha Liao · Xun Xu

28 August 2025 at 08:00

Stereo-seq V2 facilitates single-cell-resolution spatial RNA mapping in FFPE samples through random primer capture, uncovering ncRNAs, host-pathogen transcriptome profiling, and spatial immune repertoires in situ.

PaperArena: An Evaluation Benchmark for Tool-Augmented Agentic Reasoning on Scientific Literature

cs.AI, q-bio.NC updates on arXiv.org

By: Daoyu Wang · Mingyue Cheng · Qi Liu · Shuo Yu · Zirui Liu · Ze Guo

28 October 2025 at 12:00

arXiv:2510.10909v2 Announce Type: replace Abstract: Understanding and reasoning on the web-scale scientific literature is a crucial touchstone for large language model (LLM) based agents designed to support complex knowledge-intensive tasks. However, existing works are mainly restricted to tool-free tasks within isolated papers, largely due to the lack of a benchmark for cross-paper reasoning and multi-tool orchestration in real research scenarios. In this work, we propose PaperArena, an evaluation benchmark for agents to address real-world research questions that typically require integrating information across multiple papers with the assistance of external tools. Given a research question, agents should integrate diverse formats across multiple papers through reasoning and interacting with appropriate tools, thereby producing a well-grounded answer. To support standardized evaluation, we provide a modular and extensible platform for agent execution, offering tools such as multimodal parsing, context retrieval, and programmatic computation. Experimental results reveal that even the most advanced LLM powering a well-established agent system achieves merely 38.78% average accuracy. On the hard subset, accuracy drops to only 18.47%, highlighting great potential for improvement. We also present several empirical findings, including that all agents tested exhibit inefficient tool usage, often invoking more tools than necessary to solve a task. We invite the community to adopt PaperArena to develop and evaluate more capable agents for scientific discovery. Our code and data are available https://github.com/Melmaphother/PaperArena.

Annotation Guidelines-Based Knowledge Augmentation: Towards Enhancing Large Language Models for Educational Text Classification

cs.AI, q-bio.NC updates on arXiv.org

By: Shiqi Liu · Sannyuya Liu · Lele Sha · Zijie Zeng · Dragan Gasevic · Zhi Liu

24 October 2025 at 12:00

arXiv:2406.00954v2 Announce Type: replace-cross Abstract: Various machine learning approaches have gained significant popularity for the automated classification of educational text to identify indicators of learning engagement -- i.e. learning engagement classification (LEC). LEC can offer comprehensive insights into human learning processes, attracting significant interest from diverse research communities, including Natural Language Processing (NLP), Learning Analytics, and Educational Data Mining. Recently, Large Language Models (LLMs), such as ChatGPT, have demonstrated remarkable performance in various NLP tasks. However, their comprehensive evaluation and improvement approaches in LEC tasks have not been thoroughly investigated. In this study, we propose the Annotation Guidelines-based Knowledge Augmentation (AGKA) approach to improve LLMs. AGKA employs GPT 4.0 to retrieve label definition knowledge from annotation guidelines, and then applies the random under-sampler to select a few typical examples. Subsequently, we conduct a systematic evaluation benchmark of LEC, which includes six LEC datasets covering behavior classification (question and urgency level), emotion classification (binary and epistemic emotion), and cognition classification (opinion and cognitive presence). The study results demonstrate that AGKA can enhance non-fine-tuned LLMs, particularly GPT 4.0 and Llama 3 70B. GPT 4.0 with AGKA few-shot outperforms full-shot fine-tuned models such as BERT and RoBERTa on simple binary classification datasets. However, GPT 4.0 lags in multi-class tasks that require a deep understanding of complex semantic information. Notably, Llama 3 70B with AGKA is a promising combination based on open-source LLM, because its performance is on par with closed-source GPT 4.0 with AGKA. In addition, LLMs struggle to distinguish between labels with similar names in multi-class classification.

Liquid Biopsy in CRC Management: Early Detection, Minimal Residual Disease, and Therapy Optimization-Clinical Evidence and Challenges

MRD

By: Qi Liu · Xiaoyong Li · Tiejun Jin · Sibo Huo · Shuai Su · Nan Liu

4 September 2025 at 18:00

Diagn Cytopathol. 2025 Nov;53(11):580-591. doi: 10.1002/dc.70009. Epub 2025 Sep 4.

ABSTRACT

Colorectal cancer (CRC) is a major global health burden, ranking among the leading causes of cancer-related deaths. Despite improvements in screening and treatment, challenges such as late-stage diagnosis, high recurrence rates, and therapy resistance continue to impede optimal outcomes. Liquid biopsy, a minimally invasive technique that analyzes tumor-derived components in bodily fluids-including circulating tumor DNA (ctDNA), circulating tumor cells (CTCs), and extracellular vesicles (EVs)-is emerging as a powerful tool to transform CRC management across the disease continuum. This review provides a comprehensive overview of liquid biopsy's current and emerging applications in CRC. We examine its role in early detection, where sensitive ctDNA-based assays and epigenetic biomarkers have demonstrated the ability to identify CRC at asymptomatic or early stages, potentially improving screening uptake and compliance. Furthermore, we explore how liquid biopsy enables dynamic monitoring of treatment response and clonal evolution, facilitating the timely identification of resistance mutations and supporting personalized therapy adjustments. Innovations in multi-omics integration, artificial intelligence, and ultra-sensitive sequencing technologies are also discussed as pivotal advancements that enhance the clinical utility of liquid biopsy. Despite significant progress, the widespread adoption of liquid biopsy faces several hurdles, including assay standardization, sensitivity for low-shedding tumors, regulatory approval, and cost-effectiveness. Continued research, validation in large prospective trials, and harmonization of testing protocols are essential to overcome these challenges. Ultimately, liquid biopsy holds the potential to become a cornerstone of precision oncology in CRC, enabling earlier intervention, more tailored treatment strategies, and improved patient outcomes.

PMID:40905096 | DOI:10.1002/dc.70009

Stereo-seq V2: Spatial mapping of total RNA on FFPE sections with high resolution

(Multiomics OR Omics) AND (Lung OR gastric OR Hepatocellular)

By: Yu Zhao · Young Li · Ying He · Junqi Wu · Yi Liu · Xinxing Li · Zhaoxun Li · Qiaomei Yuan · Jialuo Li · Xinya Zhang · Xiaole Hu · Guifang Wu · Ying Qiu · Jie Yuan · Xin Huang · Mengru Xu · Mei Yang · Jing Guo · Tingting Cao · Qiuling Chen · Dandan Li · Jiajun Zhang · Xiaoxi Zhou · Xia Jiang · Fan Zhu · Xuan Dong · Rong Xiang · Hailin Pan · Lei Han · Ziqing Deng · Haohao Deng · Yan Zhang · Maoyuan Liu · Qin Wu · Gaoyang Wang · Jie Zhai · Weihong Tan · Xiawei Liu · Zehao Wang · Shanshan Li · Tianbi Duan · Longqi Liu · Ao Chen · Haiying Liu · Chang Chen · Sha Liao · Xun Xu

29 August 2025 at 18:00

Cell. 2025 Aug 22:S0092-8674(25)00922-5. doi: 10.1016/j.cell.2025.08.008. Online ahead of print.

ABSTRACT

Performing total RNA profiling on formalin-fixed, paraffin-embedded (FFPE) samples, the predominant sample conservation method in clinical practice, remains challenging for current spatial transcriptomics techniques. Here, we introduce Stereo-seq V2, which employs random primers to capture and sequence RNAs in situ on FFPE sections and provides single-cell resolution. The random-priming-based strategy offers unbiased transcript capturing and uniform gene body coverage, which increase the sensitivity to marker genes, the efficiency of non-polyadenylation (poly(A)) RNA profiling, and immune repertoire coverage. We demonstrated the robust performance of Stereo-seq V2 on clinical FFPE samples using triple-negative breast cancer (TNBC) sections and identified tumor-specific alternative splicing events. In a Mycobacterium tuberculosis (Mtb)-infected mouse model, we monitored gene expression dynamics of host and pathogen transcriptomes simultaneously by utilizing Stereo-seq V2. We also assembled immune repertoires and identified Mtb-specific BCR clones, which could also be observed in human tuberculous lung samples. These results highlight Stereo-seq V2's potential in biomedical research and personalized medicine.

PMID:40882628 | DOI:10.1016/j.cell.2025.08.008