❌

Normal view

SciEvalKit: An Open-source Evaluation Toolkit for Scientific General Intelligence

arXiv:2512.22334v1 Announce Type: new Abstract: We introduce SciEvalKit, a unified benchmarking toolkit designed to evaluate AI models for science across a broad range of scientific disciplines and task capabilities. Unlike general-purpose evaluation platforms, SciEvalKit focuses on the core competencies of scientific intelligence, including Scientific Multimodal Perception, Scientific Multimodal Reasoning, Scientific Multimodal Understanding, Scientific Symbolic Reasoning, Scientific Code Generation, Science Hypothesis Generation and Scientific Knowledge Understanding. It supports six major scientific domains, spanning from physics and chemistry to astronomy and materials science. SciEvalKit builds a foundation of expert-grade scientific benchmarks, curated from real-world, domain-specific datasets, ensuring that tasks reflect authentic scientific challenges. The toolkit features a flexible, extensible evaluation pipeline that enables batch evaluation across models and datasets, supports custom model and dataset integration, and provides transparent, reproducible, and comparable results. By bridging capability-based evaluation and disciplinary diversity, SciEvalKit offers a standardized yet customizable infrastructure to benchmark the next generation of scientific foundation models and intelligent agents. The toolkit is open-sourced and actively maintained to foster community-driven development and progress in AI4Science.

Gut microbial metabolites in cancer immunomodulation

Mol Cancer. 2025 Dec 3. doi: 10.1186/s12943-025-02521-5. Online ahead of print.

ABSTRACT

Gut microbiota-derived metabolites are emerging as systemic "remote immunoregulators" that shape tumor immunity across tissues. Integrating evidence across short-chain fatty acids, tryptophan derivatives, secondary bile acids, polyamines and other metabolites, we advance a metabolite-immune pathway-cancer framework that links receptor-mediated signaling, epigenetic remodeling and metabolic reprogramming to context-dependent, bidirectional immune effects. Importantly, in addition to the g protein-coupled receptor / aryl hydrocarbon receptor pathway, the selected microbial small molecule metabolites are the true T-cell receptor ligands of unconventional T cells, directly shaping the tissue resident immune and tumor microenvironment, supplementing the receptor signaling and epigenetic programs in our framework. We synthesize how these metabolites recalibrate the tumor immune microenvironment-modulating antigen presentation, T-cell effector fitness and exhaustion, regulatory T-cell activity, and myeloid polarization-and why the same metabolite can either potentiate immune surveillance or entrench immunosuppression depending on ligand-receptor pairing, dose and tissue niche. We compare tumor-type specific patterns (e.g., colorectal, liver, lung, breast and prostate cancers) to highlight common circuits and organ-restricted idiosyncrasies. Methodologically, we outline how single-cell and spatial multi-omics, imaging mass spectrometry and functional biosensors now enable co-registration of metabolite exposure with immune-cell states in human tumors, providing an actionable basis for biomarker discovery. Given ongoing debate about signals attributed to intratumoral microbiota in low-biomass tumor tissues, we foreground quantifiable, spatially mappable and pharmacologically tractable metabolite-receptor pathways, using microbe-associated molecular patterns / translocation as comparators to judge when chemical signals should be prioritized as intervention targets. Finally, we evaluate precision intervention avenues-including fecal microbiota transplantation, rational bacterial consortia, engineered microbes and nanoparticle-enabled metabolite delivery-and propose stratification rules that pair metabolite/receptor signatures with fit-for-purpose delivery. Together, mapping tissue-specific metabolite-immune circuits and embedding them in robust biomarker frameworks may convert microbial metabolites from correlative markers into therapeutic targets and tools, improving the efficacy and durability of cancer immunotherapy.

PMID:41339918 | DOI:10.1186/s12943-025-02521-5

Generative AI for Healthcare: Fundamentals, Challenges, and Perspectives

arXiv:2510.24551v1 Announce Type: new Abstract: Generative Artificial Intelligence (GenAI) is taking the world by storm. It promises transformative opportunities for advancing and disrupting existing practices, including healthcare. From large language models (LLMs) for clinical note synthesis and conversational assistance to multimodal systems that integrate medical imaging, electronic health records, and genomic data for decision support, GenAI is transforming the practice of medicine and the delivery of healthcare, such as diagnosis and personalized treatments, with great potential in reducing the cognitive burden on clinicians, thereby improving overall healthcare delivery. However, GenAI deployment in healthcare requires an in-depth understanding of healthcare tasks and what can and cannot be achieved. In this paper, we propose a data-centric paradigm in the design and deployment of GenAI systems for healthcare. Specifically, we reposition the data life cycle by making the medical data ecosystem as the foundational substrate for generative healthcare systems. This ecosystem is designed to sustainably support the integration, representation, and retrieval of diverse medical data and knowledge. With effective and efficient data processing pipelines, such as semantic vector search and contextual querying, it enables GenAI-powered operations for upstream model components and downstream clinical applications. Ultimately, it not only supplies foundation models with high-quality, multimodal data for large-scale pretraining and domain-specific fine-tuning, but also serves as a knowledge retrieval backend to support task-specific inference via the agentic layer. The ecosystem enables the deployment of GenAI for high-quality and effective healthcare delivery.
❌