❌

Normal view

Almost Free State Prediction Separation

arXiv:2609.03807v4 Announce Type: replace-cross Abstract: State--prediction separation (SPS) relieves a language model's hidden state of two competing burdens---summarizing the context and predicting the next token---by splitting the forward pass into a state stream and a prediction stream. The separation works, but it is expensive: the prediction stream is a second pass over the whole backbone, costing $\sim$1.9$\times$ the pretraining FLOPs, and even more in terms of wall-clock time when using a flexible attention mask. This paper makes state--prediction separation almost free. We take the separation to its limit with a free pause token: a prediction stream that writes no keys or values at all and so rides the sequence's existing positions. It improves next-token prediction of a standard Transformer by 2-3 centinats in practice on a 1B parameter model, and because it adds no position it costs nothing at inference---no added context length, no KV cache, no decode steps, and essentially no latency, with the growth in inference flops typically irrelevant as it is not the active bottleneck on throughput. The cost is therefore entirely in training where we use four mechanisms to drive it down: a two-pass split that keeps FlashAttention kernels viable, the $w{=}0$ prediction window, a shared gated FFN that evaluates one FFN per position rather than one per stream, and phasing the separation onto the tail of the run. Together these bring the overhead versus an optimized pretraining pipeline to $1.33\times$ wall-clock while recovering ~94% of the gain compared to SPS, and to as low as $1.09\times$ along a graceful quality/compute tradeoff. Furthermore, the FFN optimization reduces the raw flops required at inference time. The result is an isoflop, isoparameter, and isotoken improvement over standard next token trained transformers.

The Effectiveness of Digital Intervention on Psychological Resilience in Postoperative Breast Cancer Patients During Chemotherapy Intervals: Quasi-Experimental Study

Background: Patients with breast cancer during postoperative chemotherapy intervals commonly experience psychological distress and reduced resilience while recovering at home. Digital mindfulness interventions may provide accessible psychological support during this vulnerable period; however, evidence regarding tailored interventions for postoperative patients with breast cancer during chemotherapy intervals remains limited. Objective: This study aimed to examine the effectiveness of a digital intervention on psychological resilience in postoperative patients with breast cancer during chemotherapy intervals. Methods: A quasi-experimental study with repeated measures was conducted from October 2021 to June 2022. A total of 80 eligible participants were recruited from the Department of Breast Surgery at a tertiary hospital in Zhejiang Province, China, and 71 completed the study. The control group received routine discharge instructions and nursing follow-ups, whereas the intervention group additionally received an 8-week digital psychological resilience intervention. Outcomes were assessed at baseline (T0), 3 months post intervention (T1), and 6 months post intervention (T2). The measures included the Connor-Davidson Resilience Scale (CD-RISC), Hospital Anxiety and Depression Scale (HADS), Social Support Rating Scale (SSRS), Breast Cancer Survivor Self-Efficacy Scale (BCSSS), and Functional Assessment of Cancer Therapy-Breast (FACT-B). Independent-samples tests, chi-square tests, and repeated-measures ANOVA were performed using SPSS (version 26.0; IBM Corp). Results: No statistically significant baseline differences were observed between the two groups in the outcome measures. At T1, the intervention group had higher CD-RISC scores than the control group (mean 67.58, SD 11.41 vs mean 62.09, SD 10.18; =.036) and higher BCSSS scores (mean 42.36, SD 3.59 vs mean 39.23, SD 4.90; =.003). However, these between-group differences were no longer statistically significant at T2 (>.05). Significant time effects and group×time interaction effects were observed for both psychological resilience and self-efficacy (.05), although both scales showed significant time effects (

Learning Robust Visual Features in Computed Tomography Enables Efficient Transfer Learning for Clinical Tasks

arXiv:2604.04133v1 Announce Type: cross Abstract: There is substantial interest in developing artificial intelligence systems to support radiologists across tasks ranging from segmentation to report generation. Existing computed tomography (CT) foundation models have largely focused on building generalist vision-language systems capable of tasks such as question answering and report generation. However, training reliable vision-language systems requires paired image-text data at a scale that remains unavailable in CT. Moreover, adapting the underlying visual representations to downstream tasks typically requires partial or full backbone fine-tuning, a computationally demanding process inaccessible to many research groups. Instead, foundation models should prioritise learning robust visual representations that enable efficient transfer to new tasks with minimal labelled data and without backbone fine-tuning. We present VoxelFM, a 3D CT foundation model trained with self-distillation using the DINO framework, which learns semantically rich features without language supervision. We evaluated VoxelFM across seven categories of clinically relevant downstream tasks using frozen backbone representations with lightweight probes: classification, regression, survival analysis, instance retrieval, localisation, segmentation, and report generation. VoxelFM matched or outperformed four existing CT foundation models across all task categories. Despite receiving no language supervision during pre-training, VoxelFM surpassed models explicitly trained with language-alignment objectives, including on report generation. Our results indicate that current CT foundation models perform significantly better as feature extractors for lightweight probes rather than as vision encoders for vision-language models. Model weights and training code are publicly available.

Reshaping MOFs text mining with a dynamic multi-agents framework of large language model

arXiv:2504.18880v4 Announce Type: replace Abstract: Accurately identifying the synthesis conditions of metal-organic frameworks (MOFs) is essential for guiding experimental design, yet remains challenging because relevant information in the literature is often scattered, inconsistent, and difficult to interpret. We present MOFh6, a large language model driven system that reads raw articles or crystal codes and converts them into standardized synthesis tables. It links related descriptions across paragraphs, unifies ligand abbreviations with full names, and outputs structured parameters ready for use. MOFh6 achieved 99% extraction accuracy, resolved 94.1% of abbreviation cases across five major publishers, and maintained a precision of 0.93 +/- 0.01. Processing a full text takes 9.6 s, locating synthesis descriptions 36 s, with 100 papers processed for USD 4.24. By replacing static database lookups with real-time extraction, MOFh6 reshapes MOF synthesis research, accelerating the conversion of literature knowledge into practical synthesis protocols and enabling scalable, data-driven materials discovery.
❌