arXiv:2503.00612v2 Announce Type: replace-cross Abstract: The basic problem of semantic compression is to minimize the length of a message while preserving its meaning. This differs from classical notions of compression in that the distortion is not measured directly at the level of bits, but rather in an abstract semantic space. In order to make this precise, we take inspiration from cognitive neuroscience and machine learning and model semantic space as a continuous Euclidean vector space. In such a space, stimuli like speech, images, or even ideas, are mapped to high-dimensional real vectors, and the location of these embeddings determines their meaning relative to other embeddings. This suggests that a natural metric for semantic similarity is just the Euclidean distance, which is what we use in this work. We map the optimization problem of determining the minimal-length, meaning-preserving message to a spin glass Hamiltonian and solve the resulting statistical mechanics problem using replica theory. We map out the replica symmetric phase diagram, identifying distinct phases of semantic compression: a first-order transition occurs between phases marked by the emergence of paraphrases, whereas a continuous crossover is seen from extractive to abstractive compression. We speculate on which features of the phase diagram are captured by replica symmetry, and which features change under replica symmetry breaking. We conclude by showing numerical simulations of compressions obtained by simulated annealing and greedy algorithms, and argue that while the problem of finding a meaning-preserving compression is computationally hard in the worst case, there exist efficient algorithms which achieve near optimal performance in the typical case.

arXiv:2605.25607v1 Announce Type: new Abstract: Understanding how network function constrains neural connectivity is a central challenge in neuroscience. An influential approach is to train neural networks with gradient descent on cognitive tasks and characterize the resulting connectivity. A key limitation is that the resulting structure depends on the details of the training procedure. Here we propose a complementary normative approach based on the maximum entropy principle for network connectivity, independent of any particular learning algorithm. We describe connectivity as a probability distribution over single-neuron weights, express task requirements as constraints on this distribution, and determine the unique distribution maximizing Shannon entropy subject to these constraints. A weight scale parameter controls the balance between randomness and task-induced structure. We apply this framework to context-dependent input-selection tasks in 2-layer feed-forward networks, and show that maximum entropy inference becomes analytically tractable by mapping nonlinear networks onto gain-modulated linear models. Starting from an a priori homogeneous distribution, we find that maximizing entropy under task constraints leads to the emergence of populations of neurons, each defined by its pattern of contextual gain modulation. Increasing the number of contexts drives a transition from context-specialized to unspecialized, random populations. Increasing the weight scale drives a parallel transition from structured to random stimulus selectivity. Strikingly, this maximum entropy connectivity matches both qualitatively and quantitatively the structure of networks trained with gradient descent across different learning regimes. Our results suggest that the interplay between task constraints and entropy maximization provides a fundamental principle for understanding the relationship between structure and function in neural networks.

arXiv:2605.25278v1 Announce Type: cross Abstract: Understanding the statistics of level crossings in stochastic processes is crucial across many scientific disciplines. The traditional Kac-Rice formula gives the mean rate of level crossings and has found broad use. However, that mean rate captures only a coarse summary of the crossing process. It depends entirely on local properties of the stochastic process at a given instant and is therefore blind to the correlation structure of the process over time. To understand whether crossing events, such as neuronal spikes, tend to cluster in time, spread apart, or exhibit more complex temporal organization, one must go beyond the mean rate and study higher-order crossing statistics. Here we go beyond the mean by deriving the exact analytical formulae for the variance and Fano factor of arbitrary level crossings in smooth stationary Gaussian processes. Our exact solution reveals how the full temporal correlation structure dictates whether crossings cluster or become regular. In systems with oscillatory correlations, such as a stochastic damped harmonic oscillator, a recent crossing suppresses an immediate subsequent one, producing sub-Poissonian statistics. However, as damping increases and oscillations disappear, a large and slow excursion above the threshold can produce multiple closely spaced crossings, yielding super-Poissonian statistics. In purely relaxational, non-oscillatory systems, such as a mean-reverting process driven by Ornstein-Uhlenbeck noise, the competition between the timescales of the driving noise and system relaxation produces a richer landscape, including reentrant transitions between sub- and super-Poissonian statistics as the threshold level is varied. Taken together, the exact variance and Fano factor derived here complement the Kac-Rice mean rate, enabling more robust parameter estimation and model selection across any setting where Gaussian processes are used.

arXiv:2605.26032v1 Announce Type: cross Abstract: Creating images from noise is image generation; reconstructing fine details from coarse inputs is super-resolution. Despite their practical differences, both can be understood as reversing information loss across scales. We introduce $\textbf{SKILD}$, a $\textbf{S}$cale-invariant $\textbf{K}$-Space $\textbf{I}$mage $\textbf{L}$earning $\textbf{D}$iffusion model that unifies generation and continuous super-resolution within a single unconditional framework. Both natural images and critical physical systems exhibit scale invariance, and we leverage it to design a forward process that attenuates image content from fine to coarse scales while injecting spectrum-matched Gaussian noise, making scale an explicit coordinate of the diffusion dynamics. The same trained reverse process performs generation and continuous super-resolution by varying only the starting timestep: $\textit{no task-specific architecture, no conditioning branch, no classifier-free guidance, no retraining per scale factor}$. Empirically, SKILD reaches FID $2.65$ and Inception Score $9.63$ on unconditional CIFAR-10, performs $2\times$--$8\times$ super-resolution on ImageNet from a single unconditional checkpoint while outperforming conditional models across perceptual metrics, and reconstructs critical Ising models whose connected four-point correlations closely track the ground truth.

arXiv:2604.04154v1 Announce Type: cross Abstract: Continual learning in artificial neural networks is fundamentally limited by the stability--plasticity dilemma: systems that retain prior knowledge tend to resist acquiring new knowledge, and vice versa. Existing approaches, most notably elastic weight consolidation~(EWC), address this empirically without a physical account of why plasticity eventually collapses as tasks accumulate. Separately, the distinction between sudden insight and gradual skill acquisition through repetitive practice has lacked a unified theoretical description. Here, we show that both problems admit a common resolution within non-equilibrium statistical physics. We model the state of a learning system as a particle evolving under Langevin dynamics on a double-well energy landscape, with the noise amplitude governed by a time-dependent effective temperature $T(t)$. The probability density obeys a Fokker--Planck equation, and transitions between metastable states are governed by the Kramers escape rate $k = (\omega_0\omega_b/2\pi)\,e^{-\Delta E/T}$. We make two contributions. First, we identify the EWC penalty term as an energy barrier whose height grows linearly with the number of accumulated tasks, yielding an exponential collapse of the transition rate predicted analytically and confirmed numerically. Second, we show that insight and repetitive learning correspond to two qualitatively distinct temperature protocols within the same Fokker--Planck equation: insight events produce transient spikes in $T(t)$ that drive rapid barrier crossing, whereas repetitive practice operates at a modestly elevated but fixed temperature, achieving transitions through sustained stochastic diffusion. These results establish a physically grounded framework for understanding plasticity and its failure in continual learning systems, and suggest principled design criteria for adaptive noise schedules in artificial intelligence.

arXiv:2603.22347v2 Announce Type: replace Abstract: Classical frameworks like Fisher Information approximate the cost of neural adaptation only in low-density regimes, failing to explain the explosive computational overhead incurred during deep structural reconfiguration. To address this, we introduce \textbf{Intelligence Inertia}, a property derived from the fundamental non-commutativity between rules and states ($[\hat{S}, \hat{R}] = i\mathcal{D}$). Rather than claiming a new fundamental physical law, we establish a \textbf{heuristic mathematical isomorphism} between deep learning dynamics and Minkowski spacetime. Acting as an \textit{effective theory} for high-dimensional tensor evolution, we derive a non-linear cost formula mirroring the Lorentz factor, predicting a relativistic $J$-shaped inflation curve -- a computational wall where classical approximations fail. We validate this framework via three experiments: (1) adjudicating the $J$-curve divergence under high-entropy noise, (2) mapping the optimal geodesic for architecture evolution, and (3) deploying an \textbf{inertia-aware scheduler wrapper} that prevents catastrophic forgetting. Adopting this isomorphism yields an exact quantitative metric for structural resistance, advancing the stability and efficiency of intelligent agents.

arXiv:2603.29932v1 Announce Type: cross Abstract: We explore the capability of a Large Language Model (LLM) to perform specific computations in mathematical physics: the task is to compute the coordinate Bethe Ansatz solution of selected integrable spin chain models. We select three integrable Hamiltonians for which the solutions were unpublished; two of the Hamiltonians are actually new. We observed that the LLM semi-autonomously solved the task in all cases, with a few mistakes along the way. These were corrected after the human researchers spotted them. The results of the LLM were checked against exact diagonalization (performed by separate programs), and the derivations were also checked by the authors. The Bethe Ansatz solutions are interesting in themselves. Our second model manifestly breaks left-right invariance, but it is PT-symmetric, therefore its solution could be interesting for applications in Generalized Hydrodynamics. And our third model is solved by a special form of the nested Bethe Ansatz, where the model is interacting, but the nesting level has a free fermionic structure lacking $U(1)$-invariance. This structure appears to be unique and it was found by the LLM. We used ChatGPT 5.2 Pro and 5.4 Pro by OpenAI.

arXiv:2603.22347v1 Announce Type: new Abstract: While Landauer's principle establishes the fundamental thermodynamic floor for information erasure and Fisher Information provides a metric for local curvature in parameter space, these classical frameworks function effectively only as approximations within regimes of sparse rule-constraints. They fail to explain the super-linear, and often explosive, computational and energy costs incurred when maintaining symbolic interpretability during the reconfiguration of advanced intelligent systems. This paper introduces the property of intelligence inertia and its underlying physical principles as foundational characteristics for quantifying the computational weight of intelligence. We demonstrate that this phenomenon is not merely an empirical observation but originates from the fundamental non-commutativity between rules and states, a root cause we have formally organized into a rigorous mathematical framework. By analyzing the growing discrepancy between actual adaptation costs and static information-theoretic estimates, we derive a non-linear cost formula that mirrors the Lorentz factor, characterizing a relativistic J-shaped inflation curve -- a "computational wall" that static models are blind to. The validity of these physical principles is examined through a trilogy of decisive experiments: (1) a comparative adjudication of this J-curve inflation against classical Fisher Information models, (2) a geometric analysis of the "Zig-Zag" trajectory of neural architecture evolution, and (3) the implementation of an inertia-aware scheduler wrapper that optimizes the training of deep networks by respecting the agent's physical resistance to change. Our results suggest a unified physical description for the cost of structural adaptation, offering a first-principle explanation for the computational and interpretability-maintenance overhead in intelligent agents.

arXiv:2602.19023v1 Announce Type: cross Abstract: We formulate a phenomenological effective field theory in which biological intelligence emerges as a macroscopic order parameter sustained by continuous metabolic flux. By modeling cognition as a coarse-grained neural activity field governed by a variational free energy, we derive closed-form expressions for information capacity and structural susceptibility using a Gaussian maximum entropy approximation. The theory predicts a universal algebraic divergence of the susceptibility, $\chi \sim K^{-3/2}$, as the structural stiffness $K$ approaches the instability threshold. The exponent $\gamma = 3/2$ is consistent with the mean-field branching process universality class, thereby providing a theoretical rationale for the observed avalanche size exponent $\tau \approx 3/2$ in cortical dynamics without invoking microscopic equivalence. We identify adult cognition as a metabolically pinned non-equilibrium steady state maintained near the critical regime $\Gamma \equiv K/\alpha \approx 1$ by continuous metabolic regulation, while pathological decline corresponds to a delocalization transition triggered by the violation of structural stability conditions. The framework generates concrete, falsifiable predictions for attention scaling, altered states of consciousness, and transcranial magnetic stimulation responses, each of which can be tested against existing neuroimaging and electrophysiological datasets.

arXiv:2602.14885v1 Announce Type: cross Abstract: Recurrent neural networks (RNNs) provide a theoretical framework for understanding computation in biological neural circuits, yet classical results, such as Hopfield's model of associative memory, rely on symmetric connectivity that restricts network dynamics to gradient-like flows. In contrast, biological networks support rich time-dependent behaviour facilitated by their asymmetry. Here we introduce a general framework, which we term drift-diffusion matching, for training continuous-time RNNs to represent arbitrary stochastic dynamical systems within a low-dimensional latent subspace. Allowing asymmetric connectivity, we show that RNNs can faithfully embed the drift and diffusion of a given stochastic differential equation, including nonlinear and nonequilibrium dynamics such as chaotic attractors. As an application, we construct RNN realisations of stochastic systems that transiently explore various attractors through both input-driven switching and autonomous transitions driven by nonequilibrium currents, which we interpret as models of associative and sequential (episodic) memory. To elucidate how these dynamics are encoded in the network, we introduce decompositions of the RNN based on its asymmetric connectivity and its time-irreversibility. Our results extend attractor neural network theory beyond equilibrium, showing that asymmetric neural populations can implement a broad class of dynamical computations within low-dimensional manifolds, unifying ideas from associative memory, nonequilibrium statistical mechanics, and neural computation.

arXiv:2508.04983v2 Announce Type: replace-cross Abstract: High-dimensional chaotic dynamics can emerge in a large random recurrent neural network when the synaptic gain crosses a threshold. Recent works showed that the kinetic energy of neural activity links the chaotic dynamics and the supporting unstable fixed points (equilibria) in the phase space. Here, we investigate the kinetic-energy-centric properties of random recurrent neural networks by combining dynamical mean-field theory with extensive numerical simulations. We find that the average kinetic energy shifts continuously from zero to a positive value at a critical value of coupling variance (synaptic gain) and exhibits a cubic scaling behavior near the critical point from above. This scaling behavior is supported by numerical simulations and provides a quantitative characterization of how fast the dynamics change during the onset of chaos. The steady-state activity distribution is further calculated by the theory and compared with simulations on finite-size systems from the kinetic-energy optimization perspective as well. The activity distribution is also analyzed in a geometric angle, establishing a relationship between the original chaotic dynamics and the gradient dynamics of the kinetic energy. The trajectory length on the chaotic manifold can be derived from the stationary kinetic energy, and the associated stationary behavior is analyzed as well. This study provides a kinetic-energy-centric route toward understanding the dynamics landscape of recurrent neural networks, which may provide insights for reservoir computing and even for internal synaptic learning.

arXiv:2510.24903v2 Announce Type: replace-cross Abstract: In this work, we examine the conditions for the emergence of chimera-like states in Ising systems. We study an Ising chain with periodic boundaries in contact with a thermal bath at temperature T, that induces stochastic changes in spin variables. To capture the non-locality needed for chimera formation, we introduce a model setup with non-local diffusion of spin values through the whole system. More precisely, diffusion is modeled through spin-exchange interactions between units up to a distance R, using Kawasaki dynamics. This setup mimics, e.g., neural media, as the brain, in the presence of electrical (diffusive) interactions. We explored the influence of such non-local dynamics on the emergence of complex spatiotemporal synchronization patterns of activity. Depending on system parameters we report here for the first time chimera-like states in the Ising model, characterized by relatively stable moving domains of spins with different local magnetization. We analyzed the system at T=0, both analytically and via simulations and computed the system's phase diagram, revealing rich behavior: regions with only chimeras, coexistence of chimeras and stable domains, and metastable chimeras that decay into uniform stable domains. This study offers fundamental insights into how coherent and incoherent synchronization patterns can arise in complex networked systems as it is, e.g., the brain.