Universal equation of state for emergent abilities of large language models

Abstract

Universal Equation of State for Emergent Abilities of Large Language Models Takayuki Takagi — Independent Researcher, Higashimatsuyama, Saitama, JapanSubmitted to Physical Review Letters (APS Manuscript ID: es2026feb07_736, February 7, 2026) What This Paper Does Large language models (LLMs) exhibit sudden jumps in capability as they scale — a phenomenon called emergent abilities. Whether these jumps are real phase transitions or artifacts of how we measure performance has been debated since 2022: Wei et al. (TMLR 2022): Emergence is real — abilities appear discontinuously. Schaeffer et al. (NeurIPS 2023, Outstanding Paper): Emergence is a mirage — switch the metric, and the jump disappears. This paper resolves the debate by deriving a universal equation of state (EOS): $$D_{\mathrm{ext}} = \frac{k}{k + \kappa}$$ where k = log₁₀(N/N₀) measures model capacity and κ encodes the metric's structural constraint. Key Results Universal fit: The same two-parameter function describes all 11 BIG-Bench emergent tasks (mean R² = 0.87). κ spans two orders of magnitude (0.02–1.1), explaining why some tasks appear sharp and others smooth. κ correlates with metric structure (Spearman ρ = 0.65 with chance baseline) but not with reasoning difficulty (r = −0.14). External validation: Data from Schaeffer et al. confirm that accuracy (κ = 0.046) and token edit distance (κ = 0.89) for the same model outputs follow the same EOS. Resolution of the Debate Both sides were right — partially: Claim Status Emergence has the structure of a phase transition ✅ The EOS D = k/(k+κ) is universal Apparent sharpness depends on the metric ✅ Sharpness is encoded in κ The EOS provides a common coordinate system that subsumes both positions. Theoretical Foundation The EOS is structurally identical to the Langmuir adsorption isotherm and to the order parameter D_ext = k/(k+κ) derived from first principles in constrained information systems: T. Takagi, "Discontinuous phase transition in assertion dynamics of a constrained information system," submitted to Phys. Rev. E (APS ID: es2026feb07_585).Preprint: doi:10.21203/rs.3.rs-8738395/v1 Data Availability Complete Python analysis code and fitted data for all 11 tasks:doi:10.5281/zenodo.18413041 Files File Description Takagi_2026_Universal_EOS_LLM_Emergent_Abilities.pdf Main manuscript (PRL format, 4 pages + references) Supplemental_Material.pdf Fitting details for all 11 BIG-Bench tasks + Schaeffer validation License: CC BY 4.0ubmitted to Physical Review Letters (APS Manuscript ID: es2026feb07_736). This Letter applies the EOS D=k/(k+κ) derived in the companion paper (submitted to Phys. Rev. E, APS ID: es2026feb07_585) to LLM scaling data.

Ähnliche Arbeiten