LLM: the 4 revolutions of AI

Christophe Cerisara

https://ia.loria.fr/talk

• LLMs are at the core of the AI revolution, because they enable
  • exploiting all human knowledge encoded in languages
  • reasoning mediated by language
  • interacting “naturally” with humans
• 4 main revolutions:
  • 2017: the transformer
  • 2020: pretraining scaling laws
  • 2024: test-time scaling
  • 2025: agentic

Revolution 1: the transformer (2017)

• Google’s paper: “Attention is all you need”
• Why a revolution? Because it removes important bottlenecks:
  • bottleneck of information in LSTM (h vector encodes all history) and CNN (weighted averaging of embeddings)
  • positional bias in LSTM (mitigated by “content-based similarity + positional encodings” for transformers)
  • enables parallel computation during training
• Which enables scaling

Sequence of transformations: self-attention (merge concepts) \(\rightarrow\) MLP (memory blocks) \(\rightarrow\) …

Residual stream: each layer adds an \(\varepsilon\) of information

• Modern implementations of transformers improve over the 2017 transformer:
  • RoPE, RMSNorm, Pre-Norm, SwiGLU
  • GQA, no bias in QKV, long context tricks:
    • RoPE scaling, NTK scaling, YaRN, Dynamic scaling, Position interpolation…
  • hybrid layers (MoE, Mamba)
  • …

Revolution 2: scaling laws (2020)

• First and main LLM scaling laws: pretraining

Scaling LLMs

• The more data you train on
  • the more the LLM knows about
  • the better the LLM generalizes
• scaling law = power law = \(y(x) = ax^{-\gamma} +b\)
• \(y(x) =\) test loss
• \(\gamma\) = slope

Baidu paper: DNN scaling laws 2017

Kaplan scaling laws 2020

Open-AI 2020

• RL, protein, chemistry…

Chinchilla scaling laws 2022

• Kaplan 2020: you should scale model size
• Chinchilla 2022: you should scale data

\(L=\) pretraining loss

Google: comparison of architectures 2022

Flops, Upstream (pretraining), Downstream (acc on 17 tasks), Params

Theoretical limits of linear/hybrid attentions

This paper (2026) proves that linear and hybrid attentions (including Mamba, DeltaNet…) are less expressive than full attention (e.g., for sequential function composition = multi-step reasoning).

Emerging capabilities

• Scaling laws exist in Machine Learning for a long time (cf. Paper on learning curves)
• But it’s the first time they result from emerging capabilities!
  • Most important emerging capability: In-Context Learning 2020

Jason Wei has exhibited 137 emerging capabilities:

Anthropic paper 2022

• shows that the scaling law results from combination of emerging capabilities

Revolution 3: test-time scaling (2024)

• During training, more compute ==> better results
• During inference: test-time scaling
• Chain of Thoughts (CoT) (2022)

On the “equivalence” between depth and CoT

• Can we trade depth for CoT length?
• This depends on the relative expressivity of both:
  • Theorem Depth separation: expressivity increases exponentially with depth, poly with width
  • Residual expansion theorem: training instability increases exponentially with depth
  • Theorem: CoT expressivity:
    • CoT(poly(n))=P
    • AoT(poly(n))=NP
    • CoT(exp(n))=EXP
    • AoT(exp(n))=NEXP

• Practical expressivity of CoT:

Why does CoT improve results, while it does not bring any new information?

Computer program point of view

• CoT “gives time” (compute) to reason
• “Chain-of-Thought Tokens are Computer Program Variables”
• The transformer is a deterministic (semi-) Turing machine
  • With a limited-size ribbon (nb of layers)
  • CoT increases ribbon size

• Viewed as program variables, CoT tokens may not be interpretable!
  • CoT is not required to be correct: Beyond Semantics: The Unreasonable Effectiveness of Reasonless Intermediate Tokens
  • But it does help when it is correct: Probing the Trajectories of Reasoning Traces in LLM

How do LLMs learn to reason?

Are LLMs stochastic parrots?

Or do they really reason?

• There is no reasoning without generalization
• Compression = generalization
• During DNN/LLM training, 2 phases occur: memorisation, then generalization
• Tishby, 2015

Phase transitions during training

Grokking exhibits structured latent space; Ex: modular arithmetic

LLM training dynamics

• High-dim training is not a “continuous”/regular process:
  • Phase transitions first observed in 2015 in ConvNets
  • Later studied in LLMs
  • LLMs generalize by discovering the latent structure in data

• World geography emerge

• LLMs learn to learn (in context):
  • LLM learn gradient descent
  • LLM learn 2d order descent

• So, we know that LLM can:
  • not only memorize, but generalize
  • recover latent structures in data
  • learn training algorithms
  • execute these algorithms in their residual stream
• Do they actually do that when generating CoT?

“Procedural Knowledge in Pretraining Drives Reasoning in Large Language Models”

“The approach to reasoning LLMs use looks unlike retrieval, and more like a generalisable strategy synthesising procedural knowledge from many documents doing a similar form of reasoning.”

• Method: Influence functions identify which training data is used when reasoning
• Influence functions: assume removing a training doc, retrain, does prediction change?

Reasoning Qs are weakly influenced by individual docs; factual Qs are.

• Reasoning Qs are mostly influenced by code datasets
• Factual Qs are mostly influenced by wikis

• Additional refs:
  • Transformers Pretrained on Procedural Data Contain Modular Structures for Algorithmic Reasoning
  • Scalable Influence and Fact Tracing for Large Language Model Pretraining

When are LLMs trained to reason?

• Pre: No explicit CoT
• Mid: Supervised CoT (often distilled)
• Post: Generated CoT (RL)

• RL with CoT improve performance on reasoning benchmarks e.g. “Effective Reinforcement Learning for Reasoning in Language Models”
• Q1: Does RL inject novel reasoning capacities into LLM?
• Q2: Can we reduce the large cost of RL?

• RL may be replaced by high Quality supervised CoT:
  • S1 (feb 2025): good performances with 1000 manual CoT
  • Distillation: always train an LLM (without RL) on high quality CoT generated by top-LLMs

Reasoning capacities are already in the base LLM (before post-training)

• Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model?

• Before RL, correct answer is not always first, but is not far

• They checked manually that base LLM generates CoT that are as valid as post RL
• Tested on maths, code, visual reasoning
• Base LLM can potentially solve more problems than after RL

Scalable Power Sampling: Unlocking Efficient, Training-Free Reasoning for LLMs via Distribution Sharpening

• RL does not add any new reasoning capacity, but “sharpen” the distribution \(p(x)^{\alpha}\)
• Without RL, one can draw CoT with MCMC from \(p(x)^{\alpha}\) instead of \(p(x_t | x_{<t})\)
• But MCMC is costly (x9), this paper proposes an efficient approximation

• Other lessons of the paper:
  • RL induces a “diversity collapse”
  • \(\alpha\) impacts performance: \(\alpha=4\) is a good value for maths

• CoTs are in English, because all reasoning datasets are in English, as well as generated CoTs. \(\rightarrow\) Mille-Pensées

Revolution 4: agentic (2025)

AI Agent: LLM + harness + agentic loop

harness: all the code around the LLMs

• Why a revolution?
  • Because, thanks to trial and errors, debugging, planning, agentic AI enables to solve much more complex tasks than a single LLM
  • The LLM is trained to run tools and analyze their outputs, try to understand why it failed, debug the process, rerun until it works
  • Ex: solved one of the 10 major Erdös problem

Training an agentic LLM requires moving beyond standard text corpora. While a base LLM needs general language understanding, an agent specifically requires data that teaches reasoning, planning, tool usage, environment interaction, and multi-turn error recovery.

Standard training pipeline to train an agent: Pre-training → Agentic SFT → RLVR

Agents orchestration

• Various types of orchestration
  • Central: main LLM = orchestrator
  • Hierarchic: several levels of orchestrators
  • “Control graph”: controlled programmatically
  • Swarm: many small specialized LMs

Opportunity for small LLMs

• More and more “small” focused LLMs are developed
  • for code: 3b LLM
  • for repository exploration: Microsoft 4b LLM
  • for agentic loop: Ornith 9b
  • for long-horizon planning: Agents A1 35b

Thank you!

cerisara@loria.fr

https://ia.loria.fr