MobileLLM-R1: Exploring the Limits of Sub-Billion Language Model Reasoners with Open Training Recipes

Kari Jaaskelainen

30 Sep 2025 — 1 min read

High-quality data over sheer quantity: sub‑billion models that reason competitively with only ~2T core tokens

Do we really need tens of trillions of tokens to unlock reasoning in small models? MobileLLM‑R1 argues no. By carefully selecting and resampling about ~2T tokens of high-quality open data, then pretraining to 4.2T tokens followed by a transparent post‑training stack, the authors develop sub‑billion‑parameter reasoning models that punch far above their weight.

What’s notable isn’t just efficiency—it’s results. MobileLLM‑R1‑950M posts an AIME score of 15.5, compared to 0.6 for OLMo‑2‑1.48B and 0.3 for SmolLM‑2‑1.7B. Even more striking: despite using only 11.7% of the tokens reportedly used by Qwen3 for pretraining (36T), the 950M variant matches or surpasses Qwen3‑0.6B across multiple reasoning benchmarks.

How they did it: the team defines metrics to identify beneficial datasets, then curates, mixes, and resamples to amplify signal without ballooning total tokens. The training recipe emphasizes data quality, reasoning‑oriented distributions, and a standard post‑training sequence that stays fully open. The result is a repeatable, open recipe for small reasoning‑first LLMs.

Why it matters: if reasoning can emerge reliably in sub‑billion models with tractable data budgets, high‑quality on‑device and edge reasoning becomes far more practical. Beyond performance, the release includes data sources, mixing ratios, training details, and checkpoints—lowering the barrier for the community to iterate and test new ideas.

Who should care: practitioners building fast, deployable reasoners; researchers studying data scaling laws for reasoning; and teams seeking transparent, reproducible training pipelines.

Paper: arXiv: MobileLLM‑R1
Register: AiFeta

#LLM #Reasoning #SmallModels #DataCuration #OpenSource #EdgeAI #AIME

From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by Composing Old Ones

Evidence that RL teaches genuinely new abilities: compositional skills emerge and transfer across tasks Does RL merely reweight what an LLM already knows—or can it teach genuinely new skills? This paper offers concrete evidence for the latter. Using a controlled, synthetic framework, the authors define “skills” as string transformation

OpenGPT-4o-Image: A Comprehensive Dataset for Advanced Image Generation and Editing

A structured 80k instruction–image corpus spanning 11 domains and 51 subtasks to train unified visual editors Unified models for image generation and editing hit a data ceiling: existing corpora emphasize basic manipulations but miss real‑world complexity. OpenGPT‑4o‑Image tackles this with a hierarchical task taxonomy and automated

Random Policy Valuation is Enough for LLM Reasoning with Verifiable Rewards

ROVER replaces PPO loops with uniform‑policy Q‑values—boosting quality and diversity in math reasoning Popular RLVR methods for LLM reasoning lean on generalized policy iteration (e.g., PPO/GRPO), but suffer instability and diversity collapse. This paper reframes math RLVR as a specialized finite‑horizon MDP with deterministic

CLPO: Curriculum Learning meets Policy Optimization for LLM Reasoning

A dynamic, self‑paced curriculum that restructures problems to match model ability in RLVR Online RL with Verifiable Rewards (RLVR) has boosted LLM reasoning—but most methods treat all problems equally, wasting effort on solved items and flailing on those beyond current capability. CLPO fixes that with a dynamic pedagogy:

Read more

From f(x) and g(x) to f(g(x)): LLMs Learn New Skills in RL by Composing Old Ones

OpenGPT-4o-Image: A Comprehensive Dataset for Advanced Image Generation and Editing

Random Policy Valuation is Enough for LLM Reasoning with Verifiable Rewards

CLPO: Curriculum Learning meets Policy Optimization for LLM Reasoning