DeepSeek-R1: The Open-Source Reasoning Model That Matched o1 and Changed Everything

On January 20, 2025, DeepSeek dropped a model that sent Nvidia's stock down 17% in a single day. Not because the benchmarks were impressive — they were — but because of what the benchmarks implied about cost.

DeepSeek-R1 matched OpenAI's o1-1217 on AIME 2024 math competition problems, HumanEval coding, and MMLU reasoning. It did this as a fully open-weights model, under MIT license, with an inference cost roughly 30× lower than o1 at launch. The training cost — which DeepSeek reported as approximately $5.5M for the full model — was widely disputed but never credibly refuted.

The question R1 answered: can a model learn to reason using pure reinforcement learning, without hand-labeled chain-of-thought data? The answer was yes, and the mechanism was stranger than expected.

What R1 Actually Is

R1 is not a single model. DeepSeek released two architecturally distinct systems:

DeepSeek-R1-Zero: Trained from DeepSeek-V3 using pure RL — no supervised fine-tuning, no human-labeled reasoning traces. The model spontaneously developed chain-of-thought reasoning, including self-verification and backtracking, as emergent behaviors. It matches o1-preview on benchmarks but has readability problems: inconsistent language mixing, unstructured outputs, and occasional incoherence.

DeepSeek-R1: Fixes R1-Zero's readability issues through a multi-stage pipeline. Cold-start data (a small set of curated long-CoT examples) establishes readable output format before RL begins. Two RL stages and two SFT stages alternate, progressively improving both reasoning quality and output structure.

Both share the same underlying architecture as DeepSeek-V3: 671B total parameters, 37B active per token, Mixture-of-Experts with 256 routed experts + 1 shared expert, Multi-Head Latent Attention (MLA).

Six distilled models ship alongside the full 671B: 1.5B, 7B, 8B, 14B, 32B, and 70B dense models distilled from R1's reasoning traces into Qwen and Llama base architectures. The 70B distilled model beats o1-mini on several benchmarks.

The Training Architecture

The R1 training pipeline has four stages, alternating between RL and supervised fine-tuning:

Stage 1 — Cold start SFT: A small dataset of long chain-of-thought examples teaches the model basic readable output format (markdown, language consistency). This runs before any RL.

Stage 2 — Reasoning RL: Group Relative Policy Optimization (GRPO) with rule-based rewards. The reward function is intentionally simple: correctness reward for math (verified against ground truth) and code (unit test pass/fail), plus format rewards (presence of <think> tags, no repetition). No learned reward model — this is the key design decision that eliminates reward hacking.

Stage 3 — Rejection sampling SFT: Sample rollouts from the RL-trained model, filter for quality, use as supervised fine-tuning data for a second pass. This distills RL-learned behaviors into stable SFT examples.

Stage 4 — Alignment RL: Second RL stage focused on helpfulness and safety, running against a combination of rule-based and model-based reward signals.

The emergent behaviors from Stage 2 are the paper's most surprising finding. Without being trained to do so, R1-Zero develops:

• Self-verification: The model rechecks its own outputs mid-chain
• Backtracking: When a reasoning path leads to contradiction, the model explicitly restarts with "Let me reconsider..."
• Reflection: Extended reasoning chains that surface and examine assumptions

These behaviors emerge from the RL objective alone, not from data.

Benchmarks

Key results at release vs. OpenAI o1-1217:

R1 is marginally behind o1-1217 on knowledge (MMLU) and competitive coding (Codeforces) while matching or exceeding it on math (AIME) and practical coding (HumanEval, SWE-bench).

The distilled models punch significantly above their weight:

Architecture: Why It Works

The underlying V3 architecture has two unusual choices that matter for reasoning models specifically:

Multi-Head Latent Attention (MLA): Compresses the KV cache by projecting keys/values through a low-rank bottleneck before storing. This cuts KV cache memory by ~40% vs standard attention, enabling significantly longer context at fixed memory budget. For reasoning models that need to "think" at length, this matters — R1's 128K context window is usable in practice because the KV cache doesn't explode.

MoE with auxiliary-free load balancing: Previous MoE models add explicit auxiliary loss terms to prevent routing collapse (all tokens going to a few experts). DeepSeek-V3/R1 uses a bias-correction mechanism instead, adjusting routing logits based on historical load. This avoids the performance penalty that auxiliary loss imposes while maintaining balanced routing.

Both techniques are described in the V3 technical report; R1 inherits them directly since it initializes from V3 checkpoints.

The GRPO Training Signal

Standard RL fine-tuning for language models uses PPO (Proximal Policy Optimization), which requires a separate value/critic model adding memory and compute overhead. GRPO eliminates the critic by estimating the value baseline from a group of rollouts for the same prompt.

For a given math problem, generate N rollouts. Each correct answer gets reward +1, each incorrect gets 0 (or small penalty). The advantage for each rollout is computed relative to the group mean — no separate critic needed.

This is simpler, cheaper, and empirically works as well as PPO for this use case. The critical insight is that math and code have verifiable rewards: you can check correctness without a reward model. For domains where correctness isn't mechanically checkable, the approach would require a learned reward model with all the associated hacking risks.

Availability and Pricing

R1 is fully open-source under MIT license:

• Full model (671B MoE): Requires ~80 × A100 GPUs for inference at reasonable throughput
• Distilled variants: 7B runs on a single RTX 4090; 32B fits in a 2×H100 setup
• API: Available on DeepSeek platform at $0.14/M input tokens, $2.19/M output tokens at launch (reasoning cache hits: $0.14/M)

For context: o1 API pricing at launch was $15/M input, $60/M output. R1 was approximately 27× cheaper on input and 27× cheaper on output.

What It Means for Developers

The 30× cost difference isn't just a number — it changes what's economically viable to build:

Chain-of-thought at scale: R1 makes it feasible to run step-by-step reasoning on every user query rather than selectively. At o1 pricing, long thinking chains add $0.10–$0.30 per query. At R1 pricing, the same chain costs $0.003–$0.01.

Distilled models for local deployment: R1-Distill-Qwen-7B runs on a laptop GPU and beats o1-mini on AIME 2024. For applications that don't require frontier capability, this eliminates API dependency entirely.

Self-hosted reasoning: The full 671B model is deployable with 8× H100 at roughly $15/hour cloud cost. For high-volume applications, this breaks even against API pricing quickly.

The MIT license removes every restriction on commercial use, fine-tuning, and redistribution.

The Controversy

The $5.5M training cost claim generated extensive analysis and skepticism. The main critique: DeepSeek didn't include the cost of base model training (DeepSeek-V3, which preceded R1 and cost significantly more), pre-training experiments, or failed runs in their $5.5M figure.

The consensus among researchers who analyzed this: the $5.5M figure is likely accurate for the RL fine-tuning stage specifically. The full model training cost, including V3 and all associated experiments, is plausibly $20M–$100M — still far below GPT-4's estimated $100M+ training cost, but not the figure that made headlines.

What's not in dispute: R1's benchmark performance, its open-weights release, and the fact that achieving o1-level reasoning without proprietary training data represents a significant capability demonstration.

Bottom Line

DeepSeek-R1 proved three things that weren't obvious in January 2025:

1. Reasoning capability at o1-level can emerge from RL without supervised reasoning traces
2. Open-weights models can match closed frontier models on reasoning benchmarks
3. The cost gap between Chinese AI labs and US frontier labs is narrower than it appeared

The market reaction (Nvidia's $600B+ single-day loss) was partly about implications, not just the model itself. If reasoning capability can be achieved this cheaply with open weights, the moat protecting proprietary reasoning APIs is thinner than assumed.

For developers: R1 and its distilled variants remain one of the best cost-to-performance options for reasoning tasks as of mid-2026. The 32B distill is the practical sweet spot — competitive with o1 on most tasks at a fraction of the inference cost.

Resources

• Technical Report (arXiv 2501.12948) — full training methodology and benchmarks
• GitHub Repository — model weights and inference code
• HuggingFace — R1 (671B) — full model weights
• HuggingFace — R1-Distill-Qwen-32B — practical deployment option
• DeepSeek API — API access and pricing

DeepSeek-R1 is available via API at platform.deepseek.com at $0.14/M input tokens for non-cached requests. All model weights are MIT licensed.