Kimi k1.5: Long-Context RL and Multimodal Reasoning at 1M Token Scale

Moonshot AI released Kimi k1.5 in January 2025 with a focus on an underexplored dimension of reasoning model training: scaling reinforcement learning with long-context data. While DeepSeek R1 (released the same month) made GRPO and RL-based reasoning training mainstream, Kimi k1.5 made a different bet — that the combination of long-context capability and RL scaling, applied to a multimodal model, would produce a distinctive capability profile.

The result: state-of-the-art performance on math and coding benchmarks, genuine multimodal reasoning (images and text in the same context), and a 1M token context window — all in one model.

The Long-Context RL Approach

Most reasoning model training at the time of Kimi k1.5's release used RL with relatively short contexts — the thinking chain and the problem both fit in a few thousand tokens. This is efficient and sufficient for math problems, but it doesn't represent the actual reasoning situations developers encounter: debugging a function while reading a 50K-line codebase, analyzing a research paper while cross-referencing prior work, interpreting an image in the context of a long technical document.

Kimi k1.5's approach: RL training that explicitly incorporates long contexts, training the model to reason across sequences that match real-world agentic complexity. The training pipeline uses a combination of short-context RL (for mathematical reasoning quality) and long-context RL (for coherent extended reasoning chains).

Long rollout RL: The model generates longer reasoning chains during training, and receives reward signals that evaluate the quality of the full reasoning process — not just the final answer. This pushes the model to develop more coherent multi-step reasoning strategies.

Short-to-long curriculum: Training starts with shorter context problems (where signal is cleaner) and progressively introduces longer contexts. This prevents the model from learning shortcuts that work only at short context lengths.

Architecture and Capabilities

Kimi k1.5 is a dense transformer (not MoE) optimized for reasoning and long-context coherence. The key specifications:

• Context length: 1 million tokens — matching MiniMax-01 released the same month
• Modality: Text and image input (multimodal); text output
• Reasoning mode: Generates extended thinking chains before final answer, in a format similar to QwQ-32B and DeepSeek R1
• Training: Multi-stage RL with curriculum learning across context lengths

The multimodal capability extends beyond image captioning — Kimi k1.5 can reason about mathematical diagrams, charts, screenshots of code, and technical figures while maintaining coherent reasoning chains that incorporate the visual content.

Benchmarks

Kimi k1.5 performance at January 2025 release:

The AIME score of 77.5% places Kimi k1.5 in the same tier as o1 and R1 on mathematical reasoning — competitive, though R1 edges ahead. The stronger signal is the multimodal benchmarks: MathVista (74.9%) and MMMU (73.4%) show genuine integration of visual and textual reasoning, not just added image captioning.

LiveCodeBench is the weakness — 47.3% vs. o1's 63.4% indicates the coding reasoning was less prioritized than math. For coding-intensive agentic use cases, this is where K2 (released 6 months later) addressed the gap.

Long-Context Performance

The published evaluation on long-context benchmarks:

The Loong-Reasoning benchmark specifically tests reasoning over long documents — finding the relevant information in a 50K+ token context and reasoning about it correctly. Kimi k1.5's 80.1% vs. GPT-4o's 65.4% reflects the long-context RL training paying off.

What's Distinctive vs. R1

DeepSeek R1 and Kimi k1.5 launched within days of each other, both as RL-trained reasoning models. The differences:

R1: Pure text, distilled variants for deployment, open weights, GRPO training, leads on LiveCodeBench (65.9% vs 47.3%)

k1.5: Multimodal (images + text), 1M token context, long-context RL training, leads on Loong-Reasoning (80.1% vs ~55%)

For developers building text-only coding assistants or math tools, R1 is the stronger baseline. For developers building agents that need to process mixed visual/textual long-form content — research assistants, document analysis pipelines, scientific computing tools — Kimi k1.5 addresses a different need.

What It Means for Developers

Multimodal reasoning in production: Before k1.5, combining image understanding with extended reasoning chains required either GPT-4V (expensive, API-only) or patchwork solutions. k1.5 integrates both in a single model with a coherent context window.

Long-context reasoning chains: For workflows where the model needs to reason over long documents (legal analysis, literature review, large codebase debugging), k1.5's long-context RL training produces more coherent reasoning than models trained only on short-context RL.

1M token context with reasoning: The combination of 1M context and extended reasoning mode is uncommon. Most reasoning models have short context limits. k1.5 allows reasoning chains that draw on full-document context.

Foundation for K2: Moonshot used the lessons from k1.5's post-training pipeline to inform K2's agentic data synthesis approach. Understanding k1.5 explains why K2's agentic performance is so strong — the RL methodology evolved substantially between releases.

Availability

Kimi k1.5 is available via:

• Kimi API at platform.moonshot.cn
• Kimi Chat web interface (chat.moonshot.cn) for interactive use
• The technical report is available as arXiv 2501.12599

Unlike K2, k1.5 weights have not been released publicly — it's an API-only model. The technical report details the training methodology for researchers.

Bottom Line

Kimi k1.5 established Moonshot AI as a serious player in reasoning model training, competitive with o1 on mathematical reasoning and distinctively ahead on multimodal long-context tasks. The long-context RL methodology it introduced became a design influence on K2's architecture six months later.

For developers working with multimodal long-context reasoning tasks, k1.5 remains relevant — K2 improved coding and agentic performance substantially, but k1.5's visual reasoning and long-context coherence represented a specific capability investment that K2 continued to build on.

Resources

• Technical Report (arXiv 2501.12599) — long-context RL methodology, benchmarks
• Kimi API Platform — API access
• Kimi Chat — web interface

Kimi k1.5 is available via API at platform.moonshot.cn. Weights have not been publicly released.