Alibaba's Qwen team released Qwen3 in May 2025 with a design choice that addressed a real pain point: you shouldn't need two different models for tasks that sometimes require deep reasoning and sometimes don't.
The previous generation required switching between Qwen2.5 (fast, context-driven) and QwQ-32B (reasoning model, slow). Qwen3 collapses both into one model with a mode toggle — thinking mode for complex problems, non-thinking mode for direct responses — with a thinking budget mechanism to tune the depth in between.
The result: 8 models from 0.6B to 235B, all Apache 2.0, deployed on Alibaba Cloud and available on HuggingFace.
The Model Family
Qwen3 releases two architecture types:
Dense models: 0.6B, 1.7B, 4B, 8B, 14B, 32B — standard transformers, good for fine-tuning, local deployment, and edge cases.
MoE models: Qwen3-30B-A3B (30B total / 3B active) and Qwen3-235B-A22B (235B total / 22B active) — the flagship models for production deployment.
The 235B flagship is the one that benchmarks against o3 and other frontier models. At 22B active parameters per token, it's comparable to dense ~22B models at inference time while carrying the knowledge capacity of a 235B model.
All eight models integrate both thinking and non-thinking modes. All are Apache 2.0.
Hybrid Thinking Architecture
The core innovation is how Qwen3 handles mode switching without separate models.
Non-thinking mode: Standard autoregressive generation. The model responds directly, using context from the conversation. Fast, low-latency. Equivalent to Qwen2.5 behavior.
Thinking mode: The model generates an internal chain of thought inside <think>...</think> tags before outputting the final answer. The thought process is visible to the developer (and can be displayed to users or hidden). Equivalent to QwQ-32B behavior but integrated into the same weights.
Thinking budget: A numeric parameter (0 to 38,912 tokens) that caps how many tokens the model can spend in thinking mode before producing an answer. Set to 0 for non-thinking mode. Set to maximum for maximum reasoning depth. This is the practical lever for cost-performance tradeoff — a thinking budget of 2,000 tokens gives noticeably better results than non-thinking on hard problems, at roughly 3–4× the token cost.
The budget mechanism is trained in, not hacked on top. DeepSeek R1 introduced extended thinking; Qwen3's contribution is making the depth controllable and training the model to use budgets efficiently.
How mode is set: Chat template — add /no_think or /think system-level tag, or set enable_thinking=True/False in the transformers generation config. No separate API endpoint needed.
Training at 36T Tokens
Qwen3's pre-training dataset is 36 trillion tokens — up from the ~18T tokens estimated for Qwen2.5. Key composition choices:
STEM-heavy curation: Significantly higher proportion of math, code, and scientific text vs. Qwen2.5. Qwen3's benchmark improvements are concentrated in reasoning-heavy domains, consistent with training data skew.
Multilingual expansion: Qwen2.5 supported 29 languages. Qwen3 covers 119 languages and dialects. This isn't just token count — the team added structured multilingual alignment to ensure cross-lingual transfer quality.
Synthetic reasoning data: The thinking mode capability is trained, not prompted. Qwen3 uses synthetic chain-of-thought data generated by larger models in the Qwen family, following the self-distillation pattern established by R1's distilled variants.
The thinking budget mechanism is trained via RL: the model learns to allocate thinking effort to problem difficulty, using budget tokens efficiently rather than padding.
Benchmarks
Qwen3-235B-A22B in thinking mode, at May 2025 release:
| Benchmark | Qwen3-235B | DeepSeek-R1 | o3-mini | Notes |
|---|---|---|---|---|
| AIME 2024 | 85.7% | 79.8% | 79.6% | Hard math |
| MATH-500 | 97.4% | 97.3% | 97.2% | Competitive math |
| LiveCodeBench | 70.7% | 65.9% | 66.3% | Coding contest |
| BFCL v3 | 70.8% | 56.5% | 67.5% | Function calling |
| GPQA Diamond | 71.1% | 71.5% | 71.8% | Graduate-level sci |
| Multilingual (M-MMLU) | 79.8% | 72.1% | — | 14 languages |
Qwen3-235B leads on coding (LiveCodeBench), function calling (BFCL), and math (AIME), while running close to R1 on scientific reasoning (GPQA). The multilingual gap is the clearest win — 7+ points over R1.
For the dense models: Qwen3-32B in thinking mode outperforms QwQ-32B (its predecessor reasoning specialist) on all benchmarks — same parameter count, significantly better results.
Non-thinking mode benchmarks (Qwen3-235B vs. Qwen2.5-72B):
| Task | Qwen3-235B | Qwen2.5-72B | Delta |
|---|---|---|---|
| MMLU-Pro | 81.4% | 71.1% | +10.3 |
| IFEval | 92.3% | 86.8% | +5.5 |
| Arena-Hard | 92.1% | 81.2% | +10.9 |
In direct response mode, the larger model still shows substantial gains — non-thinking mode is not "dumb mode," it's the full model without extended chains.
Practical Deployment
The 30B-A3B is the sweet spot for most applications:
- Runs on a single A100 80GB with reasonable throughput
- Non-thinking mode latency comparable to a standard 3B dense model
- Thinking mode provides R1-quality reasoning on hard tasks on demand
The 235B-A22B requires multi-GPU setup (~8×A100) but is the go-to for API deployments where cost per token matters less than quality.
Both are available on:
- Alibaba Cloud Model Studio (commercial API)
- HuggingFace (self-hosted weights)
- Together AI, Fireworks, and other inference providers
The Apache 2.0 license covers all models including commercial use, fine-tuning, and redistribution without additional licensing conditions.
What It Means for Developers
Eliminate model switching: If you're currently routing between a fast model and a reasoning model, Qwen3 lets you run a single model and toggle thinking per-request. The thinking budget parameter means you can set different depth levels per query type — simple tasks get 0 budget, hard problems get 8K, maximum-difficulty tasks get the full 38K.
Multilingual at frontier quality: 119 languages at GPT-4 tier quality changes the math for international applications. Previous open-weights options required Qwen2.5 (29 languages) or fine-tunes with variable quality.
Thinking cost is tunable: The standard developer complaint about reasoning models is unpredictable cost (you don't know how many thinking tokens will be used). The thinking budget makes this controllable: set a max, pay at most that much, and the model allocates within that budget optimally.
Fine-tuning base: Qwen3 dense models (especially 8B and 32B) are already popular fine-tuning bases. The Apache 2.0 license and strong instruction-following make them straightforward to fine-tune for domain-specific applications.
Bottom Line
Qwen3 closes the gap between specialist models and general-purpose models. The 235B flagship matches or exceeds DeepSeek-R1 on most reasoning benchmarks while also being a strong general-purpose model in non-thinking mode. The practical win is the thinking budget — it makes reasoning quality accessible without the cost unpredictability that makes models like o3 expensive to deploy at scale.
The 36T-token training and 119-language support position it as the strongest multilingual open-weights model as of mid-2025. For developers building international applications, this changes the tradeoff calculus entirely.
Resources
- Technical Report (arXiv 2505.09388) — architecture, training methodology, full benchmarks
- Qwen3 Blog Post — release announcement and model overview
- HuggingFace — Qwen3-235B-A22B — flagship model weights
- HuggingFace — Qwen3-32B — recommended dense variant
- Alibaba Cloud Model Studio — API access
Qwen3 models are available via API on Alibaba Cloud Model Studio and self-hosted via HuggingFace. All models are Apache 2.0 licensed including commercial use.
