Tencent released Hunyuan-TurboS in February 2025, making it one of the first production AI labs to ship a hybrid Mamba-Transformer model at frontier scale. Where every major 2024 model used pure transformer architecture, TurboS introduced Mamba SSM (State Space Model) layers interleaved with standard attention layers — targeting the fundamental inefficiency of transformers on long sequences.

The rationale: attention computation scales quadratically with context length; Mamba processes each token in constant memory regardless of sequence length. A hybrid model can use attention for tasks requiring precise token-to-token reasoning and Mamba for efficient processing of long sequences, combining the strengths of both.

Hunyuan-TurboS reached LMSYS Chatbot Arena top-7 at release — competitive with GPT-4o, Claude 3.5 Sonnet, and Gemini 1.5 Pro on general capability while offering significantly faster inference on long contexts.

What's New

What's New

Mamba-Transformer hybrid architecture: TurboS interleaves Mamba2 SSM layers with standard multi-head attention layers across the model depth. The specific ratio and placement is tuned for quality-performance tradeoff: attention layers at lower depths (where precise token relationships matter most) and Mamba layers at higher depths (where long-range context compression is more valuable). This is the defining architectural departure from every other frontier model at February 2025.

4× faster long-context decode: On sequences over 32K tokens, TurboS generates output tokens at approximately 4× the speed of a comparable pure-transformer model. Mamba layers process context in O(n) rather than O(n²), which translates directly to faster first-token latency and token generation speed on long documents.

256K context window: TurboS supports 256K token context — competitive with Claude 3.5 Sonnet and Gemini 1.5 Pro. The Mamba layers make processing the full 256K context practical at lower GPU memory cost than a pure transformer would require.

LMSYS Arena top-7: At February 2025 release, Hunyuan-TurboS placed in the top 7 on LMSYS Chatbot Arena — the human preference ranking that is harder to game than academic benchmarks. This placed it in the same tier as GPT-4o and Claude 3.5 Sonnet on general conversational quality.

Hunyuan ecosystem integration: TurboS is available via Tencent Cloud's Hunyuan API and integrates with Tencent's enterprise AI tooling — Tencent Meeting, WeCom, and custom enterprise deployments. For developers building on Tencent Cloud infrastructure, TurboS is the natural frontier model choice.

Architecture

Architecture

Component Hunyuan-TurboS
Architecture Mamba2-Transformer hybrid
Context window 256K tokens
Attention layers Interleaved with Mamba2 layers
Token generation 4× faster on long context vs dense transformer
Languages Chinese + English (primary), multilingual
Availability Tencent Cloud API
License Proprietary

Mamba2 SSM mechanism: Mamba2 processes sequences through a structured state space — a recurrent computation that updates a hidden state at each token position. Unlike attention, which attends to all previous tokens simultaneously, Mamba maintains a compressed state that captures long-range dependencies without quadratic memory growth. The tradeoff: Mamba is less precise for tasks requiring exact token-to-token retrieval, which is why TurboS combines it with attention rather than replacing attention entirely.

Hybrid layer placement strategy: The specific interleaving pattern — which depths get attention vs Mamba — is the core architectural decision in a hybrid model. Tencent's placement follows empirical results: attention dominates in early layers where fine-grained pattern recognition matters, Mamba in middle and late layers where sequence-level context compression is more useful.

Linear attention scaling: The Mamba layers are the key to TurboS's long-context efficiency. A pure transformer processing 256K tokens with multi-head attention has KV cache memory scaling at O(n·d) and attention computation at O(n²·d) per layer. Mamba layers replace this with O(d) memory per position regardless of sequence length. The hybrid model achieves sub-quadratic scaling in practice.

Chinese language optimization: Tencent specifically tuned TurboS for Chinese language quality — both because Tencent's primary market is domestic Chinese users and because Chinese tokenization characteristics (character-level vs byte-pair encoding) create different optimization requirements than English-first models.

Benchmarks

Benchmarks

Hunyuan-TurboS at February 2025 release:

Benchmark TurboS GPT-4o Claude 3.5S Gemini 1.5 Pro Notes
LMSYS Arena Elo Top-7 Top-5 Top-3 Top-8 Human preference
MMLU 87.4% 87.2% 88.3% 85.9% General knowledge
GSM8K 94.8% 95.3% 96.4% 94.4% Math word problems
HumanEval 88.3% 90.2% 93.7% 86.3% Code generation
Long-context TTFT 4× faster baseline ~0.9× ~1.1× Time-to-first-token, 64K ctx
Chinese-specific MOS high medium medium medium Expert subjective rating

LMSYS Arena top-7 is the most competitive benchmark: it reflects real users choosing between models in head-to-head conversations, across languages and task types. Top-7 at February 2025 placed TurboS in the same tier as the best models from OpenAI, Anthropic, and Google.

The long-context TTFT advantage is hardware-measured: on 64K token inputs, TurboS generates the first output token approximately 4× faster than a pure-transformer model of comparable size. This is the direct architectural dividend from Mamba layers.

Pricing in Context

Model Availability Context Architecture
Hunyuan-TurboS Tencent Cloud 256K Mamba-Transformer
GPT-4o OpenAI API 128K Transformer
Claude 3.5 Sonnet Anthropic API 200K Transformer
Gemini 1.5 Pro Google AI API 2M Transformer

TurboS pricing via Tencent Cloud Hunyuan API is competitive with Chinese lab alternatives — roughly comparable to Qwen2.5-Max pricing. The key advantage is Tencent ecosystem integration: for enterprises using Tencent Cloud, WeCom, or other Tencent services, TurboS avoids cross-cloud data egress and integrates with existing IAM and billing.

What It Means for Developers

Long-context applications at lower latency: The 4× faster decode on long contexts is directly useful for document analysis, legal review, and research synthesis applications where 64K–256K token inputs are the norm. Faster first-token latency improves perceived responsiveness in user-facing applications.

256K context for large document workflows: At 256K tokens, TurboS can process book-length documents (a 500-page report is roughly 150K tokens) in a single context window. For Tencent Cloud customers, this is the first model enabling full-document processing without chunking strategies.

Mamba architecture as production proof: TurboS demonstrated that Mamba-Transformer hybrids work at frontier scale in production. This was the first major lab to ship this architecture commercially, establishing that the quadratic attention bottleneck is not inevitable — setting up the architectural experimentation that continued through 2025 and 2026.

Tencent ecosystem fit: TurboS integrates with Tencent Meeting transcription, WeCom document processing, and Tencent Cloud enterprise AI. For enterprise customers already in the Tencent ecosystem, TurboS provides frontier-quality AI with native integration rather than requiring cross-cloud API calls.

Availability

Bottom Line

Hunyuan-TurboS was the first production frontier model to prove that Mamba-Transformer hybrids work at scale. The 4× faster long-context decode is not a benchmark trick — it's the direct result of replacing quadratic attention with linear-complexity Mamba layers for most of the network. LMSYS Arena top-7 confirmed the quality tradeoff was acceptable. For developers on Tencent Cloud building long-context applications, TurboS offered both the capability tier they needed and architecture-native speed advantages that pure transformers can't match.

Resources

Hunyuan-TurboS is available via Tencent Cloud at cloud.tencent.com/product/hunyuan.