The model releases get the headlines. The infrastructure drops are where DeepSeek is actually building a moat.
Over the past week, DeepSeek pushed updates to DeepGEMM (matrix multiply), FlashMLA (efficient MLA attention), and now TileKernels — a library of 44 production-grade GPU operators covering the exact architectural primitives that power V3/V4. The same day, a preview of DeepEP v2 appeared in a PR: 1.3x peak performance improvement over v1, 4x reduction in SM resource usage. All MIT licensed.
This isn't open-source as PR. These are the actual building blocks. And for anyone trying to understand DeepSeek's cost efficiency at scale, or trying to run similar architectures outside of NVIDIA's ecosystem, this release is the most practically useful thing they've published since V2.
What Just Shipped
TileKernels contains 44 GPU operators across five modules, all written in TileLang — a Python-based GPU DSL developed at Peking University, built on top of TVM. Zero CUDA C++.
The five modules map directly to DeepSeek's V3/V4 architecture:
- moe/ — 14 operators for MoE routing full pipeline
- quant/ — 15 operators for FP8 quantization and fused activations
- engram/ — 5 operators for conditional memory (first-ever production kernel release)
- mhc/ — 10 operators for Manifold HyperConnection (also first kernel-level open source)
- transpose/ — 1 batch transpose operator for quantization pipelines
Hardware requirement: SM90 (H100/H800) or SM100 (B100/B200), CUDA 13.1+. Consumer cards like RTX 4090 (SM89) don't qualify. TileLang's backend supports NVIDIA, AMD MI300X, and Huawei Ascend — the portability is baked into the DSL choice, not bolted on.
Inside the Five Modules
moe/ (14 operators) covers every step of the MoE routing pipeline: top-k gating, expert scoring, token-to-expert mapping, fused expand/reduce, tensor-parallel masks, and load-balancing loss computation. DeepSeek V3 uses 256 experts, activating 8 per token — routing overhead is a real bottleneck, and these operators are the production answer.
quant/ (15 operators) implements DeepSeek's FP8 mixed-precision training strategy, which first appeared in V3. Three quantization granularities: per-token, per-block, per-channel. The standout is a custom float format called E5M6 — designed specifically for attention gradients where FP8 is too imprecise for training stability but FP16 wastes memory bandwidth. The module also includes triple-fused operators that combine SwiGLU activation + FP8 quantization + matrix transpose in a single pass. What normally requires three memory round-trips becomes one, which matters a lot at MLP-layer scale.
engram/ (5 operators) is significant beyond its line count. Engram was published in January 2025 as an architectural proposal for adding a conditional memory dimension alongside MoE's conditional compute. Until TileKernels, only the paper existed. Now there's a fused RMSNorm + signed-sqrt gate kernel, multi-head hash addressing, weight fusion, and gradient aggregation — the first production-grade kernel implementation of Engram, anywhere.
mhc/ (10 operators) is the same story for Manifold HyperConnection (mHC). The architecture addresses residual connection signal amplification in deep models using the Sinkhorn-Knopp algorithm to constrain mixing matrices to the Birkhoff polytope. Published result: 6.7% training overhead increase in exchange for measurably more stable deep-network training. TileKernels makes this the first kernel-level open-source implementation of mHC.
transpose/ (1 operator) handles row-major/column-major conversion in quantization pipelines — unglamorous but a required primitive.
TileLang: Why Not CUDA?
The choice to write everything in TileLang rather than CUDA C++ is a deliberate engineering position, not just a styling preference. TileLang sits between Triton and hand-written CUDA: more hardware control than Triton, significantly less engineering overhead than CUDA. DeepSeek has been using it since V3.2.
The more important reason is hardware portability. TileLang's TVM-based backend compiles to NVIDIA (CUDA), AMD (ROCm / MI300X), and Huawei Ascend targets from the same source. For DeepSeek — a Chinese AI lab with obvious strategic interest in non-NVIDIA hardware paths — this is a meaningful hedge. Every kernel written in TileLang is a kernel that doesn't need to be rewritten when the hardware mix changes.
For external users, this means the TileKernels operators are not NVIDIA-exclusive even though the hardware requirements currently spec H100/H800/B100/B200. The portability ceiling is the DSL, not the algorithms.
DeepEP v2: The Communication Layer
The MoE routing operators handle the compute side of expert dispatch. DeepEP handles the communication side — the all-to-all traffic that moves tokens between GPUs when different experts live on different nodes. EPv2, previewed in a PR this week, is a full rewrite of v1.
Numbers from the PR: 1.3x peak performance improvement over v1, 4x reduction in SM resource usage for V3-class workloads (from 24 SMs down to 4-6 SMs). Scale ceiling: EP2048 (2048-way expert parallelism).
The architectural changes: unified API for high-throughput and low-latency use cases (previously split), backend swap from NVSHMEM to the lighter NCCL Gin, and an explicit goal of reducing auto-tuning dependency for better long-term maintainability. There's also an experimental 0 SM overhead mode being investigated.
The 4x SM savings matter in practice because SM contention between communication and compute kernels is a real bottleneck at scale. Freeing those SMs gives the compute kernels more headroom without adding hardware.
What It Means for Developers
The practical implications split by use case:
If you're training MoE models on H100/H800: TileKernels is drop-in reference material at minimum and usable production operators if your architecture matches (FP8 mixed precision, standard MoE routing, V3-adjacent designs). The triple-fused quant operators and the moe/ routing implementations are the highest-value immediate pickups.
If you're researching Engram or mHC: You now have working kernel code, not just paper architecture descriptions. Both Engram and mHC had zero public kernel-level implementations before this release. The TileKernels implementations are production-tested, not prototypes.
If you're planning for non-NVIDIA hardware: TileLang's portability means these operators can in principle run on AMD MI300X and Huawei Ascend. Practical support will require testing, but the portability path exists at the DSL level.
If you're evaluating DeepSeek's infrastructure maturity: The pattern here — DeepGEMM, FlashMLA, TileKernels, DeepEP v2, all in the span of weeks — suggests deliberate systematic open-sourcing of the full training and inference stack. The valuation signal (The Information reporting $20B+, with informal estimates reaching $40B) isn't unconnected to this. Open infrastructure is both goodwill and recruitment, and DeepSeek is evidently doing both intentionally.
The hardware requirement ceiling (SM90/SM100, no consumer GPUs) means this is not accessible to the average developer experimenting locally. But for teams with datacenter access or cloud H100 instances, TileKernels is the most technically specific open-source release DeepSeek has made about how their model actually runs.
Bottom Line
TileKernels is the missing documentation layer for DeepSeek's architecture. The paper for Engram and mHC existed. The kernel-level implementations didn't — until now. Combined with DeepEP v2's SM efficiency improvements, this is a coherent week of releases that collectively describe how V3/V4 actually achieves its cost efficiency numbers. All MIT licensed. All in TileLang, which means portable to AMD and Ascend in principle.
The $20B+ valuation conversation will continue. The infrastructure drops are what make it legible.
Resources
- TileKernels GitHub — 44 GPU operators, MIT license
- DeepEP GitHub — MoE expert-parallel communication library (EPv2 PR)
- TileLang GitHub — Python GPU DSL, TVM-based
- Engram Paper — arXiv:2501.12352 (January 2025)
- DeepSeek Official Site — main product and research hub
DeepSeek's text API is live at api-docs.deepseek.com. TileKernels and DeepEP are open-weight infrastructure libraries available at github.com/deepseek-ai.
