DeepSeek Open-Sources Production-Grade Inference Framework DeepInfer

The update

DeepSeek has officially announced the full open-sourcing of DeepInfer, its production-grade internal inference framework. This move continues its open-source strategy, following previous releases of its large models, TileLang compiler, and inference operator library. DeepInfer integrates high-performance operators with a dedicated optimization engine, demonstrating strong engineering capabilities. Functionally, it supports distributed inference, speculative decoding, SWA Bounded Replay, and structured output, meeting the demands of complex inference scenarios. In terms of hardware compatibility, the framework supports both NVIDIA and Ascend platforms, showing broad adaptability. Currently, DeepInfer is deployed in production, powering the high-throughput and low-latency services of DeepSeek V4.1 Flash. Additionally, DeepSeek has simultaneously open-sourced the deepseek-recipe encoding and decoding components, further enhancing its open-source ecosystem and providing developers with a more complete toolchain.