88Z AI, 6 November — Baidu has officially open-sourced the ERNIE-5.5-Flash-Base pre-trained base model, with weights now available on Hugging Face. The model offers FP8 and BF16 precision versions, is compatible with SGLang and vLLM deployment frameworks, and supports fine-tuning via Megatron-Bridge. As a sparse MoE architecture model, it features 280 billion total parameters with 14 billion activated per token, pre-trained from scratch on 33.5 trillion tokens, and supports a context window of up to 1 million tokens
At a glance
- The model has 280 billion total parameters with 14 billion activated per token and supports a 1M context window
- Available in FP8/BF16 formats, compatible with SGLang, vLLM, and Megatron-Bridge
The update
Baidu's newly open-sourced ERNIE-5.5-Flash-Base pre-trained base model has released its weights on the Hugging Face platform. To facilitate developer usage, the model provides both FP8 and BF16 precision versions and fully supports mainstream deployment frameworks like SGLang and vLLM, while also allowing efficient fine-tuning through the Megatron-Bridge tool. In terms of architectural design, the model adopts sparse MoE technology, boasting a total of 280 billion parameters but activating only 14 billion per token during inference, balancing performance and efficiency. Pre-trained from scratch on 33.5 trillion tokens of high-quality data, it possesses strong language understanding capabilities and supports an ultra-long context window of up to 1 million tokens during fine-tuning and inference, making it suitable for complex long-text tasks.
Sources & editorial note
This article is an AI-assisted synthesis by 88Z AI based on the public sources below. Product claims and availability are attributed to their publishers. It does not represent independent testing or first-hand reporting.
- 文心大模型Source published
Source dates above refer to the original announcements; the date under the headline is this article’s publication on 88Z AI.