Baidu Releases ERNIE 5.5 Flash Base: A 280B MoE Base Model With 14B Active Parameters

China's open-weight race keeps shipping sparse giants aimed at teams that want to fine-tune from a raw base, not a chat-tuned endpoint. Baidu's ERNIE 5.5 Flash Base activates 14B parameters per token inside a 280B MoE, trains to 128K context with 1M for fine-tune and inference, and stops at pretraining with no mid- or post-training applied.

Baidu's unified eval pipeline — self-reported — has it best on 24 of 30 base-model benchmarks, including MMLU-Pro at 78.4 versus 69.2 for DeepSeek-V4-Flash-Base. Weights land on Hugging Face and ModelScope in FP8 and BF16, with SGLang and vLLM recipes.

Another serious base model for labs comparing fine-tune and RL stacks from a shared starting line.

We're interested in clean base checkpoints more than leaderboard theater. Flash Base's pitch is architectural: hybrid Heavily Compressed Attention in 37 of 43 layers caching one entry per 128 tokens, MLA+DSA in the rest, Multi-Router Latent MoE with 512 routed experts (top-10) plus a shared expert, and ECHO turning the final layer into a built-in speculative draft head for cheap future-token prediction.

Pretraining on 33.5T tokens with Muon, heavy on repository-scale code and long context, is the substantive claim. Beating DeepSeek, Tencent Hy3, and Xiaomi MiMo on Baidu's own harness — especially MMLU-Pro and HumanEval+ — will need third-party replication. It loses on a handful of Chinese and long-context rerank tasks, which is a useful tell that the crown is not universal.

License terms matter as much as FLOPs: free under $50M revenue but excluding the EU, UK, and South Korea per MarkTechPost's read. That geopolitics-shaped openness is becoming normal and still constrains who can build. Useful open bases accelerate research; geography clauses and self-reported scores are the fine print. Verify before you standardize.

Serving docs show 4-way tensor parallel examples and up to 1,048,576-token contexts on SGLang/vLLM paths. As a base model it targets completions, not chat templates. FlashMask-4 kernels on SM90/SM100 are part of Baidu's training efficiency story.