# Frontier Pre-training at Microsoft AI | Microsoft AI

The MoE model has 80 billion active and 2.8 trillion total parameters, trained on 16k GB200 GPUs over 38 days across 50 trillion tokens.

- Achieves 4.2x to 12.3x higher pre-training FLOPs efficiency than DeepSeek-V4-Pro-Base across held-out evaluations.
- Trained natively on vision via an encoder-free VLM architecture, with multimodal data accounting for 18% of the 50T training tokens.
- Maintained an average 21% model FLOPs utilization normalized to 2.5 PFLOPS in BF16 and 89% goodput.
- Employs multi-round dropless MoE with static memory allocation and an optimized implementation of the Muon optimizer.
- Pre-trained entirely on in-house organic data without distillation for eventual licensing to enterprise customers via Foundry.

## Why it matters

Microsoft gains an in-house, frontier-scale multimodal base model to deploy across its product suite and license directly to enterprise customers.

## Sources

- [Microsoft AI: Frontier Pre-training at Microsoft AI | Microsoft AI](https://microsoft.ai/news/frontier-pre-training-at-microsoft-ai)

---

Summarized by dstilled on 2026-10-15. https://dstilled.ai/story/96e75333-0021-4319-89d6-de491dd30675