A 501B Model Fits on Four GPUs, and One Long Context Costs 36 GiB

Reflection published the weights for a 501 billion parameter model on Friday under Apache 2.0. The parameter count is the headline. The memory arithmetic is the story.

• NVFP4 cuts the checkpoint from about 1 TB to roughly 280 GB, which fits on four B300 GPUs.
• Hybrid attention holds the cache at 36 KiB per token, so a single 1M context reserves 36.08 GiB.
• On that four GPU node roughly 19 full length contexts fit at once, against about 3 if every layer kept full history.
• 23 billion parameters are active per token, so compute behaves like a much smaller model while memory behaves like a 501B one.

Day one support landed in Transformers, vLLM, SGLang, Unsloth and NVIDIA Dynamo, with a DFlash2 speculative drafter alongside. The checkpoint arrives with a serving path rather than a research notebook.

We ran the capacity math on the published figures. Full analysis here.