One thing that keeps sticking in my head is not the alleged 7.7 TB size, but how clean the shard layout is.
The repo shows 772 shards that are all packed to almost exactly 10 GB each. That's not impossible, but it feels unusually tidy for something that supposedly came out of a massive training run.
Maybe someone with more experience handling frontier-scale checkpoints can explain it.
My real evaluation is that real training artifacts tend to accumulate asymmetries:
embeddings are often disproportionately large,
routing layers and auxiliary tensors are tiny,
expert populations don't all evolve identically,
some experts become specialists while others remain generalists,
tensor boundaries rarely line up perfectly with storage boundaries,
exports usually end up with odd leftovers, partially filled shards, and other "real world" messiness.
In other words, I'd expect a distribution that looks more like:
Plain Text
9.2 GB
10.4 GB
8.7 GB
10.5 GB
7.1 GB
etc...
rather than hundreds upon hundreds of files that all appear engineered to land at essentially the same target size.
This isn't an authenticity claim. Modern exporters may intentionally pack shards to a fixed size limit and maybe this is exactly what we'd expect from current tooling.
But it reminds me of an old engineering rule:
Real systems are usually messy.
That's why the near-perfect 10 GB packing caught my attention more than the headline parameter count.
For people who've actually worked with multi-hundred-billion or trillion-parameter checkpoints, is this level of uniformity normal now, or does it stand out to you too?
comparing the shard layout to Kimi-K3’s layout I did notice that Kimi has a similar clean structure (16.6GB, 17GB, 17GB, 17GB, 16.6GB,...), though that’s from the layers for each shard, which is a different packing scheme from a plain size cap. Both end up looking tidy, though. save_pretrained / huggingface_hub fill each shard greedily up to max_shard_size, so with thousands of similar-sized expert tensors almost every shard lands just under the limit. Per-layer packing gives near-identical shards too, as long as the layers are the same size. Uniform shards are what you’d expect from basically any modern exporter, so I don’t think they count for or against authenticity. The one place a size cap usually shows is the last shard, and sure enough 00772 is 5.77GB vs 10GB for the rest, which is a normal leftover. So the layout looks like a standard exporter’s output, and it tells us nothing either way. The actual tensor contents are what matter.