Official checkpoint contains NaN/Inf tensors and cannot generate valid text

#1
by liangxian2 - opened

Hello, I tested the official BillyLiang/BaciCausalLM checkpoint at revision ff257c4398d57cb27ebce41ee0a0f0fca0bd7ba0.

All four downloaded shards exactly match the SHA-256 hashes recorded by Hugging Face, so this does not appear to be a download or mirror-corruption issue.

A scan of the official safetensors files found:

  • 109 floating-point tensors containing NaN or Inf
  • 757,110 non-finite elements in total
  • 585,212 non-finite elements in lm_head.weight
  • affected layers concentrated in layers 53–63, the final normalization layer and the language-model head
  • the first three shards are finite; the abnormalities are concentrated in shard 4

Reproduction results:

  • The Transformers README example loads the model, but model.generate() fails with:
    RuntimeError: probability tensor contains either inf, nan or element < 0
  • With vLLM and the Marlin W4A16 kernel on an RTX 6000 Ada 48GB, the model loads, but first-token logits are NaN and outputs degenerate into repetitions of lock.
  • Transformers 4.57.6 on a 24GB A30 loads the packed weights at approximately 17.9 GiB, but generation triggers decompression and CUDA OOM.

Full environment details, shard hashes and reproducibility questions are documented here:

https://github.com/SiyiPeng66/BaciCausalLM/issues/1

Could the uploader please confirm whether this is the checkpoint used in the paper and re-export or replace the affected checkpoint, especially shard 4?

Thank you.

Sign up or log in to comment