Official checkpoint contains NaN/Inf tensors and cannot generate valid text
#1
by liangxian2 - opened
Hello, I tested the official BillyLiang/BaciCausalLM checkpoint at revision ff257c4398d57cb27ebce41ee0a0f0fca0bd7ba0.
All four downloaded shards exactly match the SHA-256 hashes recorded by Hugging Face, so this does not appear to be a download or mirror-corruption issue.
A scan of the official safetensors files found:
- 109 floating-point tensors containing NaN or Inf
- 757,110 non-finite elements in total
- 585,212 non-finite elements in
lm_head.weight - affected layers concentrated in layers 53β63, the final normalization layer and the language-model head
- the first three shards are finite; the abnormalities are concentrated in shard 4
Reproduction results:
- The Transformers README example loads the model, but
model.generate()fails with:RuntimeError: probability tensor contains either inf, nan or element < 0 - With vLLM and the Marlin W4A16 kernel on an RTX 6000 Ada 48GB, the model loads, but first-token logits are NaN and outputs degenerate into repetitions of
lock. - Transformers 4.57.6 on a 24GB A30 loads the packed weights at approximately 17.9 GiB, but generation triggers decompression and CUDA OOM.
Full environment details, shard hashes and reproducibility questions are documented here:
https://github.com/SiyiPeng66/BaciCausalLM/issues/1
Could the uploader please confirm whether this is the checkpoint used in the paper and re-export or replace the affected checkpoint, especially shard 4?
Thank you.