Upload AuriStream checkpoint

README.md

@@ -14,7 +14,7 @@ library_name: transformers
 
 This model predicts cochlear tokens from a tokenizer such as [WavCochCausalV8192](https://huggingface.co/TuKoResearch/WavCochCausalV8192).
 
-This repository contains a freshly initialized `AuriStream100M20PredConfig` model. The weights are random and have not been trained from a checkpoint. The model was instantiated through the native training model with seed `1110` before conversion to the Hugging Face format.
+Native training step-zero initialization for the 100M 20-prediction model. This exactly uses origin seed `11101994` and historical source commit `fad735c90bd44b4bcfd6bdf9429edbb640d2ebf5`, matching the start of [W&B run 3g8jxx2k](https://wandb.ai/NaudioGPT/auristream/runs/3g8jxx2k). The weights are untrained FP32 values produced before XLA/FSDP wrapping.
 
 ## Model Details
 

config.json

@@ -8,7 +8,7 @@
     "AutoModel": "TuKoResearch/AuriStream-base--modeling_auristream.AuriStreamModel",
     "AutoModelForCausalLM": "TuKoResearch/AuriStream-base--modeling_auristream.AuriStreamModel"
   },
-  "torch_dtype": "bfloat16",
+  "torch_dtype": "float32",
   "transformers_version": "4.40.0",
   "vocab_size": 8192,
   "n_embd": 768,
@@ -18,6 +18,8 @@
   "dropout": 0.0,
   "bias": false,
   "input_conv_kernel_size": 0,
-  "random_init_seed": 1110,
-  "random_init_source": "auristream.auristream.model.AuriStream"
+  "random_init_seed": 11101994,
+  "random_init_source": "native-training-step-zero",
+  "random_init_source_commit": "fad735c90bd44b4bcfd6bdf9429edbb640d2ebf5",
+  "random_init_checkpoint_iteration": 0
 }

model.safetensors

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:adeb9808a1ad4578544bcbece627eddb2a9f991091f20a25f229f1e5a9add662
-size 434159608
+oid sha256:d2d67219043868773f06dfcb001a680333e05260a05e0c8929ac33ca73f711e1
+size 868309144