You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

Log in or Sign Up to review the conditions and access this model content.

AuriStream100M_10Pred_BigAudioDataset_500k-randinit

AuriStream is a speech language model by Greta Tuckute and Klemen Kotar.

This model predicts cochlear tokens from a tokenizer such as WavCochCausalV8192.

Untrained FP32 initialization for TuKoResearch/AuriStream100M_10Pred_BigAudioDataset_500k, using seed 11101994 and the exact frozen source used by its training run. The native model was initialized on CPU in the trainer initialization order, before conversion to Hugging Face. This is step zero, not a trained 500k checkpoint. The old seed-1110 counterpart is TuKoResearch/AuriStream100M_10Pred_BigAudioDataset_500k_seed1110-randinit.

Model Details

Parameter Value
Parameters ~0.15B
Layers 12
Hidden Size 768
Attention Heads 12
Vocab Size 8192
Prediction Steps 10

Usage

from transformers import AutoModel, AutoConfig

# Load with trust_remote_code for custom model
model = AutoModel.from_pretrained(
    "TuKoResearch/AuriStream100M_10Pred_BigAudioDataset_500k-randinit",
    trust_remote_code=True,
)

# Or load config first
config = AutoConfig.from_pretrained("TuKoResearch/AuriStream100M_10Pred_BigAudioDataset_500k-randinit", trust_remote_code=True)

Base Model Code

This checkpoint uses shared model code from TuKoResearch/AuriStream-base.

Tokenizer

This model uses cochlear tokens from WavCochCausalV8192.

Downloads last month
-
Safetensors
Model size
0.2B params
Tensor type
F32
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support