|
Download README.md from WithinUsAI/Bounty.Hunter.47m: direct link, hf CLI and curl.
- Browser
- Download file 1.88 kB
-
https://huggingface.co/WithinUsAI/Bounty.Hunter.47m/resolve/main/README.md
- Command line
-
hf download hf://WithinUsAI/Bounty.Hunter.47m/README.md
-
curl -L -H "Authorization: Bearer $HF_TOKEN" -o README.md https://huggingface.co/WithinUsAI/Bounty.Hunter.47m/resolve/main/README.md
1.88 kB
| license: mit | |
| library_name: pytorch | |
| tags: | |
| - causal-lm | |
| - decoder-only | |
| - reasoning | |
| - pretraining | |
| - think | |
| - thought | |
| - chain-of-thought | |
| # Bounty.Hunter.47m | |
| A decoder-only causal language model trained from scratch, built around a | |
| deep chain-of-thought reasoning core. | |
| ## Architecture | |
| - Parameters: 46,912,320 | |
| - Context length: 1096 | |
| - Hidden size: 672 | |
| - Layers: 8 | |
| - Attention heads: 8 | |
| - Intermediate size: 2688 | |
| - Vocabulary size: 4,096 | |
| ## Reasoning tokens | |
| - `<|input|>` | |
| - `<|think|>` | |
| - `<|thought|>` | |
| - `<|reasoning|>` | |
| - `<|answer|>` | |
| These are first-class tokens in the tokenizer and are woven into every | |
| pretraining example's text structure, not added after the fact. | |
| ## Objective | |
| True causal next-token prediction. | |
| ## Context | |
| Every individual training example is strictly limited to | |
| `1095` content tokens + EOS, padded to exactly `1096` | |
| positions. No oversized example is split across separate training examples. | |
| ## Dataset | |
| `Plans11/Organized_PreTrain_1k_Context`, restricted to: | |
| - Think | |
| - Thought | |
| - Reasoning | |
| (Chat / Instruct / Code_Instruct / Tool_Calling shards are intentionally | |
| excluded from this model's pretraining corpus.) | |
| ## Example selection | |
| New examples are selected each session via a deterministic seeded shuffle | |
| of the restricted dataset, with SHA-256 content-hash deduplication against | |
| every previously trained example. No example is ever trained twice. | |
| ## Resumability | |
| Training state is persisted to Hugging Face, including: | |
| - model.safetensors | |
| - training_state.pt | |
| - tokenizer files | |
| - config.json | |
| - progress.json | |
| - seen_examples.jsonl | |
| - training_metadata.json | |
| The tokenizer becomes immutable after its initial creation. | |
| ## Current progress | |
| - Unique examples trained: 20,000 | |
| - Global optimizer steps: 625 | |
| - Last session: 20,000 | |
| This is an experimental language model and is not guaranteed to produce | |
| factually or logically correct outputs. | |