| --- |
| library_name: transformers |
| pipeline_tag: text-generation |
| language: |
| - en |
| tags: |
| - qwen3_5 |
| - pretraining |
| --- |
| |
| # AnonymousModel-mht |
|
|
| Inference-only release of 20 budget-boundary checkpoints from two matched, |
| randomly initialized Qwen3.5-4B text-model pretraining runs on approximately |
| 20B body tokens. Each family contains ten approximately 2B-spaced checkpoints |
| from 2B through 20B. The additional final checkpoints at approximately 20.156B |
| tokens are intentionally excluded from this release. |
|
|
| - [Keen-qwen4b-20b](Keen-qwen4b-20b/README.md): factual-list-conditioned pretraining. |
| - [Vanilla-qwen4b-20b](Vanilla-qwen4b-20b/README.md): ordinary text pretraining. |
|
|
| The training context was 4,096 tokens. These are base models, not QA-SFT or |
| instruction-tuned models. Each checkpoint directory is independently loadable |
| with standard Transformers 5.6.2 and contains BF16 safetensors weights, |
| model/generation configuration and a tokenizer. Select the checkpoint using |
| `subfolder`; this repository root is a collection and has no root model weights. |
| Optimizer states, scheduler states and training logs are not distributed. |
|
|
| Checkpoint loading examples and exact token counts appear in the family and |
| checkpoint READMEs. `checkpoints.json` indexes the release. `manifest.json` |
| records artifact hashes. Configuration enables inference caching; the released |
| weights and tokenizer vocabulary are unchanged from their training checkpoints. |
|
|
