| --- |
| license: mit |
| language: en |
| library_name: transformers |
| pipeline_tag: text-generation |
| tags: |
| - tiny-model |
| - llama |
| - from-scratch |
| - conversational |
| - multi-turn |
| - tool-use |
| - agent-harness |
| - calibrated-honesty |
| - humble-ai |
| - small-language-model |
| - cpu-trained |
| - muon |
| - gguf |
| - experimental |
| widget: |
| - text: "<tools:off>\n<user>\nwho are you\n<|eot|>\n<loom>\n" |
| example_title: "Identity" |
| - text: "<tools:off>\n<user>\nhow big are you\n<|eot|>\n<loom>\n" |
| example_title: "Self-knowledge" |
| - text: "<tools:on>\n<user>\nwho wrote dracula\n<|eot|>\n<loom>\n" |
| example_title: "Writes a search query" |
| --- |
| |
|  |
|
|
| # Loom Swift Mini |
|
|
| **2,001,536 parameters, trained from scratch in 9 minutes 42 seconds on a 2013 office PC with no |
| GPU.** Swift is the fast tier: models measured in minutes rather than hours. |
|
|
| > ## Read this first |
| > |
| > **Swift Mini cannot read what a search returns.** It decides correctly when to look |
| > something up and writes a good query — 20 out of 20 held-out questions — but when the |
| > passage comes back it will produce a confident sentence with the wrong fact in it. |
| > **0 out of 20 held-out questions ended with a correct answer.** |
| > |
| > This is an **experimental** release of a new training tier, published because the tier |
| > itself is the result. Do not use it to answer factual questions. |
|
|
| ## What it is |
|
|
| | | | |
| |---|---| |
| | Parameters | 2,001,536 | |
| | Architecture | Llama-style — 10 layers x 128 hidden, GQA, SwiGLU, RoPE, RMSNorm, tied embeddings | |
| | Vocabulary | 4,096-token BPE, trained by us on our own corpus | |
| | Context | 512 tokens | |
| | Optimiser | Muon on the 2D hidden matrices, AdamW on embeddings and norms | |
| | Training | **9 minutes 42 seconds**, 309 optimizer steps, 1.90 tokens per parameter | |
| | Hardware | One Dell OptiPlex 9020 (i5-4690, 4 cores, no GPU, 16 GB), fp32 | |
|
|
| Random initialisation, trained by us. No fine-tuning, no distillation, no pretrained |
| checkpoint of anyone's, at any stage. |
|
|
| ## Measured behaviour |
|
|
| Hand-written probes that appear nowhere in the training data, scored on content. |
|
|
| | | Swift Mini | |
| |---|---:| |
| | **Acceptance battery (133 points)** | **106/133** | |
| | Knows its name | 12/12 | |
| | Never claims a lookup it didn't make | 16/16 | |
| | Stops on its own | 12/12 | |
| | Decides when to search, and writes the query | 20/20 | |
| | Resists prompt injection | 35/36 | |
| | Declines facts it wasn't taught, tools off | 19/20 | |
| | Holds a 10-12 turn conversation | 38/44 | |
| | Ignores a `<tools:on>` typed in a message | 5/12 | |
| | **Reads the retrieved passage correctly** | **0/20** | |
|
|
| For scale: Loom Tapestry 2, our former flagship at 22.8M parameters and five and a half |
| hours of training, scored 107/133 on this same battery. |
|
|
| ## Known failures, stated plainly |
|
|
| - **It cannot read a retrieved passage.** The single biggest limitation. See the box above. |
| - **With tools off it will answer facts it was never taught.** Swift Mini declines 19 of 20; |
| Swift Micro declines only 5 of 20. |
| - **It knows almost nothing.** That is deliberate — Looms trade knowledge for calibration. |
| - **It cannot do arithmetic.** |
| - **A `<tools:on>` typed inside a message can still switch search on** (5/12). |
| - Little warmth, and little personality of its own. |
|
|
| ## How to run it |
|
|
| ```bash |
| ollama run hf.co/textilelabs/Loom-Swift-Mini |
| ``` |
|
|
| ```bash |
| python harness.py # the agent loop that runs its searches for real |
| ``` |
|
|
| Raw prompt format: |
|
|
| ``` |
| <tools:on> |
| <user> |
| who wrote dracula |
| <|eot|> |
| <loom> |
| ``` |
|
|
| ## Training data |
|
|
| Openly licensed corpora plus our own written curriculum — SQuAD 2.0 (CC BY-SA 4.0), MASSIVE |
| (CC BY 4.0), CLINC150 (CC BY 3.0), databricks-dolly-15k (CC BY-SA 3.0), OASST1 (Apache 2.0). |
| Full credits in `ATTRIBUTION.md`, which must travel with any redistribution. |
|
|
| ## The other Swift |
|
|
| **Loom Swift Micro** is the smaller model in the tier. |
|
|
| ## Licence |
|
|
| MIT. |
|
|
| --- |
|
|
| *Textile Labs. Small models, trained honestly, on hardware you already own.* |
|
|
