README.md

---
license: apache-2.0
license_link: https://huggingface.co/MTEnt/dot/blob/main/LICENSE
base_model:
  - Qwen/Qwen3.5-9B
tags:
  - dot
  - recurrent-depth
  - reasoning
  - text-generation
  - transformers
inference: false
---

# Dot

Dot is an experimental 9.8B-parameter text reasoning model built by MTEnt. This
repository contains the complete BF16 Dot backbone, its separately trained
recurrent-depth core, the exact loader, and measured release evidence.

The current stable release is **Dot v0.4 Thinking**. The newest verified
checkpoint is **Dot v0.5 Spatial Process — Stage 1**. The name and runtime
identity are Dot.

## Dot v0.5 Spatial Process — Stage 1

This update adds the accepted recurrent core and repaired backbone layers from
the spatial-process recovery track. It is published as a verified intermediate
checkpoint while the longer Stage 2 process run continues.

| Held-out generation gate | Result |
| --- | ---: |
| Valid thinking envelopes, 256 spatial primitives | 100% |
| Exact spatial answers | 99.609% |
| Verified one-step transition accuracy | 99.609% |
| Valid thinking envelopes, 256 non-spatial cases | 100% |
| Non-spatial exact-answer retention | 98.047% |

The checkpoint is at
[`checkpoints/v0.5-spatial-process-stage1/`](checkpoints/v0.5-spatial-process-stage1/).
It contains the recurrent core plus repaired backbone layers 30 and 31 and the
final norm. The four unchanged BF16 backbone shards remain the base artifact.

Local mixed-precision integration used the unchanged backbone in NF4 and kept
the repaired layers and recurrent core in BF16. It allocated 10.01 GB of CUDA
memory after load on a 16 GB GPU, with CPU fallback disabled.

Two official ARC-AGI-3 development fixtures exercised the complete local model,
action gate, SQLite ledger, and replay path. Neither fixture was solved. This is
integration evidence, not an ARC-AGI-3 score. The observed failures and the next
training curriculum are documented in
[`docs/v0.5-local-harness-findings.md`](docs/v0.5-local-harness-findings.md).

## What changed in this release

Dot inserts a weight-tied recurrent-depth core after decoder layer 15. The core
copies native layers 12 through 15 and runs them four times through learned
residual gates.

| Component | Parameters |
| --- | ---: |
| Dot backbone | 8,953,803,264 |
| Recurrent-depth core | 864,945,224 |
| Total instantiated model | 9,818,748,488 |

The v0.4 repair trained the recurrent core for 938 optimizer steps and processed
8,642,015 tokens from 60,000 generated reasoning records. The backbone stayed
frozen during this stage. Final active gate values were `0.157696`, `0.079852`,
`0.059039`, and `0.059151`.

## Measured results

These are narrow internal measurements, not general model benchmarks.

On 4,096 held-out prompts from the same eight executable task generators used
to construct the curriculum:

| Teacher-forced metric | Zero-gate baseline | Dot v0.4 |
| --- | ---: | ---: |
| Response NLL | 2.128394 | 0.017367 |
| Response token accuracy | 66.60% | 99.43% |
| Exact response rate | 0.00% | 87.77% |
| Reasoning token accuracy | 60.47% | 99.32% |
| Final-answer token accuracy | 99.61% | 99.99% |

On 256 separately seeded free-running prompts from those same task families,
Dot produced a valid, non-empty thinking envelope in 100% of cases and matched
the exact final answer in 248/256 cases (96.875%). No response hit the 256-token
generation limit.

An adversarial spatial generalization probe was much weaker: Dot v0.4 scored
14/64 (21.875%) exact match on a newer spatial suite, including 0% on its
orientation subset. That failure is why v0.4 should not be described as an
ARC-AGI-capable model. A spatial repair is being evaluated separately and is not
part of this stable release.

The machine-readable release metrics are in
[`eval/thinking-v0.4.json`](eval/thinking-v0.4.json).

## Run Dot

This is a custom architecture. A normal `AutoModelForCausalLM.from_pretrained`
call loads only the backbone and silently omits Dot's trained recurrent core.
Use the included loader.

```bash
git clone https://huggingface.co/MTEnt/dot
cd dot
python -m pip install .
python examples/chat.py --model . --prompt "Explain why a passing build does not prove the UI works."
```

To run the v0.5 Stage 1 checkpoint in BF16:

```bash
python examples/chat.py \
  --model . \
  --checkpoint checkpoints/v0.5-spatial-process-stage1 \
  --prompt "Track a state change and explain which evidence determines the result."
```

The verified runtime used Python 3.12, PyTorch 2.8.0, Transformers 5.15.0,
Safetensors 0.8.0, BF16, SDPA, and an NVIDIA H200. Other hardware and precision
paths have not been verified for this release.

Cache-backed decoding is deliberately disabled. The recurrent passes do not yet
have correct cache ownership, so enabling a normal KV cache would risk silently
wrong state. Generation recomputes the sequence at every token and is therefore
slow.

## Files that matter

- `model-00001-of-00004.safetensors` through
  `model-00004-of-00004.safetensors`: the complete Dot v0.2 semantic backbone.
- `reasoning_core.safetensors`: the v0.4 recurrent-depth weights.
- `checkpoints/v0.5-spatial-process-stage1/`: the v0.5 Stage 1 recurrent core
  and repaired backbone delta, with its integrity manifest.
- `dot_recurrent_manifest.json`: architecture, source step, metrics, and core
  checksum.
- `dot_rd/`: the required architecture and integrity-checking loader.
- `release.json`: release lineage, hashes, training scope, and known limits.

## Scope and limitations

- Dot v0.4 is a text-only research release.
- The reported reasoning score measures the same generator families used for
  training, with separate seeds and hash-disjoint records. It does not prove
  broad reasoning, coding, world knowledge, ARC-AGI, or safety performance.
- No independent safety, bias, multilingual, coding, or production-agent audit
  has been completed for v0.4.
- Thinking responses can expose intermediate text. Do not put secrets in a
  prompt and assume the reasoning channel will conceal them.
- The BF16 package is roughly 20 GB before runtime allocations. Only the H200
  path described above is verified.

## Technical lineage and license

Dot's semantic backbone was initialized from `Qwen/Qwen3.5-9B` and then modified
by MTEnt through a merged semantic LoRA stage and the recurrent-depth training
described here. The original training manifest recorded the source repository
but did not record its exact commit hash. The complete modified backbone is
included so this release does not depend on reconstructing that missing adapter.

The upstream work and this repository are distributed under Apache License 2.0.
See [`LICENSE`](LICENSE), [`NOTICE`](NOTICE), and [`release.json`](release.json)
for attribution and provenance. This model is provided as-is, without a warranty
of correctness, safety, or fitness for a particular purpose.