---
license: apache-2.0
language:
- en
pipeline_tag: text-generation
tags:
- geocentric
- arc
- causal-lm
- small-language-model
---

# Geocentric Arc

Arc is a small, experimental English language model from [Geocentric](https://geocentricai.com), trained from scratch. It is built for co-writing: exploring phrasing, drafting loose story material and finding fragments to develop in your own voice. It runs on your own machine, and nothing you write is sent to us.

**Arc is experimental.** It is not ready for dependable factual assistance, arithmetic or autonomous work. Its full evaluation, including the responses it gets wrong, is on the [model card](https://geocentricai.com/models/arc/).

## Model details

| | |
|---|---|
| Developer | Geocentric |
| Parameters | 120,787,072 |
| Architecture | Decoder-only transformer: RMSNorm, rotary position embeddings, grouped-query attention, SwiGLU feed-forward, tied embeddings |
| Layers x width | 11 x 896 |
| Attention heads | 14 (2 key/value heads) |
| Context length | 1,024 tokens |
| Vocabulary | 32,000 (byte-level BPE) |
| Stage | Supervised fine-tuned (SFT) |
| Modality | Text only |
| Language | English |
| License | Apache License 2.0 |

## Evaluation

Scored with PARALLAX 1.0, the evaluation suite we built for our own models. The Parallax Index is **53.5 / 100**, and it reproduced on a second machine (Apple M4).

| Probe | Score | Measured |
|---|---:|---|
| ZENITH | 22.7 | 2.042 bits/byte on held-out text |
| MERIDIAN | 100.0 | +0.447 nats gained from context |
| SEXTANT | 26.7 | 18/40 correct (chance is 25%) |
| ASTROLABE | 58.3 | 7/12 instructions followed |
| NADIR | 100.0 | 0 of 6 generations looped |

PARALLAX is a small, purpose-built suite and not a standardized benchmark. On Geocentric's published diagnostic prompts Arc passed 1 of 24, partially passed 4 and failed 19. The unedited responses are on the [model card](https://geocentricai.com/models/arc/#evaluation).

## Files

| File | What it is |
|---|---|
| `geocentric_sft_best.pt` | The SFT checkpoint (weights and training state needed to load it) |
| `config.json` | Model configuration |
| `tokenizer.json` | The 32,000-token tokenizer |

## How to use it

The checkpoint is on Ollama and can be ran there by running the following command

```bash
ollama run GeocentricAI/Geocentric-Arc-1
```

## Intended use

Co-writing, experimentation, and research on small models and their evaluation.

## Limitations

- Arc invents facts when information is missing, and it failed all four arithmetic cases in our diagnostic suite.
- It does not reliably follow requested length, format or premise.
- Its 1,024-token context is short. Long documents will not fit.
- It reflects its training data, including that data's biases. Do not use it as a source of facts, for anything safety-critical, or unsupervised in front of users.
- Generated code can look correct and be broken. Review and test it.

## Training data

Arc was trained on public datasets, not on data we bought, and no user conversations were used. Dataset documentation is on the [Training Data Transparency](https://geocentricai.com/legal/training-data/) page.

## License

Apache License 2.0. See `LICENSE`.

## Contact

contact@geocentricai.com