README.md

---
language:
  - en
license: other
license_name: anvaya-model-license
license_link: LICENSE.md
tags:
  - state-space-model
  - custom-architecture
  - causal-lm
  - rtassm
pipeline_tag: text-generation
---

# Anavya-Rabbit-2.7B-0.5TG-beta

**RtaForge OPC Private Limited · ANVAYA Sovereign AI Research**

<img src="rabbit_slm.png" width="25%"/>

*Beta research release.*

---

# Overview

|                  |                                         |
| ---------------- | --------------------------------------- |
| **Architecture** | RtaSSM v7.4 ("Tungsten")                |
| **Model Type**   | State Space Language Model              |
| **Parameters**   | 2.7 Billion                             |
| **Layers**       | 64                                      |
| **Hidden Size**  | 2560                                    |
| **Tokenizer**    | EleutherAI GPT-NeoX (50,280 vocabulary) |
| **Release**      | 0.5TG-beta                              |
| **License**      | [Anvaya Model License](LICENSE.md) — free for individual personal/research use; revenue-gated for organizational use; resale/bundling requires permission |

---

# Summary

Rabbit 2.7B is the first public release built on the **Tungsten (RtaSSM v7.4)** architecture.

This release establishes the Tungsten training baseline for future TG releases and supersedes earlier prototype checkpoints.

Version 0.5TG-beta is intended as a stable engineering milestone and reference checkpoint for continued training and evaluation, not a finished, production-grade model.

---

# Architecture

Rabbit belongs to the **RtaSSM** family of state-space language models developed by RtaForge.

The Tungsten architecture represents the current production training stack used within the ANVAYA research program and serves as the foundation for future Rabbit development.

---

# Training

Training is performed using RtaForge's Rust-native training stack.

Current training features include:

* Tungsten training architecture
* Constitutional training validation
* Multi-phase curriculum training

Future releases continue curriculum expansion and instruction tuning.

---

# Checkpoint

```
base/
    Anavya-Rabbit-2.7B-0.5TG-beta.pt
```

---

# Context

Current release status:

| Capability          | Status                  |
| ------------------- | ----------------------- |
| Training            | ✅                       |
| Continued Training  | ✅                       |
| 4K Context Training | ✅ Verified              |
| 8K Context Training | Active engineering work |

---

# Baseline Evaluation

The following results provide a baseline for future Tungsten releases.

| Task          | Metric              |      Score |
| ------------- | ------------------- | ---------: |
| ARC Challenge | Accuracy            | **0.3379** |
| ARC Challenge | Normalized Accuracy | **0.3703** |
| ARC Easy      | Accuracy            | **0.6616** |
| ARC Easy      | Normalized Accuracy | **0.5833** |
| HellaSwag     | Accuracy            | **0.4583** |
| HellaSwag     | Normalized Accuracy | **0.6063** |
| WinoGrande    | Accuracy            | **0.6164** |

Future releases will publish comparisons against previous Rabbit checkpoints and additional public benchmarks.

---

# Historical Releases

Earlier Rabbit prototype checkpoints are retained for historical reference but are **not recommended** as starting points for continued training.

Tungsten (v0.5TG-beta) is the supported baseline for all future TG releases.

---

# Roadmap

| Release    | Focus                                                      |
| ---------- | ---------------------------------------------------------- |
| 0.5TG-beta | Tungsten baseline                                          |
| 0.6TG      | Reasoning + extended context                               |
| 0.7TG      | Mathematics                                                |
| 0.8TG      | Science / STEM                                             |
| 1.0        | Full curriculum + instruction tuning + expanded benchmarks |

---

# License

This model is released under the [Anvaya Model License Agreement](LICENSE.md). In brief:

* **Free** for an individual's own personal use or independent research.
* **Organizational use** (by or for any company, non-profit, or other entity, regardless of
  revenue) requires either staying under the revenue threshold (INR 1 crore / USD 100,000
  per year, whichever is lower) or an Enterprise License.
* **Resale, paid bundling, or hosted API access** to the model or a derivative always
  requires RtaForge's prior written permission, regardless of revenue.
* **Derivatives and fine-tunes** must credit both Anvaya-Rabbit-2.7B and RtaForge OPC
  Private Limited by name.
* **No content safeguards are mandated** — you are solely responsible for what you generate
  and how you deploy it.

Read the full [LICENSE.md](LICENSE.md) for the binding terms — this summary is for
convenience only and does not override it.

---

# About ANVAYA

ANVAYA is RtaForge's research program exploring sovereign state-space language models, Rust-native AI infrastructure, and deployable intelligence systems.

Development emphasizes reproducible engineering, inspectable mechanisms, and continual architectural refinement.

---

© RtaForge OPC Private Limited