| --- |
| language: |
| - en |
| license: other |
| license_name: anvaya-model-license |
| license_link: LICENSE.md |
| tags: |
| - state-space-model |
| - custom-architecture |
| - causal-lm |
| - rtassm |
| pipeline_tag: text-generation |
| --- |
| |
| # Anavya-Rabbit-2.7B-0.5TG-beta |
|
|
| **RtaForge OPC Private Limited · ANVAYA Sovereign AI Research** |
|
|
| <img src="rabbit_slm.png" width="25%"/> |
|
|
| *Beta research release.* |
|
|
| --- |
|
|
| # Overview |
|
|
| | | | |
| | ---------------- | --------------------------------------- | |
| | **Architecture** | RtaSSM v7.4 ("Tungsten") | |
| | **Model Type** | State Space Language Model | |
| | **Parameters** | 2.7 Billion | |
| | **Layers** | 64 | |
| | **Hidden Size** | 2560 | |
| | **Tokenizer** | EleutherAI GPT-NeoX (50,280 vocabulary) | |
| | **Release** | 0.5TG-beta | |
| | **License** | [Anvaya Model License](LICENSE.md) — free for individual personal/research use; revenue-gated for organizational use; resale/bundling requires permission | |
|
|
| --- |
|
|
| # Summary |
|
|
| Rabbit 2.7B is the first public release built on the **Tungsten (RtaSSM v7.4)** architecture. |
|
|
| This release establishes the Tungsten training baseline for future TG releases and supersedes earlier prototype checkpoints. |
|
|
| Version 0.5TG-beta is intended as a stable engineering milestone and reference checkpoint for continued training and evaluation, not a finished, production-grade model. |
|
|
| --- |
|
|
| # Architecture |
|
|
| Rabbit belongs to the **RtaSSM** family of state-space language models developed by RtaForge. |
|
|
| The Tungsten architecture represents the current production training stack used within the ANVAYA research program and serves as the foundation for future Rabbit development. |
|
|
| --- |
|
|
| # Training |
|
|
| Training is performed using RtaForge's Rust-native training stack. |
|
|
| Current training features include: |
|
|
| * Tungsten training architecture |
| * Constitutional training validation |
| * Multi-phase curriculum training |
|
|
| Future releases continue curriculum expansion and instruction tuning. |
|
|
| --- |
|
|
| # Checkpoint |
|
|
| ``` |
| base/ |
| Anavya-Rabbit-2.7B-0.5TG-beta.pt |
| ``` |
|
|
| --- |
|
|
| # Context |
|
|
| Current release status: |
|
|
| | Capability | Status | |
| | ------------------- | ----------------------- | |
| | Training | ✅ | |
| | Continued Training | ✅ | |
| | 4K Context Training | ✅ Verified | |
| | 8K Context Training | Active engineering work | |
|
|
| --- |
|
|
| # Baseline Evaluation |
|
|
| The following results provide a baseline for future Tungsten releases. |
|
|
| | Task | Metric | Score | |
| | ------------- | ------------------- | ---------: | |
| | ARC Challenge | Accuracy | **0.3379** | |
| | ARC Challenge | Normalized Accuracy | **0.3703** | |
| | ARC Easy | Accuracy | **0.6616** | |
| | ARC Easy | Normalized Accuracy | **0.5833** | |
| | HellaSwag | Accuracy | **0.4583** | |
| | HellaSwag | Normalized Accuracy | **0.6063** | |
| | WinoGrande | Accuracy | **0.6164** | |
|
|
| Future releases will publish comparisons against previous Rabbit checkpoints and additional public benchmarks. |
|
|
| --- |
|
|
| # Historical Releases |
|
|
| Earlier Rabbit prototype checkpoints are retained for historical reference but are **not recommended** as starting points for continued training. |
|
|
| Tungsten (v0.5TG-beta) is the supported baseline for all future TG releases. |
|
|
| --- |
|
|
| # Roadmap |
|
|
| | Release | Focus | |
| | ---------- | ---------------------------------------------------------- | |
| | 0.5TG-beta | Tungsten baseline | |
| | 0.6TG | Reasoning + extended context | |
| | 0.7TG | Mathematics | |
| | 0.8TG | Science / STEM | |
| | 1.0 | Full curriculum + instruction tuning + expanded benchmarks | |
|
|
| --- |
|
|
| # License |
|
|
| This model is released under the [Anvaya Model License Agreement](LICENSE.md). In brief: |
|
|
| * **Free** for an individual's own personal use or independent research. |
| * **Organizational use** (by or for any company, non-profit, or other entity, regardless of |
| revenue) requires either staying under the revenue threshold (INR 1 crore / USD 100,000 |
| per year, whichever is lower) or an Enterprise License. |
| * **Resale, paid bundling, or hosted API access** to the model or a derivative always |
| requires RtaForge's prior written permission, regardless of revenue. |
| * **Derivatives and fine-tunes** must credit both Anvaya-Rabbit-2.7B and RtaForge OPC |
| Private Limited by name. |
| * **No content safeguards are mandated** — you are solely responsible for what you generate |
| and how you deploy it. |
|
|
| Read the full [LICENSE.md](LICENSE.md) for the binding terms — this summary is for |
| convenience only and does not override it. |
|
|
| --- |
|
|
| # About ANVAYA |
|
|
| ANVAYA is RtaForge's research program exploring sovereign state-space language models, Rust-native AI infrastructure, and deployable intelligence systems. |
|
|
| Development emphasizes reproducible engineering, inspectable mechanisms, and continual architectural refinement. |
|
|
| --- |
|
|
| © RtaForge OPC Private Limited |
|
|
