Instructions to use Refract-Labs/Orion-Flagship-Mini-Base with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use Refract-Labs/Orion-Flagship-Mini-Base with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="Refract-Labs/Orion-Flagship-Mini-Base", trust_remote_code=True)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("Refract-Labs/Orion-Flagship-Mini-Base", trust_remote_code=True, device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use Refract-Labs/Orion-Flagship-Mini-Base with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "Refract-Labs/Orion-Flagship-Mini-Base"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Refract-Labs/Orion-Flagship-Mini-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker

docker model run hf.co/Refract-Labs/Orion-Flagship-Mini-Base

SGLang

How to use Refract-Labs/Orion-Flagship-Mini-Base with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "Refract-Labs/Orion-Flagship-Mini-Base" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Refract-Labs/Orion-Flagship-Mini-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "Refract-Labs/Orion-Flagship-Mini-Base" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "Refract-Labs/Orion-Flagship-Mini-Base",
		"prompt": "Once upon a time,",
		"max_tokens": 512,
		"temperature": 0.5
	}'

Docker Model Runner
How to use Refract-Labs/Orion-Flagship-Mini-Base with Docker Model Runner:
```
docker model run hf.co/Refract-Labs/Orion-Flagship-Mini-Base
```

Orion-Flagship-Mini-Base / README.md

Bc-AI

Update README.md

243f531 verified 28 days ago

preview code

Raw History Blame Contribute Delete

6.31 kB

---
library_name: transformers
pipeline_tag: text-generation
license: other
license_name: orion-t2-research
tags:
- project-prism
- orion
- custom-code
- causal-lm
---
# 🔺 Orion Flagship

**Orion Flagship** is an experimental Project Prism model using the Orion T2
architecture.

T2 separates parts of the computation into a Universal Cortex, shared sparse
Procedure Banks, a sparse Knowledge Vault, persistent Working State, and a
cheap recurrent deliberation pass.

This repository contains an experimental pretraining checkpoint and custom
Transformers integration. It is **not instruction tuned**.

## ⚠️ Access & Usage Restrictions

- **No use of the Orion T2 architecture for your own models.** The
  architecture, its code, and its design are not licensed for building,
  training, or deriving other models. You can howvever make your own models and publish privately under this organisation.
- **Private repo only.** This repository must stay private under the
  Project Prism organization. It is for Project Prism members only.
- **No outsider access (for now).** Do not share, mirror, fork, or
  redistribute the weights or code outside Project Prism.

```python
from transformers import AutoTokenizer, AutoModelForCausalLM
repo = "Project-Prism/Orion-Flagship-Mini-Preview"
tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
model = AutoModelForCausalLM.from_pretrained(
    repo,
    trust_remote_code=True,
    device_map="auto",
)
```

The current T2 v1 compatibility layer deliberately uses `use_cache=False`.
Generation therefore recomputes the full prefix each token so the recurrent
deliberation + Working State graph remains faithful to training.

## 🧠 Architecture

![Orion T2 Architecture](./assets/orion-t2-architecture.png)

### Transformer 2 (T2)

**Transformer 2 (T2)** is an experimental language-model architecture developed
under Project Prism.

The central idea behind T2 is to separate several roles that are normally
entangled inside the parameters and hidden activations of a conventional
Transformer.

Instead of relying on a single dense stack for knowledge storage, reusable
computation, and temporary reasoning state, T2 introduces dedicated systems
for each role:

**KNOW → DO → THINK → VERIFY**

### 🧠 Universal Cortex

The **Universal Cortex** is the main sequence-processing backbone of Orion.

Mini-T2 uses **12 Cortex stages** with a hidden dimension of **768**. Each
stage contains attention and a routed Procedure Bank feed-forward path, using
12 query heads and 4 KV heads.

The Cortex is responsible for integrating:

- token context,
- retrieved knowledge,
- routed procedures,
- persistent Working State,
- and recurrent deliberation.

Unlike a standard Transformer where every layer owns an independent FFN,
groups of Cortex stages access shared Procedure Banks.

### ⚙️ Procedure Banks

T2 separates reusable transformations into **shared sparse Procedure Banks**.

Mini-T2 contains:

- **3 shared Procedure Banks**
- **16 experts per bank**
- **Top-2 routed experts**
- **1 shared expert**

Each bank is reused across a group of Cortex stages. A sparse router selects
the most relevant procedures for each token while the shared expert provides
a common computation path.

The goal is to let the model learn reusable computational procedures without
requiring every Cortex stage to independently store its own complete set of
feed-forward transformations.

### 📚 Knowledge Vault

The **Knowledge Vault** provides a dedicated sparse memory system for
declarative information.

Mini-T2 contains **65,536 memory slots** and uses product-key retrieval.

For each retrieval:

1. The query searches the two product-key components.
2. The top **12 candidates from each side** form **144 candidate addresses**.
3. The best **4 memories** are retrieved.
4. Retrieved information is gated into the Cortex.

Knowledge interfaces are used at approximately **stages 3, 7, and 11**.

This creates a separate pathway through which the model can retrieve stored
information instead of requiring all declarative knowledge to be represented
only through the main computational weights.

### 📝 Working State

T2 also maintains a separate **persistent causal Working State**.

Working State acts as temporary computational memory. It can be read by the
Cortex and updated as the sequence is processed.

Updates use a learned gated replacement/blending mechanism rather than
unbounded additive accumulation.

In Mini-T2, Working State interacts with the Cortex around the same major
interface points used by the Knowledge Vault.

This gives T2 an explicit distinction between:

- **stored knowledge** — Knowledge Vault,
- **reusable computation** — Procedure Banks,
- **current computation** — Working State.

### 🔁 Deliberation

After the normal 12-stage Cortex pass, Mini-T2 performs an additional
**deliberation pass** by reusing the final three Cortex stages:

`9 → 10 → 11`

This provides additional computation without repeating the entire network.

The resulting representation is passed through the final RMSNorm and language
model head to produce the next-token distribution.

### 📐 Mini-T2 v1 Configuration

| Component | Configuration |
|---|---:|
| Total parameters | **~219M** |
| Approx. active-path parameters | **~142M** |
| Cortex stages | **12** |
| Hidden dimension | **768** |
| Attention | **12 Q / 4 KV heads** |
| Procedure Banks | **3 shared banks** |
| Experts per bank | **16** |
| Expert routing | **Top-2 + shared expert** |
| Knowledge Vault | **65,536 slots** |
| Product-key candidates | **12 × 12 = 144** |
| Retrieved memories | **Top-4** |
| Context length | **1,024 tokens** |
| Deliberation | **Repeat stages 9 → 10 → 11** |

### 🧪 Research Status

T2 is an **experimental research architecture**, not an established
replacement for the Transformer.

The current Mini-T2 checkpoint demonstrates that the architecture can be
pretrained stably and produce coherent language. Internal ablation experiments
also indicate that some of its architectural components materially affect
model behaviour.

However, controlled comparisons against parameter- and training-matched
standard Transformer baselines are still required before making claims about
relative efficiency or capability.

	---
	library_name: transformers
	pipeline_tag: text-generation
	license: other
	license_name: orion-t2-research
	tags:
	- project-prism
	- orion
	- custom-code
	- causal-lm
	---

	# 🔺 Orion Flagship

	Orion Flagship is an experimental Project Prism model using the Orion T2
	architecture.

	T2 separates parts of the computation into a Universal Cortex, shared sparse
	Procedure Banks, a sparse Knowledge Vault, persistent Working State, and a
	cheap recurrent deliberation pass.

	This repository contains an experimental pretraining checkpoint and custom
	Transformers integration. It is not instruction tuned.

	## ⚠️ Access & Usage Restrictions

	- No use of the Orion T2 architecture for your own models. The
	architecture, its code, and its design are not licensed for building,
	training, or deriving other models. You can howvever make your own models and publish privately under this organisation.
	- Private repo only. This repository must stay private under the
	Project Prism organization. It is for Project Prism members only.
	- No outsider access (for now). Do not share, mirror, fork, or
	redistribute the weights or code outside Project Prism.

	```python
	from transformers import AutoTokenizer, AutoModelForCausalLM

	repo = "Project-Prism/Orion-Flagship-Mini-Preview"
	tok = AutoTokenizer.from_pretrained(repo, trust_remote_code=True)
	model = AutoModelForCausalLM.from_pretrained(
	repo,
	trust_remote_code=True,
	device_map="auto",
	)
	```

	The current T2 v1 compatibility layer deliberately uses `use_cache=False`.
	Generation therefore recomputes the full prefix each token so the recurrent
	deliberation + Working State graph remains faithful to training.

	## 🧠 Architecture

	![Orion T2 Architecture](./assets/orion-t2-architecture.png)

	### Transformer 2 (T2)

	Transformer 2 (T2) is an experimental language-model architecture developed
	under Project Prism.

	The central idea behind T2 is to separate several roles that are normally
	entangled inside the parameters and hidden activations of a conventional
	Transformer.

	Instead of relying on a single dense stack for knowledge storage, reusable
	computation, and temporary reasoning state, T2 introduces dedicated systems
	for each role:

	KNOW → DO → THINK → VERIFY

	### 🧠 Universal Cortex

	The Universal Cortex is the main sequence-processing backbone of Orion.

	Mini-T2 uses 12 Cortex stages with a hidden dimension of 768. Each
	stage contains attention and a routed Procedure Bank feed-forward path, using
	12 query heads and 4 KV heads.

	The Cortex is responsible for integrating:

	- token context,
	- retrieved knowledge,
	- routed procedures,
	- persistent Working State,
	- and recurrent deliberation.

	Unlike a standard Transformer where every layer owns an independent FFN,
	groups of Cortex stages access shared Procedure Banks.

	### ⚙️ Procedure Banks

	T2 separates reusable transformations into shared sparse Procedure Banks.

	Mini-T2 contains:

	- 3 shared Procedure Banks
	- 16 experts per bank
	- Top-2 routed experts
	- 1 shared expert

	Each bank is reused across a group of Cortex stages. A sparse router selects
	the most relevant procedures for each token while the shared expert provides
	a common computation path.

	The goal is to let the model learn reusable computational procedures without
	requiring every Cortex stage to independently store its own complete set of
	feed-forward transformations.

	### 📚 Knowledge Vault

	The Knowledge Vault provides a dedicated sparse memory system for
	declarative information.

	Mini-T2 contains 65,536 memory slots and uses product-key retrieval.

	For each retrieval:

	1. The query searches the two product-key components.
	2. The top 12 candidates from each side form 144 candidate addresses.
	3. The best 4 memories are retrieved.
	4. Retrieved information is gated into the Cortex.

	Knowledge interfaces are used at approximately stages 3, 7, and 11.

	This creates a separate pathway through which the model can retrieve stored
	information instead of requiring all declarative knowledge to be represented
	only through the main computational weights.

	### 📝 Working State

	T2 also maintains a separate persistent causal Working State.

	Working State acts as temporary computational memory. It can be read by the
	Cortex and updated as the sequence is processed.

	Updates use a learned gated replacement/blending mechanism rather than
	unbounded additive accumulation.

	In Mini-T2, Working State interacts with the Cortex around the same major
	interface points used by the Knowledge Vault.

	This gives T2 an explicit distinction between:

	- stored knowledge — Knowledge Vault,
	- reusable computation — Procedure Banks,
	- current computation — Working State.

	### 🔁 Deliberation

	After the normal 12-stage Cortex pass, Mini-T2 performs an additional
	deliberation pass by reusing the final three Cortex stages:

	`9 → 10 → 11`

	This provides additional computation without repeating the entire network.

	The resulting representation is passed through the final RMSNorm and language
	model head to produce the next-token distribution.

	### 📐 Mini-T2 v1 Configuration

	\| Component \| Configuration \|
	\|---\|---:\|
	\| Total parameters \| ~219M \|
	\| Approx. active-path parameters \| ~142M \|
	\| Cortex stages \| 12 \|
	\| Hidden dimension \| 768 \|
	\| Attention \| 12 Q / 4 KV heads \|
	\| Procedure Banks \| 3 shared banks \|
	\| Experts per bank \| 16 \|
	\| Expert routing \| Top-2 + shared expert \|
	\| Knowledge Vault \| 65,536 slots \|
	\| Product-key candidates \| 12 × 12 = 144 \|
	\| Retrieved memories \| Top-4 \|
	\| Context length \| 1,024 tokens \|
	\| Deliberation \| Repeat stages 9 → 10 → 11 \|

	### 🧪 Research Status

	T2 is an experimental research architecture, not an established
	replacement for the Transformer.

	The current Mini-T2 checkpoint demonstrates that the architecture can be
	pretrained stably and produce coherent language. Internal ablation experiments
	also indicate that some of its architectural components materially affect
	model behaviour.

	However, controlled comparisons against parameter- and training-matched
	standard Transformer baselines are still required before making claims about
	relative efficiency or capability.