nmysore/manifold-1.0-1b

Manifold 1.0 — 1B

Attention manifold checkpoint for LLaMA 3.2 1B Instruct.

This model extends the standard transformer attention mechanism with learned manifold structures on the value path. The checkpoint contains the manifold parameters only — the base model weights are loaded from meta-llama/Llama-3.2-1B-Instruct.

Usage

pip install torch transformers inkan pygame numpy

# Inspect with the LLM Inspector
python inspector.py --model meta-llama/Llama-3.2-1B-Instruct --checkpoint formation.pt

See the LLM Inspector for the full visualization and editing toolkit.

Details

• Base model: LLaMA 3.2 1B Instruct
• Manifold parameters: 133K (0.01% of base model)
• Training data: Clean Wikipedia articles
• Sequence length: 1024 tokens

Paper forthcoming.