Instructions to use mackenzietechdocs/lexsi-tiny-aya-earth-rmu with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use mackenzietechdocs/lexsi-tiny-aya-earth-rmu with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="mackenzietechdocs/lexsi-tiny-aya-earth-rmu") messages = [ {"role": "user", "content": "Who are you?"}, ] pipe(messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("mackenzietechdocs/lexsi-tiny-aya-earth-rmu") model = AutoModelForCausalLM.from_pretrained("mackenzietechdocs/lexsi-tiny-aya-earth-rmu", device_map="auto") messages = [ {"role": "user", "content": "Who are you?"}, ] inputs = tokenizer.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use mackenzietechdocs/lexsi-tiny-aya-earth-rmu with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "mackenzietechdocs/lexsi-tiny-aya-earth-rmu" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mackenzietechdocs/lexsi-tiny-aya-earth-rmu", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/mackenzietechdocs/lexsi-tiny-aya-earth-rmu
- SGLang
How to use mackenzietechdocs/lexsi-tiny-aya-earth-rmu with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "mackenzietechdocs/lexsi-tiny-aya-earth-rmu" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mackenzietechdocs/lexsi-tiny-aya-earth-rmu", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "mackenzietechdocs/lexsi-tiny-aya-earth-rmu" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "mackenzietechdocs/lexsi-tiny-aya-earth-rmu", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use mackenzietechdocs/lexsi-tiny-aya-earth-rmu with Docker Model Runner:
docker model run hf.co/mackenzietechdocs/lexsi-tiny-aya-earth-rmu
lexsi-tiny-aya-earth-rmu
This repository contains a full bf16 RMU-unlearned checkpoint of CohereLabs/tiny-aya-earth after fact injection (LoRA) and Representation Misdirection for Unlearning (RMU). It is a complete saved model (not a LoRA or adapter). The gated base weights are not included.
- Base model: CohereLabs/tiny-aya-earth (gated, CC-BY-NC-4.0)
- Architecture: Cohere2ForCausalLM, 3.35B, 36 layers, hidden size 2048, tied embeddings
- Dtype: bfloat16
- Run id:
20261113T074830Z-inject+verify+unlearn+eval+probes+interp+relearn--d55bdb88 - Seed: one seed (this run)
- RMU: steers layer 18, updates layers 16โ18,
alpha=100,steering_coeff=20, 80 steps, content forget mask - Variant: Earth
What this does and does not claim
These weights are released so others can inspect suppression, utility, multilingual transfer, probes, and relearning. They do not establish that the forgotten facts were erased. A probe or a short relearning pass can still recover the facts. Logit lens, activation patching, head ablation, and CircuitKIT circuit analysis are not analyzed in this version and are left to future work.
How to load
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = "mackenzietechdocs/lexsi-tiny-aya-earth-rmu"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id, torch_dtype="bfloat16", device_map="auto")
You need access to the gated base tokenizer/config family if you also want to compare against CohereLabs/tiny-aya-earth itself.
License
CC-BY-NC-4.0, inherited from Tiny Aya. Non-commercial use only. Please also respect the Tiny Aya license and acceptable-use terms.
Citation / companion artifacts
Paper PDF, scrubbed traces, and translations are in the dataset mackenzietechdocs/lexsi-unlearning-eval. The demo Space (mackenzietechdocs/lexsi-unlearning-demo) browses those precomputed outputs and does not run live inference.
- Downloads last month
- 14