LoRA: Low-Rank Adaptation of Large Language Models
Paper • 2106.09685 • Published • 67
TROUT
David Driscoll · inpuT Reconstruction from Output Using Transformers. Predicts a plausible user prompt from one assistant response.
Preview, supervised step 500 · Mean prompt cosine similarity 0.696 on 24 UltraChat validation responses with the included 8-bit base. Final held-out evaluation is pending.
Download with hf download ddrisco11/TROUT --local-dir TROUT. On Apple silicon, load with mlx_lm.load("TROUT/base-8bit", adapter_path="TROUT").
GitHub and inference instructions
Sources · Qwen2.5-0.5B-Instruct · UltraChat 200k · MiniLM sentence encoder used for evaluation · Hu et al., LoRA · Zhang, Morris & Shmatikov, Output2Prompt (EMNLP 2024).
Hardware compatibility
Log In to add your hardware
8-bit
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # if on a CUDA device, also pip install mlx[cuda] # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("ddrisco11/TROUT") prompt = "Once upon a time in" text = generate(model, tokenizer, prompt=prompt, verbose=True)