Announcing Scalar-Phos-Flash

#2
by sclr-sf - opened
Scalar AI org

Announcing Scalar-Phos-Flash: Ultra-Fast 3B Llama-3.2 Fine-Tune with Native GGUF & LoRA Support

We are excited to share Scalar-Phos-Flash, an open-weights 3.2-billion parameter instruction-tuned model built on Meta's Llama 3.2 architecture and fine-tuned via 4-bit QLoRA using the Unsloth framework.


Project Goals & Approach

When deploying models to resource-constrained environments—such as free-tier accelerators, edge devices, and consumer laptops—balancing ultra-low latency with strong multi-domain reasoning is crucial.

With Scalar-Phos-Flash, our goal was to build an agile, high-throughput model that punches well above its weight class. By optimizing instruction-following routines without bloating memory overhead, Flash delivers competitive performance across graduate-level benchmarks while staying lightweight enough to run effortlessly on minimal hardware.


Benchmark Highlights (MMLU-Pro 5-Shot)

We evaluated Scalar-Phos-Flash using EleutherAI's lm-evaluation-harness under 5-shot conditions powered by the vLLM backend. Below is a detailed performance breakdown alongside leading open-weights models in the 3B parameter class on the rigorous MMLU-Pro benchmark:

Benchmark / Category Scalar Phos Flash (Ours) Llama-3.2-3B-Instruct Qwen2.5-3B-Instruct Phi-3.5-mini-instruct
Parameters / Class 3.2B (QLoRA) 3.2B 3.0B 3.8B
Overall MMLU-Pro 26.79% 24.52% 25.21% 27.08%
Biology 45.00% 41.20% 42.00% 44.50%
Business 45.00% 42.80% 43.10% 45.20%
Computer Science 25.00% 22.50% 24.00% 26.00%
Engineering 30.00% 27.50% 28.10% 29.80%
History 30.00% 28.00% 28.50% 30.50%
Law 30.00% 27.20% 28.00% 29.50%
Math 25.00% 22.00% 23.50% 25.80%
Physics 20.00% 18.50% 19.00% 21.00%
Psychology 30.00% 27.80% 28.20% 30.20%

How to Use the Model

1. Python (transformers + peft)

import torch
from transformers import AutoTokenizer, AutoModelForCausalLM
from peft import PeftModel

base_model_id = "meta-llama/Llama-3.2-3B-Instruct"
adapter_id = "scalar-ai/scalar-phos-flash"

tokenizer = AutoTokenizer.from_pretrained(base_model_id)
model = AutoModelForCausalLM.from_pretrained(
    base_model_id,
    torch_dtype=torch.bfloat16,
    device_map="auto"
)

# Load LoRA adapter directly from root
model = PeftModel.from_pretrained(model, adapter_id)

prompt = "Explain quantum computing in two simple sentences."
inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
outputs = model.generate(**inputs, max_new_tokens=100)
print(tokenizer.decode(outputs[0], skip_special_tokens=True))

2. Local Inference via Ollama or LM Studio

To run locally on macOS, Windows, or Linux using our quantized GGUF build:

ollama run hf.co/scalar-ai/scalar-phos-flash:Q4_K_M

Feedback & Future Work

We are releasing Scalar-Phos-Flash under the Llama 3.2 Community License. We welcome feedback, community evaluations, and suggestions for future domain dataset mixes. Feel free to open a discussion on our Hugging Face Community tab!

Happy hacking! 🚀

Sign up or log in to comment