Text Generation
Transformers
Safetensors
English
qwen3_5
image-text-to-text
weight-space-modeling
zero-gradient
math
pedagogy
tutor
conversational
Eval Results (legacy)
Instructions to use DJLougen/Qwen3.5-0.8B-Math-Tutor with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use DJLougen/Qwen3.5-0.8B-Math-Tutor with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="DJLougen/Qwen3.5-0.8B-Math-Tutor") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] pipe(text=messages)# pip install -U transformers accelerate # Load model directly from transformers import AutoProcessor, AutoModelForMultimodalLM processor = AutoProcessor.from_pretrained("DJLougen/Qwen3.5-0.8B-Math-Tutor") model = AutoModelForMultimodalLM.from_pretrained("DJLougen/Qwen3.5-0.8B-Math-Tutor", device_map="auto") messages = [ { "role": "user", "content": [ {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"}, {"type": "text", "text": "What animal is on the candy?"} ] }, ] inputs = processor.apply_chat_template( messages, add_generation_prompt=True, tokenize=True, return_dict=True, return_tensors="pt", ).to(model.device) outputs = model.generate(**inputs, max_new_tokens=256) print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:])) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- vLLM
How to use DJLougen/Qwen3.5-0.8B-Math-Tutor with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "DJLougen/Qwen3.5-0.8B-Math-Tutor" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DJLougen/Qwen3.5-0.8B-Math-Tutor", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/DJLougen/Qwen3.5-0.8B-Math-Tutor
- SGLang
How to use DJLougen/Qwen3.5-0.8B-Math-Tutor with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "DJLougen/Qwen3.5-0.8B-Math-Tutor" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DJLougen/Qwen3.5-0.8B-Math-Tutor", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "DJLougen/Qwen3.5-0.8B-Math-Tutor" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "DJLougen/Qwen3.5-0.8B-Math-Tutor", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Docker Model Runner
How to use DJLougen/Qwen3.5-0.8B-Math-Tutor with Docker Model Runner:
docker model run hf.co/DJLougen/Qwen3.5-0.8B-Math-Tutor
|
Download README.md from DJLougen/Qwen3.5-0.8B-Math-Tutor: direct link, hf CLI and curl.
- Browser
- Download file 8.32 kB
-
https://huggingface.co/DJLougen/Qwen3.5-0.8B-Math-Tutor/resolve/main/README.md
- Command line
-
hf download hf://DJLougen/Qwen3.5-0.8B-Math-Tutor/README.md
-
curl -L -o README.md https://huggingface.co/DJLougen/Qwen3.5-0.8B-Math-Tutor/resolve/main/README.md
8.32 kB
| license: apache-2.0 | |
| license_link: https://huggingface.co/Qwen/Qwen3.5-0.8B-Base/blob/main/LICENSE | |
| base_model: Qwen/Qwen3.5-0.8B-Base | |
| library_name: transformers | |
| pipeline_tag: text-generation | |
| language: | |
| - en | |
| tags: | |
| - weight-space-modeling | |
| - zero-gradient | |
| - math | |
| - pedagogy | |
| - tutor | |
| - qwen3_5 | |
| model-index: | |
| - name: Qwen3.5-0.8B-Math-Tutor | |
| results: | |
| - task: | |
| type: text-generation | |
| name: Zero-Shot Math Tutoring (Plain + ChatML Held-Out) | |
| dataset: | |
| name: Carbon-MathTutor-HeldOut-180 | |
| type: custom | |
| split: test | |
| metrics: | |
| - type: accuracy | |
| value: 82.78 | |
| name: Overall Math Accuracy (%) | |
| - type: accuracy | |
| value: 80.00 | |
| name: Plain Q&A Math Accuracy (%) | |
| - type: accuracy | |
| value: 85.56 | |
| name: ChatML Math Accuracy (%) | |
| - type: accuracy | |
| value: 54.44 | |
| name: Verified Pedagogical Teaching (%) | |
| - type: accuracy | |
| value: 96.23 | |
| name: Intermediate Equation Soundness (%) | |
| <p align="center"> | |
| <img src="assets/hero_banner.png" alt="Qwen3.5-0.8B-Math-Tutor β Pure Static Weight-Space Math Surgery" width="100%"> | |
| </p> | |
| <p align="center"> | |
| <a href="https://huggingface.co/Qwen/Qwen3.5-0.8B-Base"><img src="https://img.shields.io/badge/Base-Qwen3.5--0.8B--Base-38bdf8?style=flat-square" alt="Base Model"></a> | |
| <img src="https://img.shields.io/badge/Params-0.8B%20(Drop--In%2024L)-10b981?style=flat-square" alt="0.8B Params"> | |
| <img src="https://img.shields.io/badge/Gradients-0%20(Static%20Weight%20Surgery)-34d399?style=flat-square" alt="Zero Gradients"> | |
| <img src="https://img.shields.io/badge/Edited%20Weights-292%2C600%20(0.039%25)-8b7cf6?style=flat-square" alt="Edited Weights"> | |
| <a href="https://huggingface.co/DJLougen/Qwen3.5-0.8B-Math-GraphCalc"><img src="https://img.shields.io/badge/Companion-35L%20In--Weights%20RISC--V%20GraphCalc-f59e0b?style=flat-square" alt="Companion Model"></a> | |
| </p> | |
| **`Qwen3.5-0.8B-Math-Tutor`** is a drop-in 24-layer `Qwen3.5-0.8B` pedagogical math tutor created strictly by editing **`292,600` (`0.039%`) static weights** of [`Qwen/Qwen3.5-0.8B-Base`](https://huggingface.co/Qwen/Qwen3.5-0.8B-Base) β with **100% zero gradients (`torch.set_grad_enabled(False)`), zero optimizer steps, and zero fitted activations**. | |
| Whereas untouched `Qwen3.5-0.8B-Base` collapses into bare 1-line wrong guesses (`13.33%` accuracy, `1.11%` verified teaching) on zero-shot `"Question: ...\nAnswer:"` prompts, `Qwen3.5-0.8B-Math-Tutor` derives solutions step by step across **both** plain Q&A (`80.00%` accuracy = **`6.0Γ`** Base) and native `ChatML` (`85.56%`), while shifting general `FineWeb-Edu` causal NLL by just **`+0.0030` nats** (`2.6202 -> 2.6232`). | |
| > Need exact 256-bit bignum arithmetic, CORDIC scientific functions, definite integrals, and function graphing compiled directly into the model's layers? See companion checkpoint **[`DJLougen/Qwen3.5-0.8B-Math-GraphCalc`](https://huggingface.co/DJLougen/Qwen3.5-0.8B-Math-GraphCalc)**. | |
| --- | |
| ## Example Output β Untouched Base vs. Weight-Space Math Tutor | |
| > **Zero-Shot Plain Prompt:** | |
| > ```text | |
| > Question: Solve for x: 7x + 9 = 3x + 41. | |
| > Answer: | |
| > ``` | |
| - **Untouched `Qwen/Qwen3.5-0.8B-Base`** (`0` weights edited β bare 1-line wrong guess): | |
| ```text | |
| x = 10 | |
| ``` | |
| - **`Qwen3.5-0.8B-Math-Tutor`** (`292,600` static weights edited, `0` gradients β verbatim greedy output): | |
| ```text | |
| We need to solve the linear equation: 7x + 9 = 3x + 41. | |
| We can solve by isolating x. | |
| Subtract 3x from both sides: 7x - 3x + 9 = 41 => 4x + 9 = 41. | |
| Subtract 9 from both sides: 4x = 41 - 9 = 32. | |
| Divide both sides by 4: x = 32/4 = 8. | |
| Thus x = 8. | |
| We can also check: 7*8 + 9 = 56 + 9 = 65; 3*8 + 41 = 24 + 41 = 65. Works. | |
| ``` | |
| --- | |
| ## Held-Out Benchmark (`N = 180` Unseen Problems Γ 2 Formats = `360` Generations) | |
| <p align="center"> | |
| <img src="assets/benchmarks.png" alt="Phase 1 Zero-Gradient Weight-Space Math & Pedagogy Benchmark" width="100%"> | |
| </p> | |
| Evaluated on `180` strictly disjoint problems (`30` each in `arithmetic`, `fractions_pct`, `algebra`, `geometry`, `gsm8k`, and `math_comp`, zero overlap with probe pool) plus `256` `FineWeb-Edu` 512-token blocks: | |
| | Arm | Changed Weights | Math Acc (`plain` / `chatml`) | Verified Teaching (`plain` / `chatml`) | Eq Soundness | Teach Score | FineWeb-Edu NLL | | |
| |---|---:|---:|---:|---:|---:|---:| | |
| | `Qwen/Qwen3.5-0.8B-Base` (`A0`) | `0` | `50.83%` (`13.33%` / `88.33%`) | `35.83%` (`1.11%` / `70.56%`) | `97.04%` | `0.6039` | `2.6202` | | |
| | `E1` Turn Bridge Only (`A1`) | `3,068` (`0.0004%`) | `80.83%` (`80.00%` / `81.67%`) | `51.11%` (`43.33%` / `58.89%`) | `95.27%` | `0.8833` | `2.6204` | | |
| | Zero-Leak Pareto (`A2`) | `623,927` (`0.083%`) | `78.06%` (`71.67%` / `84.44%`) | **`58.61%`** (**`53.89%`** / `63.33%`) | **`97.19%`** | **`0.8978`** | `2.6232` | | |
| | **`Qwen3.5-0.8B-Math-Tutor` (`A3`)** | **`292,600` (`0.039%`)** | **`82.78%` (`80.00%` / `85.56%`)** | `54.44%` (`42.22%` / **`66.67%`**) | `96.23%` | `0.8888` | `2.6232` | | |
| --- | |
| ## How It Works (Zero Gradients) | |
| Every edit is a closed-form algebraic function of the pretrained static weight tensors and BPE token IDs: | |
| 1. **`E1` β Closed-Form Turn-Embedding Bridge (`3,068` scalars)**: | |
| - In Qwen's BPE vocabulary, digits `'0'..'9'` carry no leading space while English openers (`' We'`, `' Let'`, `' Step'`) do β causing bare `' '` + 1-line guesses to win right after `Answer:`. | |
| - Re-anchoring `'Question'` (`14162`) and `'Answer'` (`15666`) in `embed_tokens.weight` toward Qwen3.5's latent mid-trained tutor circuit (`<|im_start|>`, `user`, `assistant`, `<think>`, `</think>`, `Step`/`Solution`) at `plain_norm_scale = 0.72` respects the Layer-0 un-normalized residual skip connection vs. tied `lm_head` self-echo threshold, jumping `plain` math accuracy **`6.0Γ`** (`13.33% -> 80.00%`) with `+0.0002` nats LM shift. | |
| 2. **`E2` β Anti-Pedagogical Late-MLP Suppression (`L14..L23`)**: | |
| - Damps specialist worksheet-continuation neurons (`"Question 2:"`), multi-newline collapse neurons, bare multiple-choice guess neurons, and a 6D CJK exam-leakage subspace orthogonalized against English math vocabulary. | |
| 3. **`E3` β Pedagogical Step-Starter & Equation-Writer Boost (`L13..L23`)**: | |
| - Boosts 32 specialist MLP neurons writing English step initiators, deductive connectors (`"therefore"`, `"thus"`, `"since"`), and explicit intermediate equations, plus turn-boundary `gate_proj`/`up_proj` key routing on 9 core step starters. | |
| --- | |
| ## Quick Start (`transformers`) | |
| `Qwen3.5-0.8B-Math-Tutor` preserves exact architectural parity with `Qwen/Qwen3.5-0.8B-Base` (`tie_word_embeddings = True`, standard 24-layer `Qwen3_5ForConditionalGeneration`, compatible with HuggingFace `transformers`, `vLLM`, and `SGLang`): | |
| ```python | |
| import torch | |
| from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration | |
| ckpt = "DJLougen/Qwen3.5-0.8B-Math-Tutor" | |
| tok = AutoTokenizer.from_pretrained(ckpt) | |
| model = Qwen3_5ForConditionalGeneration.from_pretrained( | |
| ckpt, | |
| dtype=torch.bfloat16, | |
| device_map="auto", | |
| ).eval() | |
| # Works zero-shot with plain "Question: ...\nAnswer:" OR ChatML | |
| prompt = "Question: Solve for x: 7x + 9 = 3x + 41.\nAnswer:" | |
| inputs = tok(prompt, return_tensors="pt").to(model.device) | |
| with torch.no_grad(): | |
| out = model.generate(**inputs, max_new_tokens=256, do_sample=False) | |
| print(tok.decode(out[0][inputs["input_ids"].shape[1]:], skip_special_tokens=True)) | |
| ``` | |
| --- | |
| ## Honest Limitations | |
| 1. **Mental-Math Digit Ceiling**: Without compiled tool layers, a `0.8B` model still fails 8β10 digit long multiplication (`20%` accuracy) and cannot plot coordinate graphs. Use **[`DJLougen/Qwen3.5-0.8B-Math-GraphCalc`](https://huggingface.co/DJLougen/Qwen3.5-0.8B-Math-GraphCalc)** when exact bignum/rational/CORDIC/graphing execution is required. | |
| 2. **Tied-Embedding Scratchpad vs. Clean-Stop Tradeoff**: Preserving `tie_word_embeddings = True` creates a slight Pareto tradeoff between maximum scratchpad self-verification (`82.78%` `math_acc`) and occasional literal `</think>` emissions when `skip_special_tokens=False`. | |
| --- | |
| ## License | |
| Apache-2.0 (inherited from [`Qwen/Qwen3.5-0.8B-Base`](https://huggingface.co/Qwen/Qwen3.5-0.8B-Base)). | |