Text Generation
Transformers
Safetensors
MLX
code
llama
fill-in-the-middle
multi-token-prediction
speculative-decoding
apple-silicon
text-generation-inference
Instructions to use philipjohnbasile/wisp-coder-110m with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use philipjohnbasile/wisp-coder-110m with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="philipjohnbasile/wisp-coder-110m")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("philipjohnbasile/wisp-coder-110m") model = AutoModelForCausalLM.from_pretrained("philipjohnbasile/wisp-coder-110m", device_map="auto") - MLX
How to use philipjohnbasile/wisp-coder-110m with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # if on a CUDA device, also pip install mlx[cuda] # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("philipjohnbasile/wisp-coder-110m") prompt = "Once upon a time in" text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- vLLM
How to use philipjohnbasile/wisp-coder-110m with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "philipjohnbasile/wisp-coder-110m" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "philipjohnbasile/wisp-coder-110m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/philipjohnbasile/wisp-coder-110m
- SGLang
How to use philipjohnbasile/wisp-coder-110m with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "philipjohnbasile/wisp-coder-110m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "philipjohnbasile/wisp-coder-110m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "philipjohnbasile/wisp-coder-110m" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "philipjohnbasile/wisp-coder-110m", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }' - MLX LM
How to use philipjohnbasile/wisp-coder-110m with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Generate some text mlx_lm.generate --model "philipjohnbasile/wisp-coder-110m" --prompt "Once upon a time"
- Docker Model Runner
How to use philipjohnbasile/wisp-coder-110m with Docker Model Runner:
docker model run hf.co/philipjohnbasile/wisp-coder-110m
- Atomic Chat
Clear prior rendered model card before superseding publication
Browse files
README.md
DELETED
|
@@ -1,230 +0,0 @@
|
|
| 1 |
-
---
|
| 2 |
-
license: apache-2.0
|
| 3 |
-
language:
|
| 4 |
-
- code
|
| 5 |
-
tags:
|
| 6 |
-
- code
|
| 7 |
-
- fill-in-the-middle
|
| 8 |
-
- multi-token-prediction
|
| 9 |
-
- speculative-decoding
|
| 10 |
-
- mlx
|
| 11 |
-
- apple-silicon
|
| 12 |
-
library_name: transformers
|
| 13 |
-
pipeline_tag: text-generation
|
| 14 |
-
---
|
| 15 |
-
|
| 16 |
-
# Wisp Coder 110M
|
| 17 |
-
|
| 18 |
-
A small code model built for one job: completing code at the cursor. It targets
|
| 19 |
-
editor inline suggestions, but no production-runtime latency claim is made.
|
| 20 |
-
|
| 21 |
-
Two things make it unusual, and they are the reason it exists rather than a
|
| 22 |
-
feature list.
|
| 23 |
-
|
| 24 |
-
**It does fill in the middle natively.** Each tokenized source document was split
|
| 25 |
-
into chunks of at most 1024 tokens, and 70 percent of those chunks were
|
| 26 |
-
independently transformed so the model predicts a middle span given both the
|
| 27 |
-
prefix and the suffix. This is not a per-document or per-window rate. Cursor
|
| 28 |
-
completion is the primary format, not an inference-time prompt trick added to an
|
| 29 |
-
ordinary left-to-right checkpoint.
|
| 30 |
-
|
| 31 |
-
**It has multi-token-prediction heads trained in from step zero.** A shared MTP
|
| 32 |
-
module, applied recursively in the Qwen3-Next style, predicts two or more tokens
|
| 33 |
-
ahead from the trunk's own hidden state. The LM head is shared between trunk and
|
| 34 |
-
MTP module, which ties the two output distributions to a common projection. That
|
| 35 |
-
is a useful architectural choice, not a guarantee that draft and target
|
| 36 |
-
distributions agree; how close they actually are is what acceptance measures.
|
| 37 |
-
|
| 38 |
-
**What is and is not claimed here.** Small models with MTP drafters exist: Google
|
| 39 |
-
publishes `gemma-4-E2B-it-assistant` and related MTP assistant checkpoints. The
|
| 40 |
-
artifact contribution is the *combination* at this scale, a small code model
|
| 41 |
-
trained from step zero with both fill-in-the-middle on most tokenized chunks and a
|
| 42 |
-
recursively shared MTP module, plus a paired measurement of how suffix information
|
| 43 |
-
changes draft acceptance. A targeted search found no exact prior measurement, but
|
| 44 |
-
that is not proof of absence. No exclusivity or first-of-its-kind claim is made.
|
| 45 |
-
|
| 46 |
-
## What it is not
|
| 47 |
-
|
| 48 |
-
It is not a chat model, an instruction-following model, or an agent. It completes
|
| 49 |
-
text. Asking it to refactor a module produces nonsense.
|
| 50 |
-
|
| 51 |
-
It is also not competitive with Qwen2.5-Coder on raw completion quality, and the
|
| 52 |
-
arithmetic says it cannot be: Qwen2.5-Coder-0.5B saw roughly 5.5 trillion tokens
|
| 53 |
-
and Wisp saw 5 billion, three orders of magnitude fewer. If you want the best
|
| 54 |
-
completions available at small scale, use Qwen. Use Wisp to study FIM plus native
|
| 55 |
-
speculative drafting on Apple Silicon, or how those two interact.
|
| 56 |
-
|
| 57 |
-
## Architecture
|
| 58 |
-
|
| 59 |
-
| | |
|
| 60 |
-
|---|---|
|
| 61 |
-
| Parameters | 108.2M total, 100.7M trunk |
|
| 62 |
-
| Layers | 12, d_model 768, 12 heads / 4 KV heads, SwiGLU 2048 |
|
| 63 |
-
| Context | 2048, RoPE theta 100k |
|
| 64 |
-
| Vocab | 32,768 byte-level BPE, digits split, FIM sentinels |
|
| 65 |
-
| MTP | 1 shared module (1 transformer block), trained to depth 2, recursive at inference |
|
| 66 |
-
| Precision | bfloat16 weights, trained with float32 master weights |
|
| 67 |
-
|
| 68 |
-
The trunk is an ordinary Llama decoder. It loads in `transformers` and `mlx_lm`
|
| 69 |
-
with no custom modelling code and no `trust_remote_code`. The MTP module ships
|
| 70 |
-
alongside as `mtp.safetensors` and is ignorable by runtimes that cannot use it.
|
| 71 |
-
Generic Llama runtimes do not consume the sidecar automatically. The package
|
| 72 |
-
therefore also ships `wisp_mtp_model.py` and `wisp_mtp_reference.py`, a
|
| 73 |
-
manifest-bound MLX correctness decoder that requires the exact sidecar schema,
|
| 74 |
-
verifies every declared package hash, exposes the resolved runtime contract and
|
| 75 |
-
executed route, and fails unless its MTP path reproduces target greedy tokens.
|
| 76 |
-
It recomputes full prefixes and is not a production speed path.
|
| 77 |
-
|
| 78 |
-
The export was verified against `transformers` numerically, not assumed: relative
|
| 79 |
-
logit delta 2.16e-3 and argmax agreement 1.0000 against the MLX original on the
|
| 80 |
-
same tokens, which pins the RoPE convention, the RMSNorm epsilon, and the grouped
|
| 81 |
-
query head order.
|
| 82 |
-
|
| 83 |
-
## Training data
|
| 84 |
-
|
| 85 |
-
The configured target mixture was 92 percent code from
|
| 86 |
-
`bigcode/starcoderdata` and 8 percent `HuggingFaceFW/fineweb-edu`. This is a
|
| 87 |
-
source percentage, not a permissive-license percentage. StarCoderData declares
|
| 88 |
-
`license: other`, and its original repository terms and relevant attribution
|
| 89 |
-
clauses remain applicable. FineWeb-Edu is ODC-By 1.0 and remains subject to
|
| 90 |
-
Common Crawl terms. Run 1 retained no row-level source manifest, so no per-file
|
| 91 |
-
licensing or attribution guarantee is made.
|
| 92 |
-
|
| 93 |
-
Every run 1 document received structural size, line-length, and character
|
| 94 |
-
distribution filters. Because the loader expected `path` instead of
|
| 95 |
-
StarCoderData's `max_stars_repo_path`, Python AST and JSON extension filtering
|
| 96 |
-
did not activate for run 1. The schema-aware repair applies only to future
|
| 97 |
-
source streaming, not to these weights. Corpus content was never executed.
|
| 98 |
-
Full details and the machine-checked disclosure are in `DATA.md` and
|
| 99 |
-
`config/training_data_receipt.json`.
|
| 100 |
-
|
| 101 |
-
The 32K tokenizer was trained on 400,000 documents sampled round-robin across
|
| 102 |
-
the eleven source entries, not according to the later token-budget weights.
|
| 103 |
-
Current shard bytes are fully attested after the build, but the receipt is not a
|
| 104 |
-
raw-row manifest and cannot prove exact original example boundaries.
|
| 105 |
-
|
| 106 |
-
The Apache 2.0 metadata describes the released Wisp artifact. It does not
|
| 107 |
-
override training-source terms or licenses applicable to generated code.
|
| 108 |
-
|
| 109 |
-
## Usage
|
| 110 |
-
|
| 111 |
-
Plain completion:
|
| 112 |
-
|
| 113 |
-
```python
|
| 114 |
-
from transformers import AutoModelForCausalLM, AutoTokenizer
|
| 115 |
-
|
| 116 |
-
model = AutoModelForCausalLM.from_pretrained("philipjohnbasile/wisp-coder-110m")
|
| 117 |
-
tok = AutoTokenizer.from_pretrained("philipjohnbasile/wisp-coder-110m")
|
| 118 |
-
print(tok.decode(model.generate(**tok("def quicksort(arr):", return_tensors="pt"),
|
| 119 |
-
max_new_tokens=64)[0]))
|
| 120 |
-
```
|
| 121 |
-
|
| 122 |
-
Fill in the middle. The model was trained with both orderings, evenly split:
|
| 123 |
-
|
| 124 |
-
```python
|
| 125 |
-
# PSM: prefix, suffix, then generate the middle
|
| 126 |
-
prompt = f"<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>"
|
| 127 |
-
```
|
| 128 |
-
|
| 129 |
-
Exercise the native sidecar explicitly. This is a correctness check, not a
|
| 130 |
-
latency benchmark:
|
| 131 |
-
|
| 132 |
-
```python
|
| 133 |
-
import os
|
| 134 |
-
import sys
|
| 135 |
-
from huggingface_hub import snapshot_download
|
| 136 |
-
from tokenizers import Tokenizer
|
| 137 |
-
|
| 138 |
-
package = snapshot_download(
|
| 139 |
-
"philipjohnbasile/wisp-coder-110m",
|
| 140 |
-
local_dir=os.path.abspath("wisp-coder-110m-package"),
|
| 141 |
-
)
|
| 142 |
-
sys.path.insert(0, package)
|
| 143 |
-
from wisp_mtp_reference import WispMTPReferenceRuntime
|
| 144 |
-
|
| 145 |
-
tok = Tokenizer.from_file(os.path.join(package, "tokenizer.json"))
|
| 146 |
-
ids = tok.encode(prompt, add_special_tokens=False).ids
|
| 147 |
-
runtime = WispMTPReferenceRuntime.load(package)
|
| 148 |
-
result = runtime.verify_greedy_parity(ids, max_new_tokens=16, depth=2)
|
| 149 |
-
print(result["mtp_route"])
|
| 150 |
-
```
|
| 151 |
-
|
| 152 |
-
For evaluation, the trunk can be configured as Continue.dev's autocomplete model
|
| 153 |
-
while leaving a larger model as the chat and agent backend. Target-runtime batch-1
|
| 154 |
-
latency and editor user experience have not yet been measured, so this is not a
|
| 155 |
-
sub-100 ms claim.
|
| 156 |
-
|
| 157 |
-
## Evaluation
|
| 158 |
-
|
| 159 |
-
### Registered final results
|
| 160 |
-
|
| 161 |
-
| Measurement | Result |
|
| 162 |
-
|---|---|
|
| 163 |
-
| Final validation main NLL | 1.3873 [1.3428, 1.4335], perplexity 4.00 |
|
| 164 |
-
| Validation MTP depth 1 NLL | 1.5727 [1.5236, 1.6234] |
|
| 165 |
-
| Validation MTP depth 2 NLL | 1.6513 [1.6013, 1.7028] |
|
| 166 |
-
| FIM / shuffled-suffix acceptance, depth 2 | 1.0268 [1.0209, 1.0331], POSITIVE, 200 documents |
|
| 167 |
-
| Trained minus initialized acceptance-ratio lift | +0.0268 [+0.0210, +0.0331], CLEARS_CONTROL |
|
| 168 |
-
| Acceptance interpretation | POSITIVE_AND_CLEARS_CONTROL |
|
| 169 |
-
| FIM-training effect on shuffled-FIM minus L2R acceptance | +0.0814 [+0.0747, +0.0883], POSITIVE, 200 documents |
|
| 170 |
-
| FIM-training effect on true-suffix minus shuffled-suffix acceptance | +0.0049 [-0.0026, +0.0122], NULL: the interval includes zero |
|
| 171 |
-
| Adaptive minus fixed accepted drafts per verification | +0.0000 [+0.0000, +0.0000], NULL: the interval includes 0, 60 test documents |
|
| 172 |
-
| Adaptive minus fixed output tokens per target forward | +0.0000 [+0.0000, +0.0000], NULL: the interval includes 0, 60 test documents |
|
| 173 |
-
| Adaptive minus fixed drafts issued per output token | +0.0000 [+0.0000, +0.0000], NULL: the interval includes 0, 60 test documents (issuance proxy) |
|
| 174 |
-
| Adaptive minus fixed draft recursions per output token | +0.0000 [+0.0000, +0.0000], NULL: the interval includes 0, 60 test documents |
|
| 175 |
-
| Rollout policies selected on calibration | adaptive_h0.7 versus fixed_d4 |
|
| 176 |
-
| Branch-local greedy replay | Exact argmax for all 38400 emitted tokens across 600 scored policy-document rollouts |
|
| 177 |
-
| Cross-policy output identity | Identical realized output branches for all 100 calibration/test documents across compared policies |
|
| 178 |
-
|
| 179 |
-
Validation intervals measure Monte Carlo uncertainty from the frozen random-window sampler. Acceptance and rollout intervals resample paired target documents. They do not measure training-run or model uncertainty. Null and negative outcomes are retained rather than filtered from the release.
|
| 180 |
-
|
| 181 |
-
Rollout endpoint scope: The primary endpoint measures accepted drafts per verification. It does not establish verification-width cost or deployment latency. Draft recursions per output token is the registered drafter-work companion; issued drafts per output token is retained only as an issuance proxy. Target forwards exclude the added post-hoc branch-replay forward and independent verification pass.
|
| 182 |
-
|
| 183 |
-
Independent replay provenance scope: Unsigned local attestation bound to a pushed pre-execution receipt commit and the registered source, inputs, checkpoint, and argv; it is not a signed external or trusted-execution witness.
|
| 184 |
-
|
| 185 |
-
Format-ablation provenance limitation: This is repository-revision evidence, not a per-document raw-corpus manifest for run 1. Source drift between the run 1 build and the captured cache refs cannot be ruled out.
|
| 186 |
-
|
| 187 |
-
Format-ablation runtime limitation: Run 1 did not record source-file hashes or sampler state in its pre-fix checkpoints. A post-build receipt now hashes all 52 current shard files and binds deterministic visible-grammar normalization, but it does not prove that the bytes were unchanged since training began or recover exact original units. The step-300 process recovery reset the legacy sampler, causing 78,643,200 scheduled token positions, 1.57 percent of the training budget, to replay earlier random windows. sampler_reset_steps [300] matches that reset timing in both arms. Later checkpoints preserve exact sampler RNG state. Exact sampled-example equivalence still cannot be proven.
|
| 188 |
-
|
| 189 |
-
One control worth repeating here, because acceptance rate is easy to report
|
| 190 |
-
dishonestly: **an initialized model scored higher acceptance than a trained one
|
| 191 |
-
in the E0 instrument pilot** (0.482 against 0.331 for a model deliberately
|
| 192 |
-
overfit on 100k tokens), because two near-uniform distributions agree trivially.
|
| 193 |
-
Those are pilot diagnostics, not run 1 results. Any final acceptance number
|
| 194 |
-
without its exact initialized floor beside it is uninterpretable.
|
| 195 |
-
|
| 196 |
-
## Limitations
|
| 197 |
-
|
| 198 |
-
- 5B training tokens. Small, and it shows on unfamiliar APIs, which it will
|
| 199 |
-
hallucinate confidently.
|
| 200 |
-
- 2048 token context.
|
| 201 |
-
- Ten languages, weighted toward Python, JavaScript, and TypeScript. Weaker
|
| 202 |
-
everywhere else, and untested outside that set.
|
| 203 |
-
- No safety tuning of any kind. It is a base completion model trained on public
|
| 204 |
-
GitHub code, and it will reproduce patterns from that data, including insecure
|
| 205 |
-
ones. Review what it writes.
|
| 206 |
-
- No row-level training-source manifest, license mapping, or attribution index.
|
| 207 |
-
Exact aggregate train-token totals survive in the recovered final build log,
|
| 208 |
-
but accepted rows, rejection counts, and per-source validation overshoot
|
| 209 |
-
cannot be reconstructed from the shards.
|
| 210 |
-
- The tokenizer document sampler was round-robin by source entry rather than
|
| 211 |
-
weighted like the training-token mixture.
|
| 212 |
-
- Training windows are sampled with replacement. The registered schedule's
|
| 213 |
-
uniform-interval approximation covers about 62.57 percent of corpus positions
|
| 214 |
-
at least once, rather than exposing every position once.
|
| 215 |
-
- The source-stratified validation shard is not repository- or document-disjoint.
|
| 216 |
-
A chunked source document can cross the validation-to-training boundary. The
|
| 217 |
-
separate publication holdout is repository-disjoint.
|
| 218 |
-
- The MTP sidecar is not consumed automatically by generic Llama runtimes, and
|
| 219 |
-
no production-runtime MTP latency or speedup is claimed.
|
| 220 |
-
- Trained and evaluated on one machine, one seed, one run.
|
| 221 |
-
|
| 222 |
-
## Citation
|
| 223 |
-
|
| 224 |
-
```bibtex
|
| 225 |
-
@software{wisp_coder,
|
| 226 |
-
title = {Wisp: fill-in-the-middle and native multi-token prediction in a small code model},
|
| 227 |
-
year = {2026},
|
| 228 |
-
url = {https://huggingface.co/philipjohnbasile/wisp-coder-110m}
|
| 229 |
-
}
|
| 230 |
-
```
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|