Unify the Wisp Fieldbook into a continuous illustrated page
Browse filesReplace the fragmented card layout with one paper canvas, a continuous chapter line, borderless editorial sections, integrated botanical artwork and an inline chapter index. Preserve hover behavior and default all 13 research panels to open. Retain every original technical paragraph, table entry, code block, metadata field, link and section anchor. Verified the complete Hugging Face Preview at desktop reading width and 390px mobile width, with no horizontal overflow and six native code blocks.
README.md
CHANGED
|
@@ -13,140 +13,134 @@ library_name: transformers
|
|
| 13 |
pipeline_tag: text-generation
|
| 14 |
---
|
| 15 |
|
| 16 |
-
<!-- The Wisp Fieldbook: a native Hugging Face model card. -->
|
| 17 |
-
<div id="wisp-cover"
|
| 18 |
-
<div style="display:flex;justify-content:space-between;align-items:center;flex-wrap:wrap;gap:12px;
|
| 19 |
<h1 id="wisp-coder-110m" style="font-family:Arial,sans-serif;font-size:13px;font-weight:600;letter-spacing:1.4px;text-transform:uppercase;color:#263f32;margin:0;line-height:1.6;">Wisp Coder 110M</h1>
|
| 20 |
<span style="font-family:Georgia,serif;font-size:13px;font-style:italic;color:#8e3049;">The Fieldbook / No. 001</span>
|
| 21 |
</div>
|
| 22 |
-
<div style="display:flex;align-items:center;flex-wrap:wrap;gap:20px;">
|
| 23 |
-
<div style="flex:1 1 250px;min-width:0;">
|
| 24 |
<p style="font-size:10px;letter-spacing:1.5px;color:#5d704f;margin:0 0 18px;">CODE · CONTEXT · CURIOSITY</p>
|
| 25 |
-
<p style="font-family:Georgia,serif;font-size:clamp(
|
| 26 |
-
<p style="font-family:Georgia,serif;font-size:
|
| 27 |
-
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:scale-[1.01] hover:brightness-[102%] focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" href="https://huggingface.co/spaces/philipjohnbasile/wisp-coder-demo" style="display:inline-block;
|
| 28 |
-
<p style="font-size:11px;line-height:1.7;color:#50614e;margin:22px 0 0;">An open research experiment<br>by Philip John Basile</p>
|
| 29 |
</div>
|
| 30 |
-
<div style="flex:1 1 270px;min-width:0;max-width:100%;padding:12px 8px;">
|
| 31 |
-
<img class="wisp-botanical motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-hover:scale-105 group-hover:rotate-6" src="./assets/wisp-wildflowers.png"
|
| 32 |
<p style="font-family:Georgia,serif;font-style:italic;color:#5d704f;font-size:14px;text-align:right;margin:20px 5px 0;">Small model. Open questions.</p>
|
| 33 |
</div>
|
| 34 |
</div>
|
| 35 |
-
<div style="display:flex;flex-wrap:wrap;gap:14px 25px;
|
| 36 |
-
<div><span style="font-family:Georgia,serif;font-size:26px;color:#263f32;">100.7M</span><br><span style="font-size:10px;letter-spacing:1px;color:#50614e;">TRUNK PARAMETERS</span></div>
|
| 37 |
-
<div><span style="font-family:Georgia,serif;font-size:26px;color:#ad3f5a;">108.2M</span><br><span style="font-size:10px;letter-spacing:1px;color:#50614e;">INCLUDING MTP</span></div>
|
| 38 |
-
<div><span style="font-family:Georgia,serif;font-size:26px;color:#263f32;">2,048</span><br><span style="font-size:10px;letter-spacing:1px;color:#50614e;">TOKEN CONTEXT</span></div>
|
| 39 |
</div>
|
| 40 |
</div>
|
| 41 |
-
|
| 42 |
-
<
|
| 43 |
-
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:
|
| 44 |
-
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:
|
| 45 |
-
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:
|
| 46 |
-
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:
|
| 47 |
-
<
|
|
|
|
|
|
|
|
|
|
|
|
|
| 48 |
</div>
|
| 49 |
|
| 50 |
-
|
|
|
|
| 51 |
|
| 52 |
-
<div style="margin:38px 0 22px;padding:20px 24px;background:#f4f1e8;border-left:3px solid #ad3f5a;">
|
| 53 |
-
<p style="margin:0 0 6px;color:#ad3f5a;font-size:11px;letter-spacing:1.8px;text-transform:uppercase;">01 / A place to begin.</p>
|
| 54 |
-
<h2 id="start-here" style="font-family:Georgia,serif;font-size:30px;line-height:1.2;font-weight:400;letter-spacing:-0.6px;color:#263f32;margin:0;padding:0;border:0;">Start here</h2>
|
| 55 |
-
</div>
|
| 56 |
|
| 57 |
-
|
| 58 |
-
|
| 59 |
-
[
|
| 60 |
-
|
| 61 |
-
<
|
| 62 |
-
<
|
| 63 |
-
<h3 style="font-family:Georgia,serif;font-weight:400;font-size:32px;line-height:1.2;color:#f7f2e6;margin:0 0 12px;">A little room <em style="color:#f3a4b8;">for possibility.</em></h3>
|
| 64 |
-
<p style="font-size:13px;line-height:1.7;color:#dae1cd;margin:0 0 20px;">Explore each piece of the prompt. Collapse any panel to focus your view.</p>
|
| 65 |
-
<div style="display:flex;align-items:flex-start;flex-wrap:wrap;gap:10px;">
|
| 66 |
-
<details open id="wisp-prefix" class="group motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:scale-[1.01] hover:brightness-[102%]" style="flex:1 1 150px;min-width:0;border:1px solid #80917a;background:#2b4436;color:#f7f2e6;">
|
| 67 |
-
<summary class="focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:16px;">
|
| 68 |
-
<span style="display:block;font-size:10px;letter-spacing:1px;color:#dae1cd;margin:0 0 10px;">YOU PROVIDE</span>
|
| 69 |
-
<span style="display:flex;align-items:center;justify-content:space-between;font-family:Georgia,serif;font-size:24px;color:#f7f2e6;margin:0 0 6px;">Prefix<span class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-open:rotate-180" style="display:inline-block;font-family:Arial,sans-serif;font-size:18px;">⌄</span></span>
|
| 70 |
-
<span style="display:block;font-size:13px;line-height:1.6;color:#dae1cd;">Code before the gap</span>
|
| 71 |
</summary>
|
| 72 |
-
<div style="
|
| 73 |
</details>
|
| 74 |
-
<details
|
| 75 |
-
<summary class="focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:
|
| 76 |
-
<span style="display:block;font-size:10px;letter-spacing:1px;color:#
|
| 77 |
-
<span style="display:flex;align-items:center;justify-content:space-between;font-family:Georgia,serif;font-size:24px;color:#
|
| 78 |
-
<span style="display:block;font-size:13px;line-height:1.6;color:#
|
| 79 |
</summary>
|
| 80 |
-
<div style="
|
| 81 |
</details>
|
| 82 |
-
<details
|
| 83 |
-
<summary class="focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:
|
| 84 |
-
<span style="display:block;font-size:10px;letter-spacing:1px;color:#
|
| 85 |
-
<span style="display:flex;align-items:center;justify-content:space-between;font-family:Georgia,serif;font-size:24px;color:#
|
| 86 |
-
<span style="display:block;font-size:13px;line-height:1.6;color:#
|
| 87 |
</summary>
|
| 88 |
-
<div style="
|
| 89 |
</details>
|
| 90 |
</div>
|
| 91 |
-
<p style="font-size:13px;line-height:1.7;color:#
|
| 92 |
</div>
|
| 93 |
-
|
| 94 |
-
<
|
| 95 |
-
<
|
| 96 |
-
<
|
| 97 |
-
<
|
| 98 |
-
<span style="display:
|
| 99 |
-
<span style="display:block;font-family:Georgia,serif;font-size:24px;font-weight:400;line-height:1.2;color:#263f32;">What is it for?</span>
|
| 100 |
</summary>
|
| 101 |
-
<div style="padding:0
|
| 102 |
</details>
|
| 103 |
-
<details
|
| 104 |
-
<summary class="focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:
|
| 105 |
-
<span style="display:flex;justify-content:space-between;align-items:center;font-size:
|
| 106 |
-
<span style="display:block;font-family:Georgia,serif;font-size:
|
| 107 |
</summary>
|
| 108 |
-
<div style="padding:0
|
| 109 |
</details>
|
| 110 |
-
<details
|
| 111 |
-
<summary class="focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:
|
| 112 |
-
<span style="display:flex;justify-content:space-between;align-items:center;font-size:
|
| 113 |
-
<span style="display:block;font-family:Georgia,serif;font-size:
|
| 114 |
</summary>
|
| 115 |
-
<div style="padding:0
|
| 116 |
</details>
|
| 117 |
-
<details
|
| 118 |
-
<summary class="focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:
|
| 119 |
-
<span style="display:flex;justify-content:space-between;align-items:center;font-size:
|
| 120 |
-
<span style="display:block;font-family:Georgia,serif;font-size:
|
| 121 |
</summary>
|
| 122 |
-
<div style="padding:0
|
| 123 |
</details>
|
| 124 |
-
<details
|
| 125 |
-
<summary class="focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:
|
| 126 |
-
<span style="display:flex;justify-content:space-between;align-items:center;font-size:
|
| 127 |
-
<span style="display:block;font-family:Georgia,serif;font-size:
|
| 128 |
</summary>
|
| 129 |
-
<div style="padding:0
|
| 130 |
</details>
|
| 131 |
-
<details id="wisp-note-6"
|
| 132 |
-
<summary class="focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:
|
| 133 |
-
<span style="display:flex;justify-content:space-between;align-items:center;font-size:
|
| 134 |
-
<span style="display:block;font-family:Georgia,serif;font-size:
|
| 135 |
</summary>
|
| 136 |
-
<div style="padding:0
|
| 137 |
</details>
|
| 138 |
</div>
|
|
|
|
|
|
|
|
|
|
|
|
|
| 139 |
|
| 140 |
-
### Quick start — ordinary completion on CPU
|
| 141 |
-
|
| 142 |
-
<details class="group" open style="border:1px solid #829471;margin:20px 0;padding:0;">
|
| 143 |
-
<summary class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:scale-[1.01] hover:brightness-[102%] focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;padding:16px 20px;background:#ecefdf;color:#263f32;font-family:Georgia,serif;font-size:20px;">Run an ordinary completion on CPU</summary>
|
| 144 |
-
<div style="padding:4px 20px 16px;">
|
| 145 |
|
| 146 |
```bash
|
| 147 |
pip install "torch==2.13.0" "transformers==5.14.1"
|
| 148 |
```
|
| 149 |
|
|
|
|
|
|
|
|
|
|
| 150 |
```python
|
| 151 |
import torch
|
| 152 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
@@ -164,136 +158,140 @@ with torch.inference_mode():
|
|
| 164 |
print(tokenizer.decode(output[0], skip_special_tokens=True))
|
| 165 |
```
|
| 166 |
|
|
|
|
| 167 |
</div>
|
| 168 |
</details>
|
| 169 |
-
|
| 170 |
-
|
| 171 |
-
|
| 172 |
-
|
| 173 |
-
|
| 174 |
-
|
| 175 |
-
|
| 176 |
-
|
| 177 |
-
Two things make it unusual, and they are the reason it exists rather than a
|
| 178 |
-
feature list.
|
| 179 |
-
|
| 180 |
-
**It does fill in the middle natively.** Each tokenized source document was split
|
| 181 |
into chunks of at most 1024 tokens, and 70 percent of those chunks were
|
| 182 |
independently transformed so the model predicts a middle span given both the
|
| 183 |
prefix and the suffix. This is not a per-document or per-window rate. Cursor
|
| 184 |
completion is the primary format, not an inference-time prompt trick added to an
|
| 185 |
-
ordinary left-to-right checkpoint.
|
| 186 |
-
|
| 187 |
-
**It has multi-token-prediction heads trained in from step zero.** A shared MTP
|
| 188 |
module, applied recursively in the Qwen3-Next style, predicts two or more tokens
|
| 189 |
ahead from the trunk's own hidden state. The LM head is shared between trunk and
|
| 190 |
MTP module, which ties the two output distributions to a common projection. That
|
| 191 |
is a useful architectural choice, not a guarantee that draft and target
|
| 192 |
-
distributions agree; how close they actually are is what acceptance measures.
|
| 193 |
-
|
| 194 |
-
|
| 195 |
-
|
| 196 |
-
artifact contribution is the *combination* at this scale, a small code model
|
| 197 |
trained from step zero with both fill-in-the-middle on most tokenized chunks and a
|
| 198 |
recursively shared MTP module, plus a paired measurement of how suffix information
|
| 199 |
changes draft acceptance. A targeted search found no exact prior measurement, but
|
| 200 |
-
that is not proof of absence. No exclusivity or first-of-its-kind claim is made.
|
| 201 |
-
|
| 202 |
-
<
|
| 203 |
-
<
|
| 204 |
-
<h2 id="what-it-is-not" style="font-family:Georgia,serif;font-size:30px;line-height:1.2;font-weight:400;letter-spacing:-0.6px;color:#263f32;margin:0;padding:0;border:0;">What it is not</h2>
|
| 205 |
</div>
|
| 206 |
-
|
| 207 |
-
|
| 208 |
-
|
| 209 |
-
|
| 210 |
-
It is also not competitive with Qwen2.5-Coder on raw completion quality, and the
|
| 211 |
arithmetic says it cannot be: Qwen2.5-Coder-0.5B saw roughly 5.5 trillion tokens
|
| 212 |
and Wisp saw 5 billion, three orders of magnitude fewer. If you want the best
|
| 213 |
completions available at small scale, use Qwen. Use Wisp to study FIM plus native
|
| 214 |
-
speculative drafting on Apple Silicon, or how those two interact.
|
| 215 |
-
|
| 216 |
-
<
|
| 217 |
-
<
|
| 218 |
-
<h2 id="architecture" style="font-family:Georgia,serif;font-size:30px;line-height:1.2;font-weight:400;letter-spacing:-0.6px;color:#263f32;margin:0;padding:0;border:0;">Architecture</h2>
|
| 219 |
</div>
|
| 220 |
-
|
| 221 |
-
<
|
| 222 |
-
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:scale-[1.01] hover:brightness-[102%] focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" href="https://huggingface.co/philipjohnbasile/wisp-coder-110m/blob/main/config.json" style="display:block;background:#ecefdf;border:1px solid #aebaa0;padding:22px;color:#263f32;text-decoration:none;">
|
| 223 |
<span style="display:block;font-size:10px;letter-spacing:1.4px;color:#50614e;margin-bottom:12px;">ROUTE A / THE TRUNK</span><span style="display:block;font-family:Georgia,serif;font-size:27px;line-height:1.2;color:#263f32;margin-bottom:12px;">An ordinary Llama decoder.</span><span style="display:block;font-size:13px;line-height:1.7;color:#354b3b;">100.7M parameters. Standard Transformers and MLX-LM loading.</span><span style="display:block;font-size:12px;color:#8e3049;margin-top:16px;">Inspect the configuration ↗</span>
|
| 224 |
</a>
|
| 225 |
-
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:
|
| 226 |
<span style="display:block;font-size:10px;letter-spacing:1.4px;color:#8e3049;margin-bottom:12px;">ROUTE B / NATIVE MTP</span><span style="display:block;font-family:Georgia,serif;font-size:27px;line-height:1.2;color:#263f32;margin-bottom:12px;">A shared drafting module.</span><span style="display:block;font-size:13px;line-height:1.7;color:#354b3b;">108.2M parameters with the sidecar. Use the explicit MLX correctness runtime.</span><span style="display:block;font-size:12px;color:#8e3049;margin-top:16px;">Read the reference runtime ↗</span>
|
| 227 |
</a>
|
| 228 |
</div>
|
| 229 |
-
<p style="font-size:13px;"><a href="https://huggingface.co/philipjohnbasile/wisp-coder-110m/blob/main/export_manifest.json">Export manifest ↗</a>
|
| 230 |
-
|
| 231 |
-
|
| 232 |
-
|
| 233 |
-
|
| 234 |
-
|
| 235 |
-
|
| 236 |
-
|
| 237 |
-
|
| 238 |
-
|
| 239 |
-
|
| 240 |
-
|
| 241 |
-
|
| 242 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 243 |
Generic Llama runtimes do not consume the sidecar automatically. The package
|
| 244 |
-
therefore also ships
|
| 245 |
manifest-bound MLX correctness decoder that requires the exact sidecar schema,
|
| 246 |
verifies every declared package hash, exposes the resolved runtime contract and
|
| 247 |
executed route, and fails unless its MTP path reproduces target greedy tokens.
|
| 248 |
-
It recomputes full prefixes and is not a production speed path.
|
| 249 |
-
|
| 250 |
-
The export was verified against `transformers` numerically, not assumed: relative
|
| 251 |
logit delta 2.16e-3 and argmax agreement 1.0000 against the MLX original on the
|
| 252 |
same tokens, which pins the RoPE convention, the RMSNorm epsilon, and the grouped
|
| 253 |
-
query head order.
|
| 254 |
-
|
| 255 |
-
<
|
| 256 |
-
<
|
| 257 |
-
<h2 id="training-data" style="font-family:Georgia,serif;font-size:30px;line-height:1.2;font-weight:400;letter-spacing:-0.6px;color:#263f32;margin:0;padding:0;border:0;">Training data</h2>
|
| 258 |
</div>
|
| 259 |
-
|
| 260 |
-
|
| 261 |
-
`bigcode/starcoderdata` and 8 percent `HuggingFaceFW/fineweb-edu`. This is a
|
| 262 |
source percentage, not a permissive-license percentage. StarCoderData declares
|
| 263 |
-
|
| 264 |
clauses remain applicable. FineWeb-Edu is ODC-By 1.0 and remains subject to
|
| 265 |
Common Crawl terms. Run 1 retained no row-level source manifest, so no per-file
|
| 266 |
-
licensing or attribution guarantee is made.
|
| 267 |
-
|
| 268 |
-
<
|
| 269 |
-
<
|
| 270 |
-
<
|
| 271 |
-
|
| 272 |
-
|
| 273 |
-
distribution filters. Because the loader expected `path` instead of
|
| 274 |
-
StarCoderData's `max_stars_repo_path`, Python AST and JSON extension filtering
|
| 275 |
did not activate for run 1. The schema-aware repair applies only to future
|
| 276 |
source streaming, not to these weights. Corpus content was never executed.
|
| 277 |
-
Read the published
|
| 278 |
-
and
|
| 279 |
-
|
| 280 |
-
The 32K tokenizer was trained on 400,000 documents sampled round-robin across
|
| 281 |
the eleven source entries, not according to the later token-budget weights.
|
| 282 |
Current shard bytes are fully attested after the build, but the receipt is not a
|
| 283 |
-
raw-row manifest and cannot prove exact original example boundaries.
|
| 284 |
-
|
| 285 |
</div>
|
| 286 |
</details>
|
| 287 |
-
|
| 288 |
-
|
| 289 |
-
|
| 290 |
-
|
| 291 |
-
<
|
| 292 |
-
<p style="margin:0 0 6px;color:#ad3f5a;font-size:11px;letter-spacing:1.8px;text-transform:uppercase;">05 / Give it a few lines.</p>
|
| 293 |
-
<h2 id="usage" style="font-family:Georgia,serif;font-size:30px;line-height:1.2;font-weight:400;letter-spacing:-0.6px;color:#263f32;margin:0;padding:0;border:0;">Usage</h2>
|
| 294 |
</div>
|
|
|
|
| 295 |
|
| 296 |
-
Plain completion:
|
| 297 |
|
| 298 |
```python
|
| 299 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
@@ -304,15 +302,19 @@ print(tok.decode(model.generate(**tok("def quicksort(arr):", return_tensors="pt"
|
|
| 304 |
max_new_tokens=64)[0]))
|
| 305 |
```
|
| 306 |
|
| 307 |
-
|
|
|
|
|
|
|
| 308 |
|
| 309 |
```python
|
| 310 |
# PSM: prefix, suffix, then generate the middle
|
| 311 |
prompt = f"<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>"
|
| 312 |
```
|
| 313 |
|
| 314 |
-
|
| 315 |
-
|
|
|
|
|
|
|
| 316 |
|
| 317 |
```python
|
| 318 |
import os
|
|
@@ -334,107 +336,147 @@ result = runtime.verify_greedy_parity(ids, max_new_tokens=16, depth=2)
|
|
| 334 |
print(result["mtp_route"])
|
| 335 |
```
|
| 336 |
|
| 337 |
-
|
|
|
|
| 338 |
while leaving a larger model as the chat and agent backend. Target-runtime batch-1
|
| 339 |
latency and editor user experience have not yet been measured, so this is not a
|
| 340 |
-
sub-100 ms claim.
|
| 341 |
-
|
| 342 |
-
<
|
| 343 |
-
<
|
| 344 |
-
<h2 id="evaluation" style="font-family:Georgia,serif;font-size:30px;line-height:1.2;font-weight:400;letter-spacing:-0.6px;color:#f8f3e7;margin:0;padding:0;border:0;">Evaluation</h2>
|
| 345 |
</div>
|
| 346 |
-
|
| 347 |
-
<div style="
|
| 348 |
-
<div
|
| 349 |
-
<div
|
| 350 |
-
<div class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:scale-[1.01] hover:brightness-[102%]" style="padding:18px;background:#f6e8e8;border-top:3px solid #ad3f5a;color:#263f32;"><p style="font-size:10px;letter-spacing:1px;color:#8e3049;margin:0 0 9px;">MTP RUNTIME</p><p style="font-family:Georgia,serif;font-size:23px;line-height:1.2;color:#263f32;margin:0 0 10px;">Speed is open.</p><p style="font-size:13px;line-height:1.7;color:#354b3b;margin:0;">No production MTP speedup is claimed. The packaged MLX runtime checks correctness.</p></div>
|
| 351 |
</div>
|
| 352 |
-
|
| 353 |
-
|
| 354 |
-
|
| 355 |
-
<
|
| 356 |
-
<
|
| 357 |
-
<
|
| 358 |
-
|
| 359 |
-
|
| 360 |
-
|
| 361 |
-
|
| 362 |
-
|
| 363 |
-
|
| 364 |
-
|
| 365 |
-
|
| 366 |
-
|
| 367 |
-
|
| 368 |
-
|
| 369 |
-
|
| 370 |
-
|
| 371 |
-
|
| 372 |
-
|
| 373 |
-
|
| 374 |
-
|
| 375 |
-
|
| 376 |
-
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 377 |
</div>
|
| 378 |
</details>
|
| 379 |
-
|
| 380 |
-
|
| 381 |
-
|
| 382 |
-
<
|
| 383 |
-
<
|
| 384 |
-
<
|
| 385 |
-
|
| 386 |
-
|
| 387 |
-
|
| 388 |
-
Independent replay provenance scope: Unsigned local attestation bound to a pushed pre-execution receipt commit and the registered source, inputs, checkpoint, and argv; it is not a signed external or trusted-execution witness.
|
| 389 |
-
|
| 390 |
-
Format-ablation provenance limitation: This is repository-revision evidence, not a per-document raw-corpus manifest for run 1. Source drift between the run 1 build and the captured cache refs cannot be ruled out.
|
| 391 |
-
|
| 392 |
-
Format-ablation runtime limitation: Run 1 did not record source-file hashes or sampler state in its pre-fix checkpoints. A post-build receipt now hashes all 52 current shard files and binds deterministic visible-grammar normalization, but it does not prove that the bytes were unchanged since training began or recover exact original units. The step-300 process recovery reset the legacy sampler, causing 78,643,200 scheduled token positions, 1.57 percent of the training budget, to replay earlier random windows. sampler_reset_steps [300] matches that reset timing in both arms. Later checkpoints preserve exact sampler RNG state. Exact sampled-example equivalence still cannot be proven.
|
| 393 |
-
|
| 394 |
</div>
|
| 395 |
</details>
|
| 396 |
-
|
| 397 |
-
|
| 398 |
-
|
| 399 |
-
in the E0 instrument pilot** (0.482 against 0.331 for a model deliberately
|
| 400 |
overfit on 100k tokens), because two near-uniform distributions agree trivially.
|
| 401 |
Those are pilot diagnostics, not run 1 results. Any final acceptance number
|
| 402 |
-
without its exact initialized floor beside it is uninterpretable.
|
| 403 |
-
|
| 404 |
-
<
|
| 405 |
-
<
|
| 406 |
-
<h2 id="limitations" style="font-family:Georgia,serif;font-size:30px;line-height:1.2;font-weight:400;letter-spacing:-0.6px;color:#263f32;margin:0;padding:0;border:0;">Limitations</h2>
|
| 407 |
</div>
|
| 408 |
-
|
| 409 |
-
|
| 410 |
-
|
| 411 |
-
|
| 412 |
-
|
| 413 |
-
|
| 414 |
-
|
| 415 |
-
|
| 416 |
-
|
| 417 |
-
|
| 418 |
-
|
| 419 |
-
|
| 420 |
-
|
| 421 |
-
|
| 422 |
-
|
| 423 |
-
|
| 424 |
-
|
| 425 |
-
|
| 426 |
-
|
| 427 |
-
|
| 428 |
-
|
| 429 |
-
|
| 430 |
-
|
| 431 |
-
|
| 432 |
-
|
| 433 |
-
<div style="margin:
|
| 434 |
-
<p style="margin:0 0
|
| 435 |
-
<h2 id="citation" style="font-family:Georgia,serif;font-size:
|
| 436 |
</div>
|
| 437 |
|
|
|
|
| 438 |
```bibtex
|
| 439 |
@software{wisp_coder,
|
| 440 |
title = {Wisp: fill-in-the-middle and native multi-token prediction in a small code model},
|
|
@@ -444,13 +486,15 @@ without its exact initialized floor beside it is uninterpretable.
|
|
| 444 |
```
|
| 445 |
|
| 446 |
|
| 447 |
-
<div
|
| 448 |
-
<
|
| 449 |
-
<div style="
|
|
|
|
|
|
|
| 450 |
<p style="font-size:11px;letter-spacing:1.5px;color:#50614e;margin:0 0 8px;">SMALL MODEL. OPEN QUESTIONS.</p>
|
| 451 |
<p style="font-family:Georgia,serif;font-size:42px;letter-spacing:-2px;line-height:1.15;color:#263f32;margin:0 0 12px;">Stay <em style="color:#ad3f5a;">curious.</em></p>
|
| 452 |
-
<p style="font-size:14px;line-height:1.8;margin:0;"><a href="https://huggingface.co/spaces/philipjohnbasile/wisp-coder-demo" style="color:#
|
| 453 |
</div>
|
| 454 |
</div>
|
| 455 |
-
|
| 456 |
-
<
|
|
|
|
| 13 |
pipeline_tag: text-generation
|
| 14 |
---
|
| 15 |
|
| 16 |
+
<div id="wisp-experience" style="color:#354b3b;background:#f4f1e8;padding:clamp(18px,3vw,38px);margin:8px 0 0;position:relative;background-image:radial-gradient(#d7dbc9 0.55px,transparent 0.55px);background-size:8px 8px;font-size:15px;line-height:1.8;overflow-wrap:anywhere;"><!-- The Wisp Fieldbook: a native Hugging Face model card. -->
|
| 17 |
+
<div class="group" id="wisp-cover" style="color:#263f32;padding:0;margin:0;background:transparent;">
|
| 18 |
+
<div style="display:flex;justify-content:space-between;align-items:center;flex-wrap:wrap;gap:12px;padding-bottom:18px;margin-bottom:36px;background:transparent;border-bottom:1px solid #bfccb1;">
|
| 19 |
<h1 id="wisp-coder-110m" style="font-family:Arial,sans-serif;font-size:13px;font-weight:600;letter-spacing:1.4px;text-transform:uppercase;color:#263f32;margin:0;line-height:1.6;">Wisp Coder 110M</h1>
|
| 20 |
<span style="font-family:Georgia,serif;font-size:13px;font-style:italic;color:#8e3049;">The Fieldbook / No. 001</span>
|
| 21 |
</div>
|
| 22 |
+
<div style="display:flex;align-items:center;flex-wrap:wrap;gap:20px;background:transparent;">
|
| 23 |
+
<div style="flex:1 1 250px;min-width:0;background:transparent;">
|
| 24 |
<p style="font-size:10px;letter-spacing:1.5px;color:#5d704f;margin:0 0 18px;">CODE · CONTEXT · CURIOSITY</p>
|
| 25 |
+
<p style="font-family:Georgia,serif;font-size:clamp(62px,7vw,84px);font-weight:400;letter-spacing:-4px;line-height:1;color:#263f32;margin:0 0 24px;">Mind<br/><em style="color:#ad3f5a;">the gap.</em></p>
|
| 26 |
+
<p style="font-family:Georgia,serif;font-size:20px;line-height:1.5;color:#3b5041;margin:0 0 24px;">A small code model.<br/>A little room for possibility.</p>
|
| 27 |
+
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:scale-[1.01] hover:brightness-[102%] focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" href="https://huggingface.co/spaces/philipjohnbasile/wisp-coder-demo" style="display:inline-block;color:#fffdf5;padding:13px 19px;text-decoration:none;font-size:14px;font-weight:600;text-decoration-thickness:1px;text-underline-offset:3px;background:#263f32;border-bottom:3px solid #a1b68c;">Enter the playground <span class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-hover:translate-x-0.5" style="display:inline-block;">↗</span></a>
|
| 28 |
+
<p style="font-size:11px;line-height:1.7;color:#50614e;margin:22px 0 0;">An open research experiment<br/>by Philip John Basile</p>
|
| 29 |
</div>
|
| 30 |
+
<div style="flex:1 1 270px;min-width:0;max-width:100%;padding:12px 8px;background:transparent;">
|
| 31 |
+
<img alt="Original painted wildflowers in pink, coral, cobalt and green. They gently turn when you hover over the cover." class="wisp-botanical motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-hover:scale-105 group-hover:rotate-6" src="./assets/wisp-wildflowers.png" style="width:100%;max-width:100%;height:auto;object-fit:contain;margin:0;border:0;"/>
|
| 32 |
<p style="font-family:Georgia,serif;font-style:italic;color:#5d704f;font-size:14px;text-align:right;margin:20px 5px 0;">Small model. Open questions.</p>
|
| 33 |
</div>
|
| 34 |
</div>
|
| 35 |
+
<div style="display:flex;flex-wrap:wrap;gap:14px 25px;margin-top:34px;padding-top:22px;background:transparent;border-top:1px solid #bfccb1;">
|
| 36 |
+
<div style=";background:transparent;"><span style="font-family:Georgia,serif;font-size:26px;color:#263f32;">100.7M</span><br/><span style="font-size:10px;letter-spacing:1px;color:#50614e;">TRUNK PARAMETERS</span></div>
|
| 37 |
+
<div style=";background:transparent;"><span style="font-family:Georgia,serif;font-size:26px;color:#ad3f5a;">108.2M</span><br/><span style="font-size:10px;letter-spacing:1px;color:#50614e;">INCLUDING MTP</span></div>
|
| 38 |
+
<div style=";background:transparent;"><span style="font-family:Georgia,serif;font-size:26px;color:#263f32;">2,048</span><br/><span style="font-size:10px;letter-spacing:1px;color:#50614e;">TOKEN CONTEXT</span></div>
|
| 39 |
</div>
|
| 40 |
</div>
|
| 41 |
+
<div id="wisp-index" style="display:flex;flex-wrap:wrap;gap:10px 18px;margin:0;background:#f4f1e8;padding:16px 0;border-bottom:1px solid #bfccb1;">
|
| 42 |
+
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" href="#start-here" style="display:inline-block;padding:0;color:#354b3b;font-size:12px;text-decoration:none;text-decoration-thickness:1px;text-underline-offset:3px;border:0;"><span style="color:#ad3f5a;">01</span> Start here</a>
|
| 43 |
+
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" href="#architecture" style="display:inline-block;padding:0;color:#354b3b;font-size:12px;text-decoration:none;text-decoration-thickness:1px;text-underline-offset:3px;border:0;"><span style="color:#ad3f5a;">02</span> Architecture</a>
|
| 44 |
+
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" href="#usage" style="display:inline-block;padding:0;color:#354b3b;font-size:12px;text-decoration:none;text-decoration-thickness:1px;text-underline-offset:3px;border:0;"><span style="color:#ad3f5a;">03</span> Usage</a>
|
| 45 |
+
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" href="#evaluation" style="display:inline-block;padding:0;color:#354b3b;font-size:12px;text-decoration:none;text-decoration-thickness:1px;text-underline-offset:3px;border:0;"><span style="color:#ad3f5a;">04</span> Evidence</a>
|
| 46 |
+
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" href="#limitations" style="display:inline-block;padding:0;color:#354b3b;font-size:12px;text-decoration:none;text-decoration-thickness:1px;text-underline-offset:3px;border:0;"><span style="color:#ad3f5a;">05</span> Limits</a>
|
| 47 |
+
</div><div id="wisp-path" style="margin:0 0 0 6px;padding:0 0 0 clamp(14px,2vw,23px);border-left:1px solid #becdad;">
|
| 48 |
+
|
| 49 |
+
<div style="margin:0 0 24px;padding:44px 0 0;background:transparent;position:relative;"><span aria-hidden="true" style="position:absolute;left:calc(-1 * clamp(14px,2vw,23px) - 7px);top:43px;color:#ad3f5a;background:#f4f1e8;font-size:14px;line-height:1;">❋</span>
|
| 50 |
+
<p style="margin:0 0 12px;color:#9f3d58;font-size:10px;letter-spacing:1.6px;text-transform:uppercase;">01 / START HERE</p>
|
| 51 |
+
<h2 id="start-here" style="font-family:Georgia,serif;font-size:clamp(32px,4.3vw,46px);line-height:1.08;font-weight:400;letter-spacing:-1.6px;color:#263f32;margin:0;padding:0;border:0;scroll-margin-top:24px;">At the cursor.</h2>
|
| 52 |
</div>
|
| 53 |
|
| 54 |
+
<div id="wisp-cursor" style="color:#263f32;padding:0;margin:0 0 30px;background:transparent;">
|
| 55 |
+
<p style="font-size:9px;letter-spacing:1.1px;color:#64775a;margin:0 0 9px;">THE CODE BEFORE. THE POSSIBILITY WITHIN. THE CODE AFTER.</p>
|
| 56 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 57 |
|
| 58 |
+
<div style="display:flex;align-items:flex-start;flex-wrap:wrap;gap:10px;background:transparent;">
|
| 59 |
+
<details class="group" id="wisp-prefix" open="" style="flex:1 1 150px;min-width:0;color:#263f32;padding:14px 10px 0 0;border-top:2px solid #b7c8a6;">
|
| 60 |
+
<summary class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:0;color:#263f32;">
|
| 61 |
+
<span style="display:block;font-size:10px;letter-spacing:1px;color:#354b3b;margin:0 0 10px;">YOU PROVIDE</span>
|
| 62 |
+
<span style="display:flex;align-items:center;justify-content:space-between;font-family:Georgia,serif;font-size:24px;color:#354b3b;margin:0 0 6px;">Prefix<span class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-open:rotate-180" style="display:inline-block;font-family:Arial,sans-serif;font-size:18px;color:#354b3b;">⌄</span></span>
|
| 63 |
+
<span style="display:block;font-size:13px;line-height:1.6;color:#354b3b;">Code before the gap</span>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 64 |
</summary>
|
| 65 |
+
<div style="padding:16px 0 0;background:transparent;"><p style="font-size:13px;line-height:1.7;color:#354b3b;margin:0 0 12px;">The source context before the missing span.</p><code style="display:block;overflow-wrap:anywhere;white-space:normal;background:#e8eddf;color:#263f32;font-size:11px;padding:2px 5px;font-weight:500;"><|fim_prefix|></code></div>
|
| 66 |
</details>
|
| 67 |
+
<details class="group" id="wisp-middle" open="" style="flex:1 1 150px;min-width:0;color:#263f32;padding:14px 10px 0 0;border-top:2px solid #ad3f5a;">
|
| 68 |
+
<summary class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:0;color:#263f32;">
|
| 69 |
+
<span style="display:block;font-size:10px;letter-spacing:1px;color:#963c54;margin:0 0 10px;">WISP SUGGESTS</span>
|
| 70 |
+
<span style="display:flex;align-items:center;justify-content:space-between;font-family:Georgia,serif;font-size:24px;color:#963c54;margin:0 0 6px;">Middle<span class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-open:rotate-180" style="display:inline-block;font-family:Arial,sans-serif;font-size:18px;color:#963c54;">⌄</span></span>
|
| 71 |
+
<span style="display:block;font-size:13px;line-height:1.6;color:#963c54;">The missing span</span>
|
| 72 |
</summary>
|
| 73 |
+
<div style="padding:16px 0 0;background:transparent;"><p style="font-size:13px;line-height:1.7;color:#354b3b;margin:0 0 12px;">The model predicts this span using both sides of the gap. Review its suggestion before use.</p><code style="display:block;overflow-wrap:anywhere;white-space:normal;background:#e8eddf;color:#263f32;font-size:11px;padding:2px 5px;font-weight:500;"><|fim_middle|></code></div>
|
| 74 |
</details>
|
| 75 |
+
<details class="group" id="wisp-suffix" open="" style="flex:1 1 150px;min-width:0;color:#263f32;padding:14px 10px 0 0;border-top:2px solid #b7c8a6;">
|
| 76 |
+
<summary class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:0;color:#263f32;">
|
| 77 |
+
<span style="display:block;font-size:10px;letter-spacing:1px;color:#354b3b;margin:0 0 10px;">YOU PROVIDE</span>
|
| 78 |
+
<span style="display:flex;align-items:center;justify-content:space-between;font-family:Georgia,serif;font-size:24px;color:#354b3b;margin:0 0 6px;">Suffix<span class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-open:rotate-180" style="display:inline-block;font-family:Arial,sans-serif;font-size:18px;color:#354b3b;">⌄</span></span>
|
| 79 |
+
<span style="display:block;font-size:13px;line-height:1.6;color:#354b3b;">Code after the gap</span>
|
| 80 |
</summary>
|
| 81 |
+
<div style="padding:16px 0 0;background:transparent;"><p style="font-size:13px;line-height:1.7;color:#354b3b;margin:0 0 12px;">The source context after the missing span. In PSM prompt order, this comes before the generated middle.</p><code style="display:block;overflow-wrap:anywhere;white-space:normal;background:#e8eddf;color:#263f32;font-size:11px;padding:2px 5px;font-weight:500;"><|fim_suffix|></code></div>
|
| 82 |
</details>
|
| 83 |
</div>
|
| 84 |
+
<p style="font-size:13px;line-height:1.7;color:#354b3b;margin:18px 0 0;">Provide the code on both sides of a gap. Wisp predicts the middle. <a href="https://huggingface.co/spaces/philipjohnbasile/wisp-coder-demo" style="color:#963c54;text-decoration:underline;text-decoration-thickness:1px;text-underline-offset:3px;">Explore it in the playground ↗</a></p>
|
| 85 |
</div>
|
| 86 |
+
<p style="font-size:10px;letter-spacing:1px;margin:30px 0 0;color:#64775a;">A FEW THINGS TO KNOW ALONG THE WAY</p>
|
| 87 |
+
<div style="display:block;grid-template-columns:repeat(auto-fit,minmax(min(100%,250px),1fr));align-items:start;gap:14px;margin:0 0 30px;background:transparent;">
|
| 88 |
+
<details class="group" id="wisp-note-1" open="" style="color:#263f32;padding:18px 0 0;margin:0;border-bottom:1px solid #d4ddc8;">
|
| 89 |
+
<summary class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:0 0 10px;color:#263f32;">
|
| 90 |
+
<span style="display:flex;justify-content:space-between;align-items:center;font-size:14px;letter-spacing:1.6px;color:#8e3049;margin:0;float:right;"><span class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-open:rotate-180" style="font-size:17px;display:inline-block;">⌄</span></span>
|
| 91 |
+
<span style="display:block;font-family:Georgia,serif;font-size:22px;font-weight:400;line-height:1.3;color:#263f32;">What is it for?</span>
|
|
|
|
| 92 |
</summary>
|
| 93 |
+
<div style="padding:0 0 18px;background:transparent;"><p style="font-size:14px;line-height:1.75;color:#354b3b;margin:0;border-top:1px dashed #aebaa0;padding-top:15px;padding:0;border:0;">Research on code completion, fill-in-the-middle (FIM), and native multi-token prediction (MTP).</p></div>
|
| 94 |
</details>
|
| 95 |
+
<details class="group" id="wisp-note-2" open="" style="color:#263f32;padding:18px 0 0;margin:0;border-bottom:1px solid #d4ddc8;">
|
| 96 |
+
<summary class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:0 0 10px;color:#263f32;">
|
| 97 |
+
<span style="display:flex;justify-content:space-between;align-items:center;font-size:14px;letter-spacing:1.6px;color:#8e3049;margin:0;float:right;"><span class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-open:rotate-180" style="font-size:17px;display:inline-block;">⌄</span></span>
|
| 98 |
+
<span style="display:block;font-family:Georgia,serif;font-size:22px;font-weight:400;line-height:1.3;color:#263f32;">What did I build?</span>
|
| 99 |
</summary>
|
| 100 |
+
<div style="padding:0 0 18px;background:transparent;"><p style="font-size:14px;line-height:1.75;color:#354b3b;margin:0;border-top:1px dashed #aebaa0;padding-top:15px;padding:0;border:0;">Philip John Basile's model trained from step zero: 100.7M trunk parameters, 108.2M including the MTP module.</p></div>
|
| 101 |
</details>
|
| 102 |
+
<details class="group" id="wisp-note-3" open="" style="color:#263f32;padding:18px 0 0;margin:0;border-bottom:1px solid #d4ddc8;">
|
| 103 |
+
<summary class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:0 0 10px;color:#263f32;">
|
| 104 |
+
<span style="display:flex;justify-content:space-between;align-items:center;font-size:14px;letter-spacing:1.6px;color:#8e3049;margin:0;float:right;"><span class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-open:rotate-180" style="font-size:17px;display:inline-block;">⌄</span></span>
|
| 105 |
+
<span style="display:block;font-family:Georgia,serif;font-size:22px;font-weight:400;line-height:1.3;color:#263f32;">What runs it?</span>
|
| 106 |
</summary>
|
| 107 |
+
<div style="padding:0 0 18px;background:transparent;"><p style="font-size:14px;line-height:1.75;color:#354b3b;margin:0;border-top:1px dashed #aebaa0;padding-top:15px;padding:0;border:0;">The trunk uses standard Transformers or MLX-LM. MTP requires the explicit packaged MLX reference runtime; generic loaders do not activate the sidecar.</p></div>
|
| 108 |
</details>
|
| 109 |
+
<details class="group" id="wisp-note-4" open="" style="color:#263f32;padding:18px 0 0;margin:0;border-bottom:1px solid #d4ddc8;">
|
| 110 |
+
<summary class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:0 0 10px;color:#263f32;">
|
| 111 |
+
<span style="display:flex;justify-content:space-between;align-items:center;font-size:14px;letter-spacing:1.6px;color:#8e3049;margin:0;float:right;"><span class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-open:rotate-180" style="font-size:17px;display:inline-block;">⌄</span></span>
|
| 112 |
+
<span style="display:block;font-family:Georgia,serif;font-size:22px;font-weight:400;line-height:1.3;color:#263f32;">What was checked recently?</span>
|
| 113 |
</summary>
|
| 114 |
+
<div style="padding:0 0 18px;background:transparent;"><p style="font-size:14px;line-height:1.75;color:#354b3b;margin:0;border-top:1px dashed #aebaa0;padding-top:15px;padding:0;border:0;">CPU float32 completions with PyTorch 2.13.0 and Transformers 5.14.1 on 2026-09-10. These are functional smokes, not a coding-quality or latency benchmark.</p></div>
|
| 115 |
</details>
|
| 116 |
+
<details class="group" id="wisp-note-5" open="" style="color:#263f32;padding:18px 0 0;margin:0;border-bottom:1px solid #d4ddc8;">
|
| 117 |
+
<summary class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:0 0 10px;color:#263f32;">
|
| 118 |
+
<span style="display:flex;justify-content:space-between;align-items:center;font-size:14px;letter-spacing:1.6px;color:#8e3049;margin:0;float:right;"><span class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-open:rotate-180" style="font-size:17px;display:inline-block;">⌄</span></span>
|
| 119 |
+
<span style="display:block;font-family:Georgia,serif;font-size:22px;font-weight:400;line-height:1.3;color:#263f32;">How much memory?</span>
|
| 120 |
</summary>
|
| 121 |
+
<div style="padding:0 0 18px;background:transparent;"><p style="font-size:14px;line-height:1.75;color:#354b3b;margin:0;border-top:1px dashed #aebaa0;padding-top:15px;padding:0;border:0;">Trunk weight file: about 201 MB in BF16, or about 403 MB as FP32 tensors. Runtime overhead and the KV cache require additional RAM; these are weight sizes, not total-memory requirements.</p></div>
|
| 122 |
</details>
|
| 123 |
+
<details class="group" id="wisp-note-6" open="" style="color:#263f32;padding:18px 0 0;margin:0;border-bottom:1px solid #d4ddc8;">
|
| 124 |
+
<summary class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;list-style:none;padding:0 0 10px;color:#263f32;">
|
| 125 |
+
<span style="display:flex;justify-content:space-between;align-items:center;font-size:14px;letter-spacing:1.6px;color:#8e3049;margin:0;float:right;"><span class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-open:rotate-180" style="font-size:17px;display:inline-block;">⌄</span></span>
|
| 126 |
+
<span style="display:block;font-family:Georgia,serif;font-size:22px;font-weight:400;line-height:1.3;color:#263f32;">What are the limits?</span>
|
| 127 |
</summary>
|
| 128 |
+
<div style="padding:0 0 18px;background:transparent;"><p style="font-size:14px;line-height:1.75;color:#354b3b;margin:0;border-top:1px dashed #aebaa0;padding-top:15px;padding:0;border:0;">2,048-token context; limited completion quality; no chat tuning. The registered short-hole comparison established no superiority, and no production MTP speedup is claimed.</p></div>
|
| 129 |
</details>
|
| 130 |
</div>
|
| 131 |
+
<h3 id="quick-start--ordinary-completion-on-cpu" style="color:#263f32;font-family:Georgia,serif;font-size:25px;font-weight:400;line-height:1.3;margin:30px 0 16px;">Quick start — ordinary completion on CPU</h3>
|
| 132 |
+
<details class="group" open="" style="margin:24px 0;padding:0;color:#263f32;border-top:1px solid #bdcdb0;">
|
| 133 |
+
<summary class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;padding:14px 0;color:#536d43;font-family:Georgia,serif;font-size:19px;background:transparent;">Run an ordinary completion on CPU</summary>
|
| 134 |
+
<div style="padding:0;background:transparent;">
|
| 135 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 136 |
|
| 137 |
```bash
|
| 138 |
pip install "torch==2.13.0" "transformers==5.14.1"
|
| 139 |
```
|
| 140 |
|
| 141 |
+
|
| 142 |
+
|
| 143 |
+
|
| 144 |
```python
|
| 145 |
import torch
|
| 146 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
| 158 |
print(tokenizer.decode(output[0], skip_special_tokens=True))
|
| 159 |
```
|
| 160 |
|
| 161 |
+
|
| 162 |
</div>
|
| 163 |
</details>
|
| 164 |
+
<p style="color:#354b3b;">Review generated code before using it. The demo also provides FIM prompts and displays raw suggestions without executing them. The full training, data-provenance, and evaluation disclosures follow.</p>
|
| 165 |
+
<h3 id="the-idea" style="color:#263f32;font-family:Georgia,serif;font-size:25px;font-weight:400;line-height:1.3;margin:30px 0 16px;">The idea</h3>
|
| 166 |
+
<p style="color:#354b3b;">A small code model built for one job: completing code at the cursor. It targets
|
| 167 |
+
editor inline suggestions, but no production-runtime latency claim is made.</p>
|
| 168 |
+
<p style="color:#354b3b;">Two things make it unusual, and they are the reason it exists rather than a
|
| 169 |
+
feature list.</p>
|
| 170 |
+
<p style="color:#354b3b;"><strong style="color:#263f32;">It does fill in the middle natively.</strong> Each tokenized source document was split
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 171 |
into chunks of at most 1024 tokens, and 70 percent of those chunks were
|
| 172 |
independently transformed so the model predicts a middle span given both the
|
| 173 |
prefix and the suffix. This is not a per-document or per-window rate. Cursor
|
| 174 |
completion is the primary format, not an inference-time prompt trick added to an
|
| 175 |
+
ordinary left-to-right checkpoint.</p>
|
| 176 |
+
<p style="color:#354b3b;"><strong style="color:#263f32;">It has multi-token-prediction heads trained in from step zero.</strong> A shared MTP
|
|
|
|
| 177 |
module, applied recursively in the Qwen3-Next style, predicts two or more tokens
|
| 178 |
ahead from the trunk's own hidden state. The LM head is shared between trunk and
|
| 179 |
MTP module, which ties the two output distributions to a common projection. That
|
| 180 |
is a useful architectural choice, not a guarantee that draft and target
|
| 181 |
+
distributions agree; how close they actually are is what acceptance measures.</p>
|
| 182 |
+
<p style="color:#354b3b;"><strong style="color:#263f32;">What is and is not claimed here.</strong> Small models with MTP drafters exist: Google
|
| 183 |
+
publishes <code style="color:#263f32;background:#e8eddf;overflow-wrap:anywhere;white-space:normal;font-weight:500;padding:2px 5px;">gemma-4-E2B-it-assistant</code> and related MTP assistant checkpoints. The
|
| 184 |
+
artifact contribution is the <em>combination</em> at this scale, a small code model
|
|
|
|
| 185 |
trained from step zero with both fill-in-the-middle on most tokenized chunks and a
|
| 186 |
recursively shared MTP module, plus a paired measurement of how suffix information
|
| 187 |
changes draft acceptance. A targeted search found no exact prior measurement, but
|
| 188 |
+
that is not proof of absence. No exclusivity or first-of-its-kind claim is made.</p>
|
| 189 |
+
<div style="margin:0 0 24px;padding:44px 0 0;background:transparent;position:relative;"><span aria-hidden="true" style="position:absolute;left:calc(-1 * clamp(14px,2vw,23px) - 7px);top:43px;color:#ad3f5a;background:#f4f1e8;font-size:14px;line-height:1;">❋</span>
|
| 190 |
+
<p style="margin:0 0 12px;color:#9f3d58;font-size:10px;letter-spacing:1.6px;text-transform:uppercase;">02 / WHAT IT IS NOT</p>
|
| 191 |
+
<h2 id="what-it-is-not" style="font-family:Georgia,serif;font-size:clamp(32px,4.3vw,46px);line-height:1.08;font-weight:400;letter-spacing:-1.6px;color:#263f32;margin:0;padding:0;border:0;scroll-margin-top:24px;">Keep the expectations honest.</h2>
|
|
|
|
| 192 |
</div>
|
| 193 |
+
<p style="color:#354b3b;">It is not a chat model, an instruction-following model, or an agent. It completes
|
| 194 |
+
text. Asking it to refactor a module produces nonsense.</p>
|
| 195 |
+
<p style="color:#354b3b;">It is also not competitive with Qwen2.5-Coder on raw completion quality, and the
|
|
|
|
|
|
|
| 196 |
arithmetic says it cannot be: Qwen2.5-Coder-0.5B saw roughly 5.5 trillion tokens
|
| 197 |
and Wisp saw 5 billion, three orders of magnitude fewer. If you want the best
|
| 198 |
completions available at small scale, use Qwen. Use Wisp to study FIM plus native
|
| 199 |
+
speculative drafting on Apple Silicon, or how those two interact.</p>
|
| 200 |
+
<div class="group" style="margin:0 0 24px;padding:44px 0 0;background:transparent;position:relative;"><span aria-hidden="true" style="position:absolute;left:calc(-1 * clamp(14px,2vw,23px) - 7px);top:43px;color:#ad3f5a;background:#f4f1e8;font-size:14px;line-height:1;">❋</span><img alt="" class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-hover:scale-105 group-hover:-rotate-6" src="./assets/wisp-wildflowers.png" style="width:180px;max-width:45%;height:auto;object-fit:contain;float:right;margin:-5px 0 12px 15px;border:0;"/>
|
| 201 |
+
<p style="margin:0 0 12px;color:#9f3d58;font-size:10px;letter-spacing:1.6px;text-transform:uppercase;">03 / ARCHITECTURE</p>
|
| 202 |
+
<h2 id="architecture" style="font-family:Georgia,serif;font-size:clamp(32px,4.3vw,46px);line-height:1.08;font-weight:400;letter-spacing:-1.6px;color:#263f32;margin:0;padding:0;border:0;scroll-margin-top:24px;">Under the petals.</h2>
|
|
|
|
| 203 |
</div>
|
| 204 |
+
<div style="display:grid;grid-template-columns:repeat(auto-fit,minmax(min(100%,250px),1fr));gap:14px;margin:24px 0;background:transparent;">
|
| 205 |
+
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" href="https://huggingface.co/philipjohnbasile/wisp-coder-110m/blob/main/config.json" style="display:block;padding:10px 0 20px;color:#263f32;text-decoration:none;text-decoration-thickness:1px;text-underline-offset:3px;margin:0;border-bottom:1px solid #bdcdb0;">
|
|
|
|
| 206 |
<span style="display:block;font-size:10px;letter-spacing:1.4px;color:#50614e;margin-bottom:12px;">ROUTE A / THE TRUNK</span><span style="display:block;font-family:Georgia,serif;font-size:27px;line-height:1.2;color:#263f32;margin-bottom:12px;">An ordinary Llama decoder.</span><span style="display:block;font-size:13px;line-height:1.7;color:#354b3b;">100.7M parameters. Standard Transformers and MLX-LM loading.</span><span style="display:block;font-size:12px;color:#8e3049;margin-top:16px;">Inspect the configuration ↗</span>
|
| 207 |
</a>
|
| 208 |
+
<a class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" href="https://huggingface.co/philipjohnbasile/wisp-coder-110m/blob/main/wisp_mtp_reference.py" style="display:block;padding:10px 0 20px;color:#263f32;text-decoration:none;text-decoration-thickness:1px;text-underline-offset:3px;margin:0;border-bottom:1px solid #bdcdb0;">
|
| 209 |
<span style="display:block;font-size:10px;letter-spacing:1.4px;color:#8e3049;margin-bottom:12px;">ROUTE B / NATIVE MTP</span><span style="display:block;font-family:Georgia,serif;font-size:27px;line-height:1.2;color:#263f32;margin-bottom:12px;">A shared drafting module.</span><span style="display:block;font-size:13px;line-height:1.7;color:#354b3b;">108.2M parameters with the sidecar. Use the explicit MLX correctness runtime.</span><span style="display:block;font-size:12px;color:#8e3049;margin-top:16px;">Read the reference runtime ↗</span>
|
| 210 |
</a>
|
| 211 |
</div>
|
| 212 |
+
<p style="font-size:13px;color:#354b3b;"><a href="https://huggingface.co/philipjohnbasile/wisp-coder-110m/blob/main/export_manifest.json" style="color:#963c54;text-decoration-thickness:1px;text-underline-offset:3px;">Export manifest ↗</a> · <a href="https://huggingface.co/philipjohnbasile/wisp-coder-110m/blob/main/mtp_config.json" style="color:#963c54;text-decoration-thickness:1px;text-underline-offset:3px;">MTP configuration ↗</a></p>
|
| 213 |
+
<table style="color:#354b3b;width:100%;max-width:100%;display:block;overflow-x:auto;border:0;border-radius:0;border-collapse:collapse;font-size:13px;margin:24px 0;">
|
| 214 |
+
<thead>
|
| 215 |
+
<tr>
|
| 216 |
+
<th style="color:#963c54;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;font-weight:600;"></th>
|
| 217 |
+
<th style="color:#963c54;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;font-weight:600;"></th>
|
| 218 |
+
</tr>
|
| 219 |
+
</thead>
|
| 220 |
+
<tbody>
|
| 221 |
+
<tr>
|
| 222 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Parameters</td>
|
| 223 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">108.2M total, 100.7M trunk</td>
|
| 224 |
+
</tr>
|
| 225 |
+
<tr>
|
| 226 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Layers</td>
|
| 227 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">12, d_model 768, 12 heads / 4 KV heads, SwiGLU 2048</td>
|
| 228 |
+
</tr>
|
| 229 |
+
<tr>
|
| 230 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Context</td>
|
| 231 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">2048, RoPE theta 100k</td>
|
| 232 |
+
</tr>
|
| 233 |
+
<tr>
|
| 234 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Vocab</td>
|
| 235 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">32,768 byte-level BPE, digits split, FIM sentinels</td>
|
| 236 |
+
</tr>
|
| 237 |
+
<tr>
|
| 238 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">MTP</td>
|
| 239 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">1 shared module (1 transformer block), trained to depth 2, recursive at inference</td>
|
| 240 |
+
</tr>
|
| 241 |
+
<tr>
|
| 242 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Precision</td>
|
| 243 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">bfloat16 weights, trained with float32 master weights</td>
|
| 244 |
+
</tr>
|
| 245 |
+
</tbody>
|
| 246 |
+
</table>
|
| 247 |
+
<p style="color:#354b3b;">The trunk is an ordinary Llama decoder. It loads in <code style="color:#263f32;background:#e8eddf;overflow-wrap:anywhere;white-space:normal;font-weight:500;padding:2px 5px;">transformers</code> and <code style="color:#263f32;background:#e8eddf;overflow-wrap:anywhere;white-space:normal;font-weight:500;padding:2px 5px;">mlx_lm</code>
|
| 248 |
+
with no custom modelling code and no <code style="color:#263f32;background:#e8eddf;overflow-wrap:anywhere;white-space:normal;font-weight:500;padding:2px 5px;">trust_remote_code</code>. The MTP module ships
|
| 249 |
+
alongside as <code style="color:#263f32;background:#e8eddf;overflow-wrap:anywhere;white-space:normal;font-weight:500;padding:2px 5px;">mtp.safetensors</code> and is ignorable by runtimes that cannot use it.
|
| 250 |
Generic Llama runtimes do not consume the sidecar automatically. The package
|
| 251 |
+
therefore also ships <code style="color:#263f32;background:#e8eddf;overflow-wrap:anywhere;white-space:normal;font-weight:500;padding:2px 5px;">wisp_mtp_model.py</code> and <code style="color:#263f32;background:#e8eddf;overflow-wrap:anywhere;white-space:normal;font-weight:500;padding:2px 5px;">wisp_mtp_reference.py</code>, a
|
| 252 |
manifest-bound MLX correctness decoder that requires the exact sidecar schema,
|
| 253 |
verifies every declared package hash, exposes the resolved runtime contract and
|
| 254 |
executed route, and fails unless its MTP path reproduces target greedy tokens.
|
| 255 |
+
It recomputes full prefixes and is not a production speed path.</p>
|
| 256 |
+
<p style="color:#354b3b;">The export was verified against <code style="color:#263f32;background:#e8eddf;overflow-wrap:anywhere;white-space:normal;font-weight:500;padding:2px 5px;">transformers</code> numerically, not assumed: relative
|
|
|
|
| 257 |
logit delta 2.16e-3 and argmax agreement 1.0000 against the MLX original on the
|
| 258 |
same tokens, which pins the RoPE convention, the RMSNorm epsilon, and the grouped
|
| 259 |
+
query head order.</p>
|
| 260 |
+
<div style="margin:0 0 24px;padding:44px 0 0;background:transparent;position:relative;"><span aria-hidden="true" style="position:absolute;left:calc(-1 * clamp(14px,2vw,23px) - 7px);top:43px;color:#ad3f5a;background:#f4f1e8;font-size:14px;line-height:1;">❋</span>
|
| 261 |
+
<p style="margin:0 0 12px;color:#9f3d58;font-size:10px;letter-spacing:1.6px;text-transform:uppercase;">04 / TRAINING DATA</p>
|
| 262 |
+
<h2 id="training-data" style="font-family:Georgia,serif;font-size:clamp(32px,4.3vw,46px);line-height:1.08;font-weight:400;letter-spacing:-1.6px;color:#263f32;margin:0;padding:0;border:0;scroll-margin-top:24px;">Where it grew.</h2>
|
|
|
|
| 263 |
</div>
|
| 264 |
+
<p style="color:#354b3b;">The configured target mixture was 92 percent code from
|
| 265 |
+
<code style="color:#263f32;background:#e8eddf;overflow-wrap:anywhere;white-space:normal;font-weight:500;padding:2px 5px;">bigcode/starcoderdata</code> and 8 percent <code style="color:#263f32;background:#e8eddf;overflow-wrap:anywhere;white-space:normal;font-weight:500;padding:2px 5px;">HuggingFaceFW/fineweb-edu</code>. This is a
|
|
|
|
| 266 |
source percentage, not a permissive-license percentage. StarCoderData declares
|
| 267 |
+
<code style="color:#263f32;background:#e8eddf;overflow-wrap:anywhere;white-space:normal;font-weight:500;padding:2px 5px;">license: other</code>, and its original repository terms and relevant attribution
|
| 268 |
clauses remain applicable. FineWeb-Edu is ODC-By 1.0 and remains subject to
|
| 269 |
Common Crawl terms. Run 1 retained no row-level source manifest, so no per-file
|
| 270 |
+
licensing or attribution guarantee is made.</p>
|
| 271 |
+
<details class="group" open="" style="margin:24px 0;padding:0;color:#263f32;border-top:1px solid #bdcdb0;">
|
| 272 |
+
<summary class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;padding:14px 0;color:#536d43;font-family:Georgia,serif;font-size:19px;background:transparent;">Training-data provenance notes</summary>
|
| 273 |
+
<div style="padding:0;background:transparent;">
|
| 274 |
+
<p style="color:#354b3b;">Every run 1 document received structural size, line-length, and character
|
| 275 |
+
distribution filters. Because the loader expected <code style="color:#263f32;background:#e8eddf;overflow-wrap:anywhere;white-space:normal;font-weight:500;padding:2px 5px;">path</code> instead of
|
| 276 |
+
StarCoderData's <code style="color:#263f32;background:#e8eddf;overflow-wrap:anywhere;white-space:normal;font-weight:500;padding:2px 5px;">max_stars_repo_path</code>, Python AST and JSON extension filtering
|
|
|
|
|
|
|
| 277 |
did not activate for run 1. The schema-aware repair applies only to future
|
| 278 |
source streaming, not to these weights. Corpus content was never executed.
|
| 279 |
+
Read the published <a href="https://huggingface.co/philipjohnbasile/wisp-coder-110m/blob/main/evidence/documents/DATA.md" style="color:#963c54;text-decoration-thickness:1px;text-underline-offset:3px;">training-data disclosure</a>
|
| 280 |
+
and <a href="https://huggingface.co/philipjohnbasile/wisp-coder-110m/blob/main/evidence/receipts/training_data_receipt.json" style="color:#963c54;text-decoration-thickness:1px;text-underline-offset:3px;">machine-checked training-data receipt</a>.</p>
|
| 281 |
+
<p style="color:#354b3b;">The 32K tokenizer was trained on 400,000 documents sampled round-robin across
|
|
|
|
| 282 |
the eleven source entries, not according to the later token-budget weights.
|
| 283 |
Current shard bytes are fully attested after the build, but the receipt is not a
|
| 284 |
+
raw-row manifest and cannot prove exact original example boundaries.</p>
|
|
|
|
| 285 |
</div>
|
| 286 |
</details>
|
| 287 |
+
<p style="color:#354b3b;">The Apache 2.0 metadata describes the released Wisp artifact. It does not
|
| 288 |
+
override training-source terms or licenses applicable to generated code.</p>
|
| 289 |
+
<div style="margin:0 0 24px;padding:44px 0 0;background:transparent;position:relative;"><span aria-hidden="true" style="position:absolute;left:calc(-1 * clamp(14px,2vw,23px) - 7px);top:43px;color:#ad3f5a;background:#f4f1e8;font-size:14px;line-height:1;">❋</span>
|
| 290 |
+
<p style="margin:0 0 12px;color:#9f3d58;font-size:10px;letter-spacing:1.6px;text-transform:uppercase;">05 / USAGE</p>
|
| 291 |
+
<h2 id="usage" style="font-family:Georgia,serif;font-size:clamp(32px,4.3vw,46px);line-height:1.08;font-weight:400;letter-spacing:-1.6px;color:#263f32;margin:0;padding:0;border:0;scroll-margin-top:24px;">Give it a few lines.</h2>
|
|
|
|
|
|
|
| 292 |
</div>
|
| 293 |
+
<p style="color:#354b3b;">Plain completion:</p>
|
| 294 |
|
|
|
|
| 295 |
|
| 296 |
```python
|
| 297 |
from transformers import AutoModelForCausalLM, AutoTokenizer
|
|
|
|
| 302 |
max_new_tokens=64)[0]))
|
| 303 |
```
|
| 304 |
|
| 305 |
+
|
| 306 |
+
<p style="color:#354b3b;">Fill in the middle. The model was trained with both orderings, evenly split:</p>
|
| 307 |
+
|
| 308 |
|
| 309 |
```python
|
| 310 |
# PSM: prefix, suffix, then generate the middle
|
| 311 |
prompt = f"<|fim_prefix|>{prefix}<|fim_suffix|>{suffix}<|fim_middle|>"
|
| 312 |
```
|
| 313 |
|
| 314 |
+
|
| 315 |
+
<p style="color:#354b3b;">Exercise the native sidecar explicitly. This is a correctness check, not a
|
| 316 |
+
latency benchmark:</p>
|
| 317 |
+
|
| 318 |
|
| 319 |
```python
|
| 320 |
import os
|
|
|
|
| 336 |
print(result["mtp_route"])
|
| 337 |
```
|
| 338 |
|
| 339 |
+
|
| 340 |
+
<p style="color:#354b3b;">For evaluation, the trunk can be configured as Continue.dev's autocomplete model
|
| 341 |
while leaving a larger model as the chat and agent backend. Target-runtime batch-1
|
| 342 |
latency and editor user experience have not yet been measured, so this is not a
|
| 343 |
+
sub-100 ms claim.</p>
|
| 344 |
+
<div style="margin:0 0 24px;padding:44px 0 0;background:transparent;position:relative;"><span aria-hidden="true" style="position:absolute;left:calc(-1 * clamp(14px,2vw,23px) - 7px);top:43px;color:#ad3f5a;background:#f4f1e8;font-size:14px;line-height:1;">❋</span>
|
| 345 |
+
<p style="margin:0 0 12px;color:#9f3d58;font-size:10px;letter-spacing:1.6px;text-transform:uppercase;">06 / EVALUATION</p>
|
| 346 |
+
<h2 id="evaluation" style="font-family:Georgia,serif;font-size:clamp(32px,4.3vw,46px);line-height:1.08;font-weight:400;letter-spacing:-1.6px;color:#263f32;margin:0;padding:0;border:0;scroll-margin-top:24px;">Follow the evidence.</h2>
|
|
|
|
| 347 |
</div>
|
| 348 |
+
<div style="display:grid;grid-template-columns:repeat(auto-fit,minmax(min(100%,190px),1fr));gap:12px;margin:0 0 24px;background:transparent;">
|
| 349 |
+
<div style="padding:12px 0 18px;color:#263f32;background:transparent;border-top:1px solid #bdcdb0;"><p style="font-size:10px;letter-spacing:1px;color:#4a603d;margin:0 0 9px;">FUNCTIONAL CHECKS</p><p style="font-family:Georgia,serif;font-size:23px;line-height:1.2;color:#263f32;margin:0 0 10px;">CPU completions.</p><p style="font-size:13px;line-height:1.7;color:#354b3b;margin:0;">CPU completion smokes are documented above. They do not measure coding quality or latency.</p></div>
|
| 350 |
+
<div style="padding:12px 0 18px;color:#263f32;background:transparent;border-top:1px solid #bdcdb0;"><p style="font-size:10px;letter-spacing:1px;color:#8e3049;margin:0 0 9px;">SHORT-HOLE COMPARISON</p><p style="font-family:Georgia,serif;font-size:23px;line-height:1.2;color:#263f32;margin:0 0 10px;">No superiority.</p><p style="font-size:13px;line-height:1.7;color:#354b3b;margin:0;">No superiority was established. <a href="https://huggingface.co/philipjohnbasile/wisp-coder-110m/blob/main/fim-competitive-v1/FIM_RESULTS.md" style="color:#963c54;text-decoration-thickness:1px;text-underline-offset:3px;">Read the comparison ↗</a></p></div>
|
| 351 |
+
<div style="padding:12px 0 18px;color:#263f32;background:transparent;border-top:1px solid #bdcdb0;"><p style="font-size:10px;letter-spacing:1px;color:#8e3049;margin:0 0 9px;">MTP RUNTIME</p><p style="font-family:Georgia,serif;font-size:23px;line-height:1.2;color:#263f32;margin:0 0 10px;">Speed is open.</p><p style="font-size:13px;line-height:1.7;color:#354b3b;margin:0;">No production MTP speedup is claimed. The packaged MLX runtime checks correctness.</p></div>
|
|
|
|
| 352 |
</div>
|
| 353 |
+
<h3 id="registered-final-results" style="color:#263f32;font-family:Georgia,serif;font-size:25px;font-weight:400;line-height:1.3;margin:30px 0 16px;">Registered final results</h3>
|
| 354 |
+
<details class="group" open="" style="margin:24px 0;padding:0;color:#263f32;border-top:1px solid #bdcdb0;">
|
| 355 |
+
<summary class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;padding:14px 0;color:#536d43;font-family:Georgia,serif;font-size:19px;background:transparent;">Complete registered results</summary>
|
| 356 |
+
<div style="padding:0;background:transparent;">
|
| 357 |
+
<table style="color:#354b3b;width:100%;max-width:100%;display:block;overflow-x:auto;border:0;border-radius:0;border-collapse:collapse;font-size:13px;margin:24px 0;">
|
| 358 |
+
<thead>
|
| 359 |
+
<tr>
|
| 360 |
+
<th style="color:#963c54;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;font-weight:600;">Measurement</th>
|
| 361 |
+
<th style="color:#963c54;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;font-weight:600;">Result</th>
|
| 362 |
+
</tr>
|
| 363 |
+
</thead>
|
| 364 |
+
<tbody>
|
| 365 |
+
<tr>
|
| 366 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Final validation main NLL</td>
|
| 367 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">1.3873 [1.3428, 1.4335], perplexity 4.00</td>
|
| 368 |
+
</tr>
|
| 369 |
+
<tr>
|
| 370 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Validation MTP depth 1 NLL</td>
|
| 371 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">1.5727 [1.5236, 1.6234]</td>
|
| 372 |
+
</tr>
|
| 373 |
+
<tr>
|
| 374 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Validation MTP depth 2 NLL</td>
|
| 375 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">1.6513 [1.6013, 1.7028]</td>
|
| 376 |
+
</tr>
|
| 377 |
+
<tr>
|
| 378 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">FIM / shuffled-suffix acceptance, depth 2</td>
|
| 379 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">1.0268 [1.0209, 1.0331], POSITIVE, 200 documents</td>
|
| 380 |
+
</tr>
|
| 381 |
+
<tr>
|
| 382 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Trained minus initialized acceptance-ratio lift</td>
|
| 383 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">+0.0268 [+0.0210, +0.0331], CLEARS_CONTROL</td>
|
| 384 |
+
</tr>
|
| 385 |
+
<tr>
|
| 386 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Acceptance interpretation</td>
|
| 387 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">POSITIVE_AND_CLEARS_CONTROL</td>
|
| 388 |
+
</tr>
|
| 389 |
+
<tr>
|
| 390 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">FIM-training effect on shuffled-FIM minus L2R acceptance</td>
|
| 391 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">+0.0814 [+0.0747, +0.0883], POSITIVE, 200 documents</td>
|
| 392 |
+
</tr>
|
| 393 |
+
<tr>
|
| 394 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">FIM-training effect on true-suffix minus shuffled-suffix acceptance</td>
|
| 395 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">+0.0049 [-0.0026, +0.0122], NULL: the interval includes zero</td>
|
| 396 |
+
</tr>
|
| 397 |
+
<tr>
|
| 398 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Adaptive minus fixed accepted drafts per verification</td>
|
| 399 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">+0.0000 [+0.0000, +0.0000], NULL: the interval includes 0, 60 test documents</td>
|
| 400 |
+
</tr>
|
| 401 |
+
<tr>
|
| 402 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Adaptive minus fixed output tokens per target forward</td>
|
| 403 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">+0.0000 [+0.0000, +0.0000], NULL: the interval includes 0, 60 test documents</td>
|
| 404 |
+
</tr>
|
| 405 |
+
<tr>
|
| 406 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Adaptive minus fixed drafts issued per output token</td>
|
| 407 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">+0.0000 [+0.0000, +0.0000], NULL: the interval includes 0, 60 test documents (issuance proxy)</td>
|
| 408 |
+
</tr>
|
| 409 |
+
<tr>
|
| 410 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Adaptive minus fixed draft recursions per output token</td>
|
| 411 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">+0.0000 [+0.0000, +0.0000], NULL: the interval includes 0, 60 test documents</td>
|
| 412 |
+
</tr>
|
| 413 |
+
<tr>
|
| 414 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Rollout policies selected on calibration</td>
|
| 415 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">adaptive_h0.7 versus fixed_d4</td>
|
| 416 |
+
</tr>
|
| 417 |
+
<tr>
|
| 418 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Branch-local greedy replay</td>
|
| 419 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Exact argmax for all 38400 emitted tokens across 600 scored policy-document rollouts</td>
|
| 420 |
+
</tr>
|
| 421 |
+
<tr>
|
| 422 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Cross-policy output identity</td>
|
| 423 |
+
<td style="color:#263f32;border:0;border-bottom:1px solid #cbd4bf;padding:10px 8px;vertical-align:top;text-align:left;line-height:1.65;">Identical realized output branches for all 100 calibration/test documents across compared policies</td>
|
| 424 |
+
</tr>
|
| 425 |
+
</tbody>
|
| 426 |
+
</table>
|
| 427 |
</div>
|
| 428 |
</details>
|
| 429 |
+
<p style="color:#354b3b;">Validation intervals measure Monte Carlo uncertainty from the frozen random-window sampler. Acceptance and rollout intervals resample paired target documents. They do not measure training-run or model uncertainty. Null and negative outcomes are retained rather than filtered from the release.</p>
|
| 430 |
+
<details class="group" open="" style="margin:24px 0;padding:0;color:#263f32;border-top:1px solid #bdcdb0;">
|
| 431 |
+
<summary class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] hover:translate-x-px focus-visible:outline-2 focus-visible:outline-offset-2 focus-visible:outline-blue-500" style="cursor:pointer;padding:14px 0;color:#536d43;font-family:Georgia,serif;font-size:19px;background:transparent;">Read the sampler, replay and provenance notes</summary>
|
| 432 |
+
<div style="padding:0;background:transparent;">
|
| 433 |
+
<p style="color:#354b3b;">Rollout endpoint scope: The primary endpoint measures accepted drafts per verification. It does not establish verification-width cost or deployment latency. Draft recursions per output token is the registered drafter-work companion; issued drafts per output token is retained only as an issuance proxy. Target forwards exclude the added post-hoc branch-replay forward and independent verification pass.</p>
|
| 434 |
+
<p style="color:#354b3b;">Independent replay provenance scope: Unsigned local attestation bound to a pushed pre-execution receipt commit and the registered source, inputs, checkpoint, and argv; it is not a signed external or trusted-execution witness.</p>
|
| 435 |
+
<p style="color:#354b3b;">Format-ablation provenance limitation: This is repository-revision evidence, not a per-document raw-corpus manifest for run 1. Source drift between the run 1 build and the captured cache refs cannot be ruled out.</p>
|
| 436 |
+
<p style="color:#354b3b;">Format-ablation runtime limitation: Run 1 did not record source-file hashes or sampler state in its pre-fix checkpoints. A post-build receipt now hashes all 52 current shard files and binds deterministic visible-grammar normalization, but it does not prove that the bytes were unchanged since training began or recover exact original units. The step-300 process recovery reset the legacy sampler, causing 78,643,200 scheduled token positions, 1.57 percent of the training budget, to replay earlier random windows. sampler_reset_steps [300] matches that reset timing in both arms. Later checkpoints preserve exact sampler RNG state. Exact sampled-example equivalence still cannot be proven.</p>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 437 |
</div>
|
| 438 |
</details>
|
| 439 |
+
<p style="color:#354b3b;">One control worth repeating here, because acceptance rate is easy to report
|
| 440 |
+
dishonestly: <strong style="color:#263f32;">an initialized model scored higher acceptance than a trained one
|
| 441 |
+
in the E0 instrument pilot</strong> (0.482 against 0.331 for a model deliberately
|
|
|
|
| 442 |
overfit on 100k tokens), because two near-uniform distributions agree trivially.
|
| 443 |
Those are pilot diagnostics, not run 1 results. Any final acceptance number
|
| 444 |
+
without its exact initialized floor beside it is uninterpretable.</p>
|
| 445 |
+
<div style="margin:0 0 24px;padding:44px 0 0;background:transparent;position:relative;"><span aria-hidden="true" style="position:absolute;left:calc(-1 * clamp(14px,2vw,23px) - 7px);top:43px;color:#ad3f5a;background:#f4f1e8;font-size:14px;line-height:1;">❋</span>
|
| 446 |
+
<p style="margin:0 0 12px;color:#9f3d58;font-size:10px;letter-spacing:1.6px;text-transform:uppercase;">07 / LIMITATIONS</p>
|
| 447 |
+
<h2 id="limitations" style="font-family:Georgia,serif;font-size:clamp(32px,4.3vw,46px);line-height:1.08;font-weight:400;letter-spacing:-1.6px;color:#263f32;margin:0;padding:0;border:0;scroll-margin-top:24px;">Room to grow.</h2>
|
|
|
|
| 448 |
</div>
|
| 449 |
+
<ul>
|
| 450 |
+
<li>5B training tokens. Small, and it shows on unfamiliar APIs, which it will
|
| 451 |
+
hallucinate confidently.</li>
|
| 452 |
+
<li>2048 token context.</li>
|
| 453 |
+
<li>Ten languages, weighted toward Python, JavaScript, and TypeScript. Weaker
|
| 454 |
+
everywhere else, and untested outside that set.</li>
|
| 455 |
+
<li>No safety tuning of any kind. It is a base completion model trained on public
|
| 456 |
+
GitHub code, and it will reproduce patterns from that data, including insecure
|
| 457 |
+
ones. Review what it writes.</li>
|
| 458 |
+
<li>No row-level training-source manifest, license mapping, or attribution index.
|
| 459 |
+
Exact aggregate train-token totals survive in the recovered final build log,
|
| 460 |
+
but accepted rows, rejection counts, and per-source validation overshoot
|
| 461 |
+
cannot be reconstructed from the shards.</li>
|
| 462 |
+
<li>The tokenizer document sampler was round-robin by source entry rather than
|
| 463 |
+
weighted like the training-token mixture.</li>
|
| 464 |
+
<li>Training windows are sampled with replacement. The registered schedule's
|
| 465 |
+
uniform-interval approximation covers about 62.57 percent of corpus positions
|
| 466 |
+
at least once, rather than exposing every position once.</li>
|
| 467 |
+
<li>The source-stratified validation shard is not repository- or document-disjoint.
|
| 468 |
+
A chunked source document can cross the validation-to-training boundary. The
|
| 469 |
+
separate publication holdout is repository-disjoint.</li>
|
| 470 |
+
<li>The MTP sidecar is not consumed automatically by generic Llama runtimes, and
|
| 471 |
+
no production-runtime MTP latency or speedup is claimed.</li>
|
| 472 |
+
<li>Trained and evaluated on one machine, one seed, one run.</li>
|
| 473 |
+
</ul>
|
| 474 |
+
<div style="margin:0 0 24px;padding:44px 0 0;background:transparent;position:relative;"><span aria-hidden="true" style="position:absolute;left:calc(-1 * clamp(14px,2vw,23px) - 7px);top:43px;color:#ad3f5a;background:#f4f1e8;font-size:14px;line-height:1;">❋</span>
|
| 475 |
+
<p style="margin:0 0 12px;color:#9f3d58;font-size:10px;letter-spacing:1.6px;text-transform:uppercase;">08 / CITATION</p>
|
| 476 |
+
<h2 id="citation" style="font-family:Georgia,serif;font-size:clamp(32px,4.3vw,46px);line-height:1.08;font-weight:400;letter-spacing:-1.6px;color:#263f32;margin:0;padding:0;border:0;scroll-margin-top:24px;">Leave a trail.</h2>
|
| 477 |
</div>
|
| 478 |
|
| 479 |
+
|
| 480 |
```bibtex
|
| 481 |
@software{wisp_coder,
|
| 482 |
title = {Wisp: fill-in-the-middle and native multi-token prediction in a small code model},
|
|
|
|
| 486 |
```
|
| 487 |
|
| 488 |
|
| 489 |
+
</div><p style="color:#354b3b;"><a href="https://huggingface.co/spaces/philipjohnbasile/local-ai-guide" style="color:#963c54;text-decoration-thickness:1px;text-underline-offset:3px;">Explore the model guide</a> · <a href="https://huggingface.co/philipjohnbasile" style="color:#963c54;text-decoration-thickness:1px;text-underline-offset:3px;">All public work</a></p><p style="color:#354b3b;"><strong style="color:#263f32;"><a href="https://huggingface.co/spaces/philipjohnbasile/wisp-coder-demo" style="color:#963c54;text-decoration-thickness:1px;text-underline-offset:3px;">Try the interactive CPU demo</a></strong> ·
|
| 490 |
+
<a href="https://huggingface.co/philipjohnbasile/wisp-coder-110m/blob/main/fim-competitive-v1/FIM_RESULTS.md" style="color:#963c54;text-decoration-thickness:1px;text-underline-offset:3px;">Short-hole evaluation</a> ·
|
| 491 |
+
<a href="https://huggingface.co/collections/philipjohnbasile/start-here-selected-work-6aa309b179b2a196db52a12b" style="color:#963c54;text-decoration-thickness:1px;text-underline-offset:3px;">Selected work</a></p><div class="group" style="margin-top:36px;padding:36px 0 10px;display:flex;align-items:center;flex-wrap:wrap;gap:18px;color:#263f32;background:transparent;margin:30px 0 0;border-top:1px solid #bdcdb0;">
|
| 492 |
+
<img alt="" class="motion-safe:transition-transform motion-safe:duration-[400ms] motion-safe:[transition-timing-function:ease] group-hover:scale-105 group-hover:-rotate-6" src="./assets/wisp-wildflowers.png" style="flex:0 1 180px;width:180px;max-width:100%;height:auto;object-fit:contain;margin:0;border:0;"/>
|
| 493 |
+
<div style="flex:1 1 210px;min-width:0;background:transparent;">
|
| 494 |
<p style="font-size:11px;letter-spacing:1.5px;color:#50614e;margin:0 0 8px;">SMALL MODEL. OPEN QUESTIONS.</p>
|
| 495 |
<p style="font-family:Georgia,serif;font-size:42px;letter-spacing:-2px;line-height:1.15;color:#263f32;margin:0 0 12px;">Stay <em style="color:#ad3f5a;">curious.</em></p>
|
| 496 |
+
<p style="font-size:14px;line-height:1.8;margin:0;color:#354b3b;"><a href="https://huggingface.co/spaces/philipjohnbasile/wisp-coder-demo" style="color:#963c54;text-decoration-thickness:1px;text-underline-offset:3px;">Return to the garden ↗</a> · <a href="https://huggingface.co/philipjohnbasile" style="color:#963c54;text-decoration-thickness:1px;text-underline-offset:3px;">More from Philip</a></p>
|
| 497 |
</div>
|
| 498 |
</div>
|
| 499 |
+
<p style="color:#354b3b;"><sub>The Wisp Fieldbook. Original AI-generated floral artwork shared with the Wisp demo. <a href="https://huggingface.co/philipjohnbasile/wisp-coder-110m/blob/main/assets/ARTWORK.md" style="color:#963c54;text-decoration-thickness:1px;text-underline-offset:3px;">Artwork notes</a>.</sub></p>
|
| 500 |
+
</div>
|