PFlash release files

.gitattributes

@@ -0,0 +1,45 @@
+*.7z filter=lfs diff=lfs merge=lfs -text
+*.arrow filter=lfs diff=lfs merge=lfs -text
+*.bin filter=lfs diff=lfs merge=lfs -text
+*.bz2 filter=lfs diff=lfs merge=lfs -text
+*.ckpt filter=lfs diff=lfs merge=lfs -text
+*.ftz filter=lfs diff=lfs merge=lfs -text
+*.gz filter=lfs diff=lfs merge=lfs -text
+*.h5 filter=lfs diff=lfs merge=lfs -text
+*.joblib filter=lfs diff=lfs merge=lfs -text
+*.lfs.* filter=lfs diff=lfs merge=lfs -text
+*.mlmodel filter=lfs diff=lfs merge=lfs -text
+*.model filter=lfs diff=lfs merge=lfs -text
+*.msgpack filter=lfs diff=lfs merge=lfs -text
+*.npy filter=lfs diff=lfs merge=lfs -text
+*.npz filter=lfs diff=lfs merge=lfs -text
+*.onnx filter=lfs diff=lfs merge=lfs -text
+*.ot filter=lfs diff=lfs merge=lfs -text
+*.parquet filter=lfs diff=lfs merge=lfs -text
+*.pb filter=lfs diff=lfs merge=lfs -text
+*.pickle filter=lfs diff=lfs merge=lfs -text
+*.pkl filter=lfs diff=lfs merge=lfs -text
+*.pt filter=lfs diff=lfs merge=lfs -text
+*.pth filter=lfs diff=lfs merge=lfs -text
+*.rar filter=lfs diff=lfs merge=lfs -text
+*.safetensors filter=lfs diff=lfs merge=lfs -text
+saved_model/**/* filter=lfs diff=lfs merge=lfs -text
+*.tar.* filter=lfs diff=lfs merge=lfs -text
+*.tar filter=lfs diff=lfs merge=lfs -text
+*.tflite filter=lfs diff=lfs merge=lfs -text
+*.tgz filter=lfs diff=lfs merge=lfs -text
+*.wasm filter=lfs diff=lfs merge=lfs -text
+*.xz filter=lfs diff=lfs merge=lfs -text
+*.zip filter=lfs diff=lfs merge=lfs -text
+*.zst filter=lfs diff=lfs merge=lfs -text
+*tfevents* filter=lfs diff=lfs merge=lfs -text
+segment-probe-v13-t0.7-min32.gguf filter=lfs diff=lfs merge=lfs -text
+segment-probe-v13-t0.8-min1.gguf filter=lfs diff=lfs merge=lfs -text
+segment-probe-v13-t0.8-min32.gguf filter=lfs diff=lfs merge=lfs -text
+segment-probe-v13-t0.8-min64.gguf filter=lfs diff=lfs merge=lfs -text
+segment-probe-v14-t0.9-min1.gguf filter=lfs diff=lfs merge=lfs -text
+segment-probe-v14-t0.9-min32.gguf filter=lfs diff=lfs merge=lfs -text
+segment-probe-v14-t0.9-min64.gguf filter=lfs diff=lfs merge=lfs -text
+segment-probe-v14.gguf filter=lfs diff=lfs merge=lfs -text
+segment-probe.gguf filter=lfs diff=lfs merge=lfs -text
+segment-probe-s3-line512.gguf filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,117 @@
+---
+license: other
+language:
+- en
+tags:
+- lucebox
+- pflash
+- prefill-compression
+- gguf
+- qwen3.5
+base_model: Qwen/Qwen3.5-0.8B
+---
+
+# pflash-segment-probe
+
+The segment probe of PFlash: it cuts a long prompt into whole structural units before the relevance head ranks them.
+
+PFlash is Lucebox's prefill compression for long prompts. A small drafter, Qwen3.5-0.8B, reads the whole prompt
+once, and three small components on its activations decide what the large reader model prefills:
+1. [`Lucebox/pflash-segment-probe`](https://huggingface.co/Lucebox/pflash-segment-probe) cuts the prompt into
+   whole units (statements and definitions, sentences, records, chat turns, tool calls);
+2. [`Lucebox/pflash-request-probe`](https://huggingface.co/Lucebox/pflash-request-probe) finds the request,
+   wherever it sits in the prompt;
+3. [`Lucebox/pflash-scorer-head`](https://huggingface.co/Lucebox/pflash-scorer-head) ranks the units against that
+   request.
+
+The highest-ranked units that fit the budget go to the reader in their original order, and the reader prefills
+only those. All three read the same tap, the drafter's block-14 residual (`post_block14_residual_before_block15`),
+from the same single pass.
+
+## How it works
+
+- **Architecture**: LayerNorm(1024) → Linear(1024, 256) → GELU → three linear rows, each with a 5-tap convolution
+  over its logit sequence plus the residual logit; about 263K parameters, 16 tensors. GGUF architecture
+  `segmentprobe`, schema `qwen3_5_0_8b_segment_probe_v2`.
+- **Three outputs per token**:
+  - **unit**: a structural unit starts here. Units are the finest whole pieces: every statement, declaration,
+    definition, comment or decorator block at file or container scope in code; sentences in prose; records, chat
+    turns, diff hunks, tool calls and observations. Nothing inside a function body is a unit cut, so a kept unit is
+    never a truncated function.
+  - **container**: the enclosing region (class, file section, paragraph, session, assistant turn). Exported, not
+    used by the served configuration.
+  - **sub-unit**: a nested definition inside a body, used only to split units that are too long.
+- **Segmentation** (settings in the GGUF): a cut where the unit score passes the threshold 0.8; cuts closer than
+  `min_segment` 32 tokens to the previous one merge forward, so small leaves such as doc comments group with what
+  follows; a unit longer than `max_segment` 512 tokens is split at line boundaries (`oversize_split` =
+  `line_aligned`), preferring sub-unit starts. No parser runs at inference.
+- **Why the threshold is asymmetric**: a missed boundary merges two whole units, which the head can still rank; a
+  cut inside a unit truncates it, which it cannot repair. 0.8 keeps harmful cuts (inside a function body or
+  mid-line) at or below 1 % of cuts on code, tool output, dialogue, records and prose, with boundary precision of
+  0.97 or more.
+
+## Files
+
+| file | sha256 | note |
+|---|---|---|
+| `segment-probe-s3-line512.gguf` | `470b7b0efe3736518dcb4c1ba8eb5b806107007e378a74668c17e26434602e68` | the served file: `max_segment` 512, line-aligned splitting |
+| `segment-probe.gguf` | `bd3f0bff1057ad0c953deb79d4b82415076906494c7bbb47a8a0f8e85433b667` | the same weights and thresholds with `max_segment` 2048; not used in the configuration below |
+
+## Using it
+
+PFlash runs inside the Lucebox server. Download the three components and the drafter, then set:
+
+```bash
+export PFLASH_SCORING_HEAD_GGUF=/path/to/seg-scoring-head.gguf
+export PFLASH_SEGMENT_PROBE_GGUF=/path/to/segment-probe-s3-line512.gguf
+export PFLASH_REQUEST_PROBE_GGUF=/path/to/request-probe.gguf
+export PFLASH_SELECT_SEGMENTS=probe PFLASH_SELECT_SCORE=density PFLASH_SELECT_MODE=budget_only
+export PFLASH_SELECT_QUERY_PROBE=1 PFLASH_QUERY_CANDIDATES=1 PFLASH_QUERY_PIN=512
+export PFLASH_SELECT_QUERY_TOKENS=128 PFLASH_SELECT_QUERY_HEAD=1
+export PFLASH_SELECT_PARAGRAPH_JOIN=0 PFLASH_SELECT_CHUNK_SIZE=256
+export PFLASH_ADMISSION=cache PFLASH_ARRIVE=1 PFLASH_CHAT_RECALL=1
+export PFLASH_CHAT_REBUILD_RATIO=2 PFLASH_CHAT_REBUILD_MIN_TOKENS=65536
+export PFLASH_FALLBACK_SPREAD=0.11 PFLASH_COLD_HISTORY=raw
+# server: --prefill-compression auto --prefill-threshold 16384
+#         --prefill-curve 16384:0.315 32768:0.270 65536:0.270 120000:0.225
+#         --prefill-skip-park --cache-type-k q4_0 --cache-type-v q4_0 --max-ctx 139264
+```
+
+- **Drafter** (required, checked by sha256 at load):
+  [`unsloth/Qwen3.5-0.8B-GGUF`](https://huggingface.co/unsloth/Qwen3.5-0.8B-GGUF) `Qwen3.5-0.8B-BF16.gguf`, sha256
+  `cedf89af31c9041b601fa58303285bc46d99c51baee1b13f5e919626ca526ee5`. Each component refuses to load against any
+  other drafter file.
+- PFlash acts when at least 16,384 prompt tokens are new to the server and less than that is already cached. The
+  reader then prefills 31.5 % of the prompt at 16K tokens, 27 % at 32K–64K and 22.5 % at 120K.
+- When the relevance scores are too flat for the budget, the request is served with full prefill
+  (`PFLASH_FALLBACK_SPREAD`). An agent's uncached earlier conversation is never compressed
+  (`PFLASH_COLD_HISTORY=raw`).
+- Runtime support: [Luce-Org/lucebox PR #746](https://github.com/Luce-Org/lucebox/pull/746).
+
+## Results
+
+Measured with all three components together, reader Qwen3.8-27B IQ4_XS, the configuration above, NVIDIA L40S,
+greedy, each benchmark's official metric:
+
+| set | n | full prefill | PFlash |
+|---|---:|---:|---:|
+| Document QA | 30 | 27 | 27 |
+| LongBench v2 | 12 | 6 | 7 |
+| NoLiMa | 24 | 14 | 14 |
+| InfiniteBench CodeDebug | 30 | 24 | 24 |
+| BRIGHT | 28 | 23 | 23 |
+| needle in a haystack, 16K–120K | 100 | 100 | 100 |
+| **public sets, unique questions** (RULER QA's 15 are inside Document QA) | 224 | 194 | **195** |
+| LongCodeQA, questions that need the code | 114 | 57 | 62 |
+
+None of these differences is outside its 95 % interval. The reader prefills a median 29 % of the prompt, and the
+first token arrives 2.7× sooner on average (mean 30.5 s → 11.2 s over the public requests; 60.4 s → 14.8 s median
+on 82K–98K-token CodeDebug prompts).
+
+## Limits
+
+- Not lossless: a request can still lose something the answer needed.
+- Cold agent histories are served with full prefill, so a resumed agent session gets no speed-up.
+- "List everything matching X" over many similar units (emails, wiki pages) is the hardest case; the flat-score
+  fallback catches most of it.
+- Measured serving one request at a time.

segment-probe-s3-line512.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:470b7b0efe3736518dcb4c1ba8eb5b806107007e378a74668c17e26434602e68
+size 1062976

segment-probe.gguf

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:bd3f0bff1057ad0c953deb79d4b82415076906494c7bbb47a8a0f8e85433b667
+size 1062912