organisation card rebuilt around the achievements: numbers with links to repos and datasets, repository table incl. the Czech VQ client and the demo Space

README.md

@@ -1,37 +1,64 @@
----
-title: thinletter
-emoji: 🪶
-colorFrom: pink
-colorTo: yellow
-sdk: static
-pinned: false
----
-
-# thinletter
-
-**Smaller models. More possibilities.** A Czech AI startup working on the compression and efficient representation of language models, and on tuning them to a specific language or domain. Everything here is measured against the unchanged system the model has to fit into, with pre-registered predictions, paired bootstrap intervals and published negative results.
-
-Web: [thinletter.io](https://thinletter.io) · code and report: [github.com/rosecky/embedding-quantization-public](https://github.com/rosecky/embedding-quantization-public) · contact: info@thinletter.io, or [GitHub issues](https://github.com/rosecky/embedding-quantization-public/issues)
-
-## What is here
-
-Query-side clients for embedding models: the document index stays as it is, only the query encoder moves to the client (llama.cpp, or a browser via WebAssembly / WebGPU). Every file is verified on the index it has to serve — nDCG@10, cosine to the full-precision query vector, top-10 overlap — and its model card states exactly which document encoder it is compatible with.
-
-| repository | base model · licence | files | what holds |
-|---|---|---|---|
-| [harrier-0.6b-query-clients](https://huggingface.co/thinletter/harrier-0.6b-query-clients) | microsoft/harrier-oss-v1-0.6b · MIT | 192–235 MiB GGUF | 3.4-bit client keeps 98.9–101 % of fp32 nDCG@10 on SciFact / NFCorpus / ArguAna / SciDocs |
-| [qwen3-embedding-0.6b-query-clients](https://huggingface.co/thinletter/qwen3-embedding-0.6b-query-clients) | Qwen/Qwen3-Embedding-0.6B · Apache-2.0 | 340–385 MiB GGUF, English and Czech calibration | 99.3–100.1 % on four English corpora; 98.7–99.5 % on a Czech legal index |
-| [bge-m3-query-clients](https://huggingface.co/thinletter/bge-m3-query-clients) | BAAI/bge-m3 · MIT | 321–355 MiB GGUF, Czech calibration | 99.2 % / 98.3 % on the Czech index, 99.6–100 % on SciFact |
-| [harrier-0.6b-vq-clients](https://huggingface.co/thinletter/harrier-0.6b-vq-clients) | microsoft/harrier-oss-v1-0.6b · MIT | 113–127 MiB `.vqw`, 1.8–2.1 bits per weight | 94.7 % / 91.3 % on SciDocs; runs in the thinletter WebGPU runtime (Apache-2.0, in the repository and in the public demo) |
-| [qwen3-embedding-0.6b-vq-clients](https://huggingface.co/thinletter/qwen3-embedding-0.6b-vq-clients) | Qwen3-Embedding-0.6B · Apache-2.0 | 237 MiB, 2-d codebooks 3.57 bpw + full token table | Czech: 96.8 % of fp32 nDCG@10 on WebFAQ-cs (cosine 0.946), browser = simulation, 112 ms on an integrated GPU; runs in the open thinletter WebGPU runtime |
-| [llm-weight-compression-evidence](https://huggingface.co/datasets/thinletter/llm-weight-compression-evidence) | dataset · MIT | per-window evidence | which objective a post-training quantizer should optimise: module-level refinement of GPTQ codes on small decoders |
-
-Each model repository has a short summary post in its Community tab with the numbers, what was found along the way, and the limits.
-
-## How to read the numbers
-
-Test split, the model's own fp32 index, native runtime. Differences under 0.01 nDCG@10 are ties: a paired interval over queries does not contain the variance of the calibration draw (~0.01) or between-machine variation (±0.006). The Czech evaluation (55 071 supreme-court segments) uses synthetic queries and no human relevance judgements; its fp32 level is the same for every base model tried, so it measures the compression, not the base.
-
-## Verify on your own index
-
-The [public repository](https://github.com/rosecky/embedding-quantization-public) has the recipe: import your corpus and existing document vectors, generate synthetic queries, quantise with `llama-quantize --imatrix` on text in your corpus language, verify. Five commands; if you run it, we would like to see the table. It also carries the vector-quantisation pipeline — quantiser, `.vqw` container compiler and the WebGPU runtime ([`client/vqweb`](https://github.com/rosecky/embedding-quantization-public/tree/main/client/vqweb)) — under Apache-2.0.
+---
+title: thinletter
+emoji: 🪶
+colorFrom: pink
+colorTo: yellow
+sdk: static
+pinned: false
+---
+
+# thinletter — a query encoder as a 120 MiB file, in the browser, open source
+
+We take an embedding model, leave its document index exactly as it is, and compile the **query side** into a file a tenth of the size that
+answers in the browser in about 100 ms. Everything behind the numbers is public under Apache-2.0: the quantiser, the container compiler,
+the WebGPU runtime, the verification recipe and every released file. Web: [thinletter.io](https://thinletter.io) · code and report:
+[github.com/rosecky/embedding-quantization-public](https://github.com/rosecky/embedding-quantization-public) · demo:
+[thinletter.io/demo](https://thinletter.io/demo) or the Space [thinletter/demo](https://huggingface.co/spaces/thinletter/demo) · contact: info@thinletter.io
+
+## What holds
+
+| number | what it is | where to check |
+|---|---|---|
+| **98.2 %** | of full-precision nDCG@10 from a 120 MiB, 2.1-bit vector-quantised harrier-0.6b client, no retraining: above the trained BitNet-270m (140 MiB, 97.0 %), below any file llama.cpp can produce (178 MiB, 93.9 %). The browser reproduces the simulation (0.7375 vs 0.7380). | [harrier-0.6b-vq-clients](https://huggingface.co/thinletter/harrier-0.6b-vq-clients), [BeIR/scifact](https://huggingface.co/datasets/BeIR/scifact) |
+| **4.3×** | faster than llama.cpp's WebGPU path on the same integrated GPU: 103 ms per query for the 119 MiB vector-quantised client against 444 ms for the 235 MiB Q3_K file, in our open WebGPU runtime (`client/vqweb`). | report §3.4b in the repository |
+| **96.8 %** | of fp32 nDCG@10 on the public Czech benchmark WebFAQ-cs from a 237 MiB vector-quantised Qwen3-Embedding-0.6B client (2-d codebooks, 3.6 bits per weight), browser = simulation, 112 ms per query. Czech-calibrated scalar clients: bge-m3 99.2 % (355 MiB), Qwen3 98.7 % (340 MiB) on a Czech legal index. | [qwen3-embedding-0.6b-vq-clients](https://huggingface.co/thinletter/qwen3-embedding-0.6b-vq-clients), [PaDaS-Lab/webfaq-retrieval](https://huggingface.co/datasets/PaDaS-Lab/webfaq-retrieval) |
+| **+0.010** | nDCG@10 at 1.8 bits from a method of ours: ship the query prompt's keys and values in full precision (2 MB) and calibrate the compressed model with them in place; three seeds on SciFact and on a Czech legal index, every paired interval above zero. | report §3.7; the `*p` files in [harrier-0.6b-vq-clients](https://huggingface.co/thinletter/harrier-0.6b-vq-clients) |
+| **96–101 %** | kept by the 3.4-bit harrier-0.6b scalar client on four English corpora, 235 MiB instead of 1 143 MiB. Not ours to claim: llama.cpp's own quantiser gets there too, and we say so. | [harrier-0.6b-query-clients](https://huggingface.co/thinletter/harrier-0.6b-query-clients) |
+
+## Repositories
+
+Query-side clients: each file is a query encoder for exactly one document encoder and its settings (the model card says which).
+The index does not change.
+
+| repository | base model · licence | files | what holds (test split, the model's own fp32 index) |
+|---|---|---|---|
+| [harrier-0.6b-query-clients](https://huggingface.co/thinletter/harrier-0.6b-query-clients) | [microsoft/harrier-oss-v1-0.6b](https://huggingface.co/microsoft/harrier-oss-v1-0.6b) · MIT | 3 GGUF, 192–235 MiB | Q3_K generic-English 98.9 / 99.3 / 101 / 99.2 % on [SciFact](https://huggingface.co/datasets/BeIR/scifact) / [NFCorpus](https://huggingface.co/datasets/BeIR/nfcorpus) / [ArguAna](https://huggingface.co/datasets/BeIR/arguana) / [SciDocs](https://huggingface.co/datasets/BeIR/scidocs); Q2_K SciDocs-calibrated 94.5 % |
+| [harrier-0.6b-vq-clients](https://huggingface.co/thinletter/harrier-0.6b-vq-clients) | microsoft/harrier-oss-v1-0.6b · MIT | 8 `.vqw`, 105–129 MiB, 1.8–2.1 bits per weight | SciDocs 94.7 / 91.3 %, SciFact 98.2 / 96.5 %; prompt-K/V variants SciDocs 96.6 / 93.8 %, SciFact 98.1 / 95.8 %; browser-verified; run in the open WebGPU runtime |
+| [qwen3-embedding-0.6b-query-clients](https://huggingface.co/thinletter/qwen3-embedding-0.6b-query-clients) | [Qwen/Qwen3-Embedding-0.6B](https://huggingface.co/Qwen/Qwen3-Embedding-0.6B) · Apache-2.0 | 4 GGUF, 340–385 MiB, English and Czech calibration | Q4_K_M + 4-bit table: 99.3–100 % on the four English corpora, 98.7 % on a Czech legal index; Q5_K_M 99.5 % Czech |
+| [qwen3-embedding-0.6b-vq-clients](https://huggingface.co/thinletter/qwen3-embedding-0.6b-vq-clients) | Qwen/Qwen3-Embedding-0.6B · Apache-2.0 | 1 `.vqw`, 237 MiB, 2-d codebooks 3.57 bpw + full token table | Czech: 96.8 % of fp32 on [WebFAQ-cs](https://huggingface.co/datasets/PaDaS-Lab/webfaq-retrieval) (cosine 0.946, overlap 0.755), browser = simulation, 112 ms on an integrated GPU |
+| [bge-m3-query-clients](https://huggingface.co/thinletter/bge-m3-query-clients) | [BAAI/bge-m3](https://huggingface.co/BAAI/bge-m3) · MIT | 2 GGUF, 321–355 MiB, Czech calibration | 99.2 % / 98.3 % on the Czech legal index, 99.6–100 % on SciFact |
+| [demo](https://huggingface.co/spaces/thinletter/demo) (Space) | — | the browser demo, static | search SciDocs (25 656 abstracts) or WebFAQ-cs (71 529 Czech passages) locally with a scalar or a vector-quantised client; models and indexes download from thinletter.io |
+| [llm-weight-compression-evidence](https://huggingface.co/datasets/thinletter/llm-weight-compression-evidence) | dataset · MIT | per-window evidence | which objective a post-training quantiser should optimise (companion project) |
+
+Each model repository has a summary post in its Community tab with the numbers, what was found along the way, and the limits.
+
+## How we measure
+
+Test split, the model's own fp32 index, native runtime (llama.cpp) or the browser. Every run is pre-registered with a prediction and a
+kill rule; comparisons change one thing at a time at the same calibration budget; differences are paired bootstrap intervals over queries
+(10 000 draws) read against the variance of the calibration draw (~0.01 nDCG@10) and between machines (±0.006): under 0.01 is a tie.
+Negative results stay published (a codebook-budget idea that did not work, a 2.6-bit headline that held on one corpus only, the 2-bit cliff
+of Qwen3-Embedding: 65–81 % for the same run on two machines).
+
+## Verify on your own index
+
+The [public repository](https://github.com/rosecky/embedding-quantization-public) has the recipe: import your corpus and existing
+document vectors, generate synthetic queries, quantise (llama-quantize, the GPTQ exporter, or the vector quantiser + `.vqw` compiler),
+evaluate the client against the unchanged index with a paired interval. Everything in the report is reproducible from it; only the raw
+per-run results, the research log and a customer's data stay out.
+
+## Licences
+
+Code Apache-2.0 (NOTICE lists wllama MIT, @huggingface/tokenizers Apache-2.0, a Q2_K decoder ported from llama.cpp MIT). Weights under
+their base model's licence (MIT / Apache-2.0). Demo corpora: SciDocs (BEIR, CC BY 4.0), WebFAQ-cs (CC BY 4.0). Files derived from
+non-commercial bases (jina-embeddings-v5, CC BY-NC) are reported as numbers only.