wpferrell commited on
Commit
a0b9790
Β·
verified Β·
1 Parent(s): 562bdcf

BigSmall card for 3.15.0

Browse files
Files changed (1) hide show
  1. README.md +63 -10
README.md CHANGED
@@ -1,10 +1,63 @@
1
- ---
2
- title: README
3
- emoji: πŸ“š
4
- colorFrom: gray
5
- colorTo: indigo
6
- sdk: static
7
- pinned: false
8
- ---
9
-
10
- Check out the configuration reference at https://huggingface.co/docs/hub/spaces-config-reference
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
1
+ # BigSmall β€” Lossless AI Model Compression
2
+
3
+ **Make any AI model ~34% smaller. Bit-identical weights. Drop-in replacement for `from_pretrained`.**
4
+
5
+ ```bash
6
+ pip install bigsmall # CLI + compression/decompression
7
+ pip install bigsmall[torch] # add this for model loading (from_pretrained)
8
+ ```
9
+
10
+ Every model on this page is a real measurement: a 14 GB Mistral-7B is 8.9 GB as a `.bs`, a
11
+ 29.5 GB Qwen2.5-14B-Instruct is 19.5 GB β€” and after decompression **every weight is bit-for-bit
12
+ identical to the original** (md5 verified per tensor on decompress).
13
+
14
+ - **Not quantization.** Nothing is rounded; the model's behaviour cannot change.
15
+ - **Not pruning, not approximation.** The same idea as ZIP for text, tuned for the statistics
16
+ of trained BF16 weights.
17
+ - **Fine-tunes ship as deltas.** A β‰₯7B official instruct tune stored against its public base
18
+ measures 34–50% of the full model. Delta size is pair-dependent (measured range: under 1% for
19
+ the best β‰₯7B SFT pairs, up to ~61% for small-model full tunes) β€” the docs carry the full
20
+ measured table.
21
+
22
+ ## Using a pre-compressed model
23
+
24
+ ```python
25
+ from transformers import AutoModelForCausalLM
26
+
27
+ model = AutoModelForCausalLM.from_pretrained(
28
+ "wpferrell/phi-3.5-mini-instruct-bigsmall"
29
+ )
30
+ ```
31
+
32
+ BigSmall decompresses transparently on load. Prefer the CLI? Download the repo and run
33
+ `bigsmall decompress model-00001-of-00002.bs -o model.safetensors`.
34
+
35
+ ## Honest numbers
36
+
37
+ The lossless floor for trained BF16 weights is real and measured: across 4,143 weight matrices
38
+ in 8 architectures the per-tensor entropy floor is flat (CV β‰ˆ 0), and trained mantissa/sign bits
39
+ are coder-equivalent to matched random controls β€” training only writes the exponent. BigSmall
40
+ codes at that wall. Head-to-head under the same accounting, BigSmall codes below DFloat11's
41
+ bound on **every layer type of every model measured** (+0.45–0.55 pp model-level, +12–18 pp on
42
+ norm scales) β€” see `docs/dfloat11.md` in the source repo.
43
+
44
+ ## New in 3.15.0
45
+
46
+ - Fail-safe delta engine: both codings measured per tensor, a delta can never come out larger
47
+ than standalone.
48
+ - KV cache entry format v3: per-depth sequential dispatcher (end-to-end wins grow with context,
49
+ 0.27% β†’ 0.58% at ≀2048 tokens; up to +7.7% on early-layer K), plus FP8 entries, 32k+ chunked
50
+ streaming entries, and device-routing `AutoKVCache`. v1/v2 entries decode forever.
51
+ - `bigsmall xray`: checkpoint forensics β€” per-tensor substream entropies vs a matched-random
52
+ control, lineage/anomaly flags, and a "looks untrained" detector that catches
53
+ silently-randomized checkpoints.
54
+ - Opt-in role-group stream packing for small same-role tensors.
55
+
56
+ ## Links
57
+
58
+ - **PyPI** β€” https://pypi.org/project/bigsmall/
59
+ - **Source** β€” https://github.com/wpferrell/Bigsmall
60
+ - **Paper / DOI** β€” https://doi.org/10.5281/zenodo.20279247
61
+ - **License** β€” Elastic License 2.0 (free for personal, research, and internal commercial use);
62
+ model weights in `.bs` format keep the license of the original model.
63
+ - **Contact** β€” wpferrell@gmail.com