Upload README.md with huggingface_hub

README.md

@@ -0,0 +1,50 @@
+---
+language:
+- pt
+- en
+tags:
+- chunking
+- rag
+- text-splitter
+- distillation
+license: apache-2.0
+---
+
+# tinyzchunk weights
+
+Tiny MLP weights for the **tinyzchunk** chunker — a GPU-free, numpy-only chunker
+distilled from the [zChunk](https://github.com/zchunk) algorithm. English and
+Brazilian Portuguese, robust to noisy/OCR'd text.
+
+The package source lives on GitHub: https://github.com/cnmoro/tinyzchunk
+
+## Install
+
+```bash
+pip install tinyzchunk
+```
+
+The pip package is weight-free: on first use the weights are fetched from this
+repo automatically (and cached). No GPU, no tokenizer.
+
+## Usage
+
+```python
+from tinyzchunk import Chunker
+
+chunker = Chunker()                    # auto-fetches weights from this repo
+chunks = chunker.chunk(long_document)  # -> list[str]
+
+# or fetch explicitly
+chunker = Chunker.from_pretrained("cnmoro/tinyzchunk")
+```
+
+## Files
+- `weights.npz` — char-level sentence/paragraph boundary model (fallback)
+- `line_weights.npz` — line-level "does a new unit start here?" model (primary)
+
+## Distillation
+Trained from Qwen2.5-7B zChunk teachers on prose + structured corpora (Q&A
+roteiros, schedules, bios, FAQs, legal, contact lists) plus simulated-PDF-noise
+training. Event schedules boundary F1 0.92–0.98; Q&A roteiros 0.74–1.0; noisy
+text 0.89–0.97. ~14 ms/document on CPU.