Update README.md
Browse files
README.md
CHANGED
|
@@ -6,4 +6,16 @@ tags:
|
|
| 6 |
- main-content-extraction
|
| 7 |
- html-cleaning
|
| 8 |
- web-scraping
|
| 9 |
-
---
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 6 |
- main-content-extraction
|
| 7 |
- html-cleaning
|
| 8 |
- web-scraping
|
| 9 |
+
---
|
| 10 |
+
|
| 11 |
+
# htmlsift model weights
|
| 12 |
+
Weights for [htmlsift](https://github.com/koivualeksi/htmlsift), a web main-content extractor (HTML in, clean text / HTML / markdown out).
|
| 13 |
+
|
| 14 |
+
- `mini/` — int8 ONNX bundle, runs on CPU. `pip install htmlsift`
|
| 15 |
+
- `base/` — 311M encoder bundle (Granite r2, first 10 layers), GPU-capable. `pip install htmlsift[full]`
|
| 16 |
+
|
| 17 |
+
The weights are fetched automatically on first use — you don't download them by hand.
|
| 18 |
+
|
| 19 |
+
## License and attribution
|
| 20 |
+
|
| 21 |
+
Weights are licensed **Apache-2.0**. They are trained on [WebMainBench](https://github.com/opendatalab/WebMainBench) (Apache-2.0) — please attribute WebMainBench when you use them.
|