File size: 2,543 Bytes
5d02fd5 2c75ac5 5d02fd5 79621d1 5d02fd5 79621d1 5d02fd5 b1b061d 79621d1 87e9886 79621d1 5d02fd5 4b4a6d6 5d02fd5 79621d1 5d02fd5 79621d1 5d02fd5 87e9886 5d02fd5 79621d1 5d02fd5 87e9886 5d02fd5 1c778c4 5d02fd5 79621d1 5d02fd5 94dd871 b036f03 5d02fd5 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | ---
library_name: transformers
license: apache-2.0
base_model: Qwen/Qwen3-4B
tags:
- web-scraping
- information-extraction
- qwen
- qwen3
- co-scraper
pipeline_tag: text-generation
---
# Qwen-HTML 4B
Qwen-HTML is a lightweight HTML-specialized model for **Co-Scraper**, a query-aware web data extraction framework. It is initialized from [Qwen/Qwen3-4B](https://huggingface.co/Qwen/Qwen3-4B) and trained for two complementary tasks:
1. **Node identification**: select the DOM nodes most likely to contain query-relevant records and attributes.
2. **Scraper synthesis**: generate reusable Python extraction code over the pruned HTML.
The model is designed to reduce the cost and failure rate of web information extraction by working with compact, query-aware DOM views instead of full raw pages.
## Model description
- **Base model:** Qwen3-4B
- **Architecture:** `Qwen3ForCausalLM`
- **Precision:** bfloat16
- **Context length:** 131,072 tokens
## Intended use
Deploy the model with [vLLM](https://docs.vllm.ai/) as an OpenAI-compatible service:
```bash
pip install vllm
vllm serve PATH/TO/Qwen-HTML \
--served-model-name qwen-html \
--dtype bfloat16 \
--tensor-parallel-size 1 \
--max-model-len 131072 \
--port 8000
```
Then use the endpoint with the Co-Scraper inference pipeline:
```bash
export OPENAI_BASE_URL="http://localhost:8000/v1"
export OPENAI_API_KEY="not-required"
coscraper \
"Extract the camera model and price." \
schema.json \
seed_1.html seed_2.html seed_3.html \
--model qwen-html
```
Set `--tensor-parallel-size` to the number of GPUs used for tensor parallelism. If GPU memory is insufficient for the full 131,072-token context, reduce `--max-model-len` according to your deployment budget.
## Evaluation
Results from the Co-Scraper paper. Higher is better except Unex.f.
| Dataset | Precision | Recall | F1 | Cor<sub>p</sub> | Cor<sub>f</sub> | Unex.<sub>f</sub> |
|---|---:|---:|---:|---:|---:|---:|
| SWDE | 95.93 | 96.24 | 96.00 | 85.87 | 95.51 | 3.19 |
| LiveWeb-IE | 92.29 | 92.68 | 92.40 | 82.00 | 91.28 | 6.08 |
Metric definitions:
- **F1** ： field-level extraction quality.
- **Cor<sub>p</sub>** ： page-level perfect extraction rate.
- **Cor<sub>f</sub>** ： field-level cross-page reuse success rate.
- **Unex.<sub>f</sub>** ： fraction of fields with unexecutable or invalid output.
## Limitations
Generated scrapers should still be reviewed before being executed on untrusted pages. Model performance can vary with HTML quality, site structure, and query clarity.
|