File size: 1,922 Bytes
795f737 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | # Pretrained diagnostic baselines
Pinned local downloads for research only; no external inference endpoint is used.
| Model | Revision | Hub license metadata | Purpose |
|---|---|---|---|
| Qwen/Qwen2.5-0.5B-Instruct | 7ae557604adf67be50417f59c2c2f167def9a775 | Apache-2.0 | Local autoregressive action baseline |
| cross-encoder/ms-marco-MiniLM-L6-v2 | 233902d25c440f23af6f7d6e94d2946bac0bee0a | Apache-2.0 | Semantic candidate reranking |
Models are loaded with local_files_only=True and trust_remote_code=False.
Model weights use safetensors. Qwen's source LICENSE and both model cards were
downloaded with the checkpoints under the intermediate work directory.
Sources: [Qwen model card](https://huggingface.co/Qwen/Qwen2.5-0.5B-Instruct),
[MiniLM model card](https://huggingface.co/cross-encoder/ms-marco-MiniLM-L6-v2).
The metadata and revisions were checked through the authenticated HF CLI.
Run from the project root after installing research extras:
```sh
python -m baim.retrieval_audit --source ../../work/mind2web-source/data/train/train_10.json --model ../../work/pretrained/minilm-cross
python -m baim.qwen_baseline --source ../../work/mind2web-source/data/train/train_10.json --model ../../work/pretrained/qwen-0.5b --limit 8
```
History is teacher-forced: only prior human action descriptions are supplied.
Neither evaluator receives the current labelled action or target as model input.
This is not autonomous multi-step success. Only aggregate metrics and numeric
per-step timing results are persisted, not raw page text, prompts or generations.
The Qwen diagnostic scores action/target matching, not typed-value correctness.
The broad BM25 candidate pool includes generic roles because source annotations
already classify those records as candidates. This offline choice is not a live
DOM interaction policy. Live generic elements still require interactability
evidence and execution validation.
|