|
Download validation_set/README.md from Basel0/qshub: direct link, hf CLI and curl.
- Browser
- Download file 2.21 kB
-
https://huggingface.co/spaces/Basel0/qshub/resolve/main/validation_set/README.md
- Command line
-
hf download hf://spaces/Basel0/qshub/validation_set/README.md
-
curl -L -o README.md https://huggingface.co/spaces/Basel0/qshub/resolve/main/validation_set/README.md
2.21 kB
| # Validation set β measure real drawingβBOQ accuracy | |
| This harness references real villa projects **in place** (it never copies the large | |
| PDFs), so the repo stays lean and runtime performance is unaffected. | |
| ## What qualifies as a validation project | |
| A project folder must contain all three: | |
| - architectural drawing β PDF whose name contains `arch` | |
| - structural drawing β PDF whose name contains `str`/`struct` | |
| - ground-truth BOQ β an Excel file that is **not** a system output | |
| (`*_QTO_HYBRID_TEST.xlsx` etc. are ignored) | |
| Incomplete folders are skipped automatically and listed under `incomplete_projects`. | |
| ## Run (cheap β no AI, no copying) | |
| ```powershell | |
| # default root = B:\work\trainning (the 8 clean arch+str+BOQ pairs) | |
| python validation_set/run_validation.py | |
| # point at another folder | |
| python validation_set/run_validation.py --root "B:\work\projects estimation\projects\..." | |
| ``` | |
| Output: `validation_set/manifest.json` β paths + each project's ground-truth BOQ | |
| parsed into normalized engine keys (excavation, foundation_concrete, β¦) via the | |
| existing `_extract_real_boqs.extract_from_workbook`. | |
| `manifest.json` is git-ignored (machine-specific absolute paths; regenerate anytime). | |
| ## Current verified set (root = trainning) | |
| All **11** projects (1β11) qualify β each has an architectural PDF + structural PDF | |
| + ground-truth BOQ (parsed to 23β29 items). The detector handles real-world naming | |
| variants (`AR-β¦` for architectural, `STC`/`stc` for structural). | |
| Note: 5 and 7 share the same BOQ file β treat as one if deduping. | |
| ## Next step β the accuracy sweep (heavy, opt-in) | |
| `--run-engine` is the place to wire the full pass: load each project's two PDFs β | |
| run the extraction + `build_boq_dataframe_from_project` β diff computed vs ground-truth | |
| per item β emit a per-item accuracy report (% deviation, π’/π‘/π΄). | |
| It is **not** run automatically because it calls the AI extractor (Gemini cost + time). | |
| ## Adding more projects | |
| Point `--root` at any folder of clean `arch + str + BOQ` projects. Noisy folders with | |
| 40β100 PDFs (NOCs, soil reports, approvals) need the right two sheets isolated first β | |
| the auto-detector picks one `arch` + one `str` PDF per folder. | |