Spaces:
Running
Running
File size: 4,155 Bytes
db2c226 85920d3 db2c226 85920d3 db2c226 85920d3 db2c226 85920d3 db2c226 85920d3 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | ---
title: ModelGuard Observatory
emoji: 🔬
colorFrom: green
colorTo: blue
sdk: static
app_file: index.html
license: mit
short_description: Reproducible NLP drift, errors and release-gate audits.
tags:
- nlp
- machine-learning
- model-evaluation
- data-drift
- explainability
- multilingual
- research
pinned: false
---
# ModelGuard Observatory | Reproducible ML Evaluation Lab
**Research project by Rodrigo Pantoja** · Free browser-only Hugging Face Static Space.
ModelGuard Observatory explores the reproducibility, robustness, and documentation of **four deterministic text classifiers** on **184 distinct synthetic-curated evaluation examples** (130 controlled IID plus 54 challenge). Predictions are recorded for every model/example pair, yielding 736 comparison records. This is an **evaluation snapshot observatory**, not a deployed-model live monitoring service.
## Eight interactive modules
1. **Overview:** accuracy, macro-F1, generalization gap, model comparisons.
2. **Drift detection:** binned text-length PSI, label/language Jensen–Shannon divergence.
3. **Slice analysis:** descriptive subgroup outcomes by language, domain, true class.
4. **Error explorer:** confusion matrix and original evaluation examples.
5. **Model regression:** paired model comparisons and seeded bootstrap intervals.
6. **Release gates:** transparent, adjustable exploratory thresholds with failure explanations.
7. **Run registry:** model provenance, input hashes, dataset integrity and evaluation ledger.
8. **Export & governance:** reproducible CSV, JSON, Markdown and local snapshot import.
## Results and honest scope
| Baseline | Controlled accuracy (130) | Novel-pattern challenge accuracy (54) |
| --- | ---: | ---: |
| Multinomial Naive Bayes | 100.0% | 74.1% |
| Word TF-IDF | 98.5% | 72.2% |
| Character TF-IDF | 95.4% | 70.4% |
| Hybrid TF-IDF | 97.7% | 77.8% |
These numbers were recomputed from the original **Academic Rhetorical Intent Mini v1.1** synthetic-curated dataset using the same deterministic four-model engine as **ModelOps Research Lab v2.0**. The challenge split purposely contains novel wording. Results are descriptive, **not** scientific evidence of external deployment accuracy, fairness, or safety.
The initial release checklist intentionally fails its text-length PSI gate for all four methods. The tool never hides failed checks or silently overrides thresholds.
## Use
Open `App`; switch between eight modules. Select a model and evaluation split. Under **Export & governance**, download original predictions, import another compatible synthetic evaluation file, or restore the bundled evidence. Imported data is processed **in your browser only**. Avoid importing personal or confidential records.
The application does not automatically retrain, monitor a production service, submit external jobs, or access credentials.
## Reproduction
```bash
node build_artifacts.js
node test_engine.js
```
This package includes the original training and evaluation splits, their source engine, the new monitoring engine, and the source manifest with hashes. Browser workflows can be checked with `test_browser.py` (Playwright plus local Chromium), though this optional script may require setting the local Chromium executable path.
## Hosting
Static Space, no servers or paid APIs. The README metadata includes `sdk: static` and `app_file: index.html`. See `DEPLOYMENT_ES.md` for a simple Mac upload guide.
## Documentation
- [`METHODS.md`](METHODS.md) for definitions, formulas, and warning thresholds.
- [`DATA_PROVENANCE.md`](DATA_PROVENANCE.md) and [`MODEL_CARD.md`](MODEL_CARD.md) for source details and limitations.
- [`GOVERNANCE.md`](GOVERNANCE.md), [`LIMITATIONS.md`](LIMITATIONS.md), and [`SECURITY.md`](SECURITY.md) for responsible use.
- [`QA_REPORT.md`](QA_REPORT.md) for test coverage and deployment status.
- [`REPRODUCE.md`](REPRODUCE.md) for commands and artifact lineage.
## Licenses
Application code: MIT. Original synthetic-curated dataset: CC BY 4.0, separate from the MIT code license. See `DATA_LICENSE.md`. Example text is not retrieved from journal articles or student work.
|