Spaces:
Running
Running
|
Download LIMITATIONS.md from Hugdrigo/modelguard-observatory: direct link, hf CLI and curl.
- Browser
- Download file 1.23 kB
-
https://huggingface.co/spaces/Hugdrigo/modelguard-observatory/resolve/main/LIMITATIONS.md
- Command line
-
hf download hf://spaces/Hugdrigo/modelguard-observatory/LIMITATIONS.md
-
curl -L -o LIMITATIONS.md https://huggingface.co/spaces/Hugdrigo/modelguard-observatory/resolve/main/LIMITATIONS.md
1.23 kB
Limitations
- Both bundled evaluation groups are synthetic-curated, not time-series observations; the product name does not imply live telemetry.
- Data drift here is descriptive shift between two designed evaluation splits. There is no evidence of model drift over calendar time.
- Performance numbers on a small, synthetic task do not generalize automatically to new scientific papers, languages, genres, or institutions.
- Bootstrap percentiles assume example resampling and do not resolve annotation dependence, distribution shift, or real-world validation gaps.
- PSI and JSD depend on chosen bins and categories; PSI thresholds are heuristics.
- No protected-group labels are available, so fairness certification is impossible.
- Imported evaluation data is not validated for correctness of human labels or provenance. The user is responsible for rights and permitted use.
- Browser-based code is inspectable and modifiable by any user. Release gate overrides only change display criteria, not data quality.
- No active monitoring, alerting, automated retraining, model hosting, or remote persistent storage is implemented.
- This package does not contain journal PDFs, real student papers, or private data.