Spaces:
Running
Running
|
Download LIMITATIONS.md from Hugdrigo/modelguard-observatory: direct link, hf CLI and curl.
- Browser
- Download file 1.23 kB
-
https://huggingface.co/spaces/Hugdrigo/modelguard-observatory/resolve/main/LIMITATIONS.md
- Command line
-
hf download hf://spaces/Hugdrigo/modelguard-observatory/LIMITATIONS.md
-
curl -L -o LIMITATIONS.md https://huggingface.co/spaces/Hugdrigo/modelguard-observatory/resolve/main/LIMITATIONS.md
1.23 kB
| # Limitations | |
| 1. Both bundled evaluation groups are *synthetic-curated*, not time-series observations; the product name does not imply live telemetry. | |
| 2. Data drift here is descriptive shift between two designed evaluation splits. There is no evidence of model drift over calendar time. | |
| 3. Performance numbers on a small, synthetic task do not generalize automatically to new scientific papers, languages, genres, or institutions. | |
| 4. Bootstrap percentiles assume example resampling and do not resolve annotation dependence, distribution shift, or real-world validation gaps. | |
| 5. PSI and JSD depend on chosen bins and categories; PSI thresholds are heuristics. | |
| 6. No protected-group labels are available, so fairness certification is impossible. | |
| 7. Imported evaluation data is not validated for correctness of human labels or provenance. The user is responsible for rights and permitted use. | |
| 8. Browser-based code is inspectable and modifiable by any user. Release gate overrides only change display criteria, not data quality. | |
| 9. No active monitoring, alerting, automated retraining, model hosting, or remote persistent storage is implemented. | |
| 10. This package does not contain journal PDFs, real student papers, or private data. | |