modelops-research-lab / REPRODUCE.md
Hugdrigo's picture
Upload 28 files
7c9e8d9 verified
|
Raw History Blame Contribute Delete
1.12 kB

Reproducing the experiments

Browser

Upload the repository files to one root folder in a Hugging Face Static Space. Open App. Models fit automatically (locally) to train.json. Open Overview, or Experiments → Run four baselines. Select challenge (54) or IID (130); enter a bootstrap seed. Download metrics JSON and prediction CSV.

Node, no dependencies

node --test test_engine.js
node benchmark_cli.js

Both commands use only Node 18+ built-ins and the bundled dataset files. No Hugging Face token, Python packages or GPU needed.

Browser test, optional

python -m pip install playwright
python -m playwright install chromium
python test_browser.py

The test injects local project files directly into the Chromium page and mocks fetch only for fixture-loading to avoid external network dependency in restricted CI. This is not a test of deployed Hugging Face hosting.

Interpreting results

Use the fixed split names. Do not compare any archived Python-model evaluations as if they used the same code. Synthetic evaluations are not proof of generalization to real scholarly work.