Spaces:
Running
Running
Commit History
-
Deploy a2e39fd
github-actions[bot]Merge pull request #26 from WINTER4000/fix/dms-manifest-citation Source: https://github.com/WINTER4000/turingDNA/commit/a2e39fd538db248f9c830bb81b779c4ffe3bdce8729d57f
-
Commit the real 3-dataset benchmark result
Tengo GzirishviliFirst genuine multi-dataset run: BLAT_ECOLX_Stiffler2015 (n=4996, rho=0.5165), PABP_YEAST_Melamed2013 (n=1188, rho=0.4998), DLG4_RAT_McLaughlin2012 (n=1577, rho=0.5246). Computed live on the Space via /api/admin/run-benchmarks against dee/data/dms_fixtures/.461718a
-
Commit the first real benchmark result + fix a self-invalidating test
Tengo Gzirishvili +1: Claude Sonnet 5Pulled the live result from /api/admin/run-benchmarks (run by the founder against the bundled BLAT_ECOLX fixture on the actual Space, real ESM-2 small): median Spearman rho 0.5165, top-decile precision 47.8%, across all 4,996 real mutations. This is now the permanent, committed state, so it survives future redeploys instead of resetting to the empty placeholder. Also fixes test_benchmarks_route_public_and_honest_empty, which asserted the bundled file is ALWAYS empty — true when it was written, but wrong the moment a real run legitimately populates it (which just happened). Replaced with a shape-only assertion (the route never fabricates, whatever the current file state is) plus a separate test that the honest-empty behavior still holds when the file is genuinely absent. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>5dc55c5
-
Receipts + seed: validation harness, DMS→commons seeding, evidence UI
Tengo Gzirishvili +1: Claude Opus 4.8Turns the glass-box thesis from claims into proof — and gives the empty commons a path to real data. Honest by construction: no benchmark number is ever hand-entered; the app serves only what a real run produced, and ships an explicit "pending" state until then (this sandbox has no torch/ESM, so the numbers get made on a box with the weights). - dee/core/benchmark.py — validation harness: average-rank Spearman ρ + top-decile precision ("of your top picks, how many are genuinely high- fitness?") + cross-dataset medians. Pure numpy, predictions injected → unit-testable without ESM. - dee/core/dms_seed.py — turns published DMS assays into de-identified substitution-prior rows via the SAME k-anonymized aggregate machinery user data uses (each assay = one contributor; a substitution needs ≥ MIN_USERS independent studies to survive). ESM-free, so it can seed the commons with real, citable data on day one; user results refine it after. - scripts/run_benchmarks.py + scripts/seed…8d1e644