Crazyfff/SPU-bucket / eval.yaml
Crazyfff's picture
download
raw
531 Bytes
name: Science Prediction Universe
description: >
Science Prediction Universe (SPU) tests whether AI agents can predict the outcomes of
real wet-lab experiments. Each of the 100 public tasks asks the agent to rank
experimental candidates by a measured outcome, and rankings are scored by
Spearman correlation. Report the number of scored tasks, model version,
reasoning effort and harness with every result.
evaluation_framework: harbor
tasks:
- id: public100-scored-mean-spearman-v1
config: default
split: test

Xet Storage Details

Size:
531 Bytes
·
Xet hash:
da5e9d4c315a5f7df58a3fac33bd3f4ed964102d5ae5b99e4b857429cd95ad7e

Xet efficiently stores files, intelligently splitting them into unique chunks and accelerating uploads and downloads. More info.