olmo-urban-part1 / eval /retention.json
sparrow8i8's picture
Anchor eval: TAC (fixed, 52) + Betley + HarvestBench + retention
9e2ee66 verified
Raw History Blame Contribute Delete
297 Bytes
{
"arm": "olmo-urban",
"n_docs": 200,
"max_tokens": 2048,
"results": {
"pretrained": {
"loss": 2.9594,
"retained": 0.0
},
"midtrained": {
"loss": 1.6368,
"retained": 1.0
},
"CompassioninMachineLearning/olmo-urban-part1": {
"loss": 1.8511,
"retained": 0.838
}
}
}