dlm / README.md
hoangp111's picture
Upload README.md with huggingface_hub
19488c0 verified
|
Raw History Blame Contribute Delete
16.8 kB
`f` = 1e18 FLOPs.
| run | step | params | wandb |
|---|---|---|---|
| arm/113M-100f/pre-train | 104137 | 155,437,824 | [link](https://wandb.ai/marksmans/dlm/runs/05e8f627) |
| arm/113M-10f/pre-train | 10413 | 155,437,824 | [link](https://wandb.ai/marksmans/dlm/runs/6267ec3a) |
| arm/113M-30f/pre-train | 31241 | 155,437,824 | [link](https://wandb.ai/marksmans/dlm/runs/2815a9c5) |
| arm/113M-60f/pre-train | 62482 | 155,437,824 | [link](https://wandb.ai/marksmans/dlm/runs/19004b12) |
| arm/113M-6f/pre-train | 6248 | 155,437,824 | [link](https://wandb.ai/marksmans/dlm/runs/7e3c6d2b) |
| arm/1233M-100f/pre-train | 12083 | 1,271,060,224 | [link](https://wandb.ai/marksmans/dlm/runs/4e99db0b) |
| arm/1233M-10f/pre-train | 1208 | 1,271,060,224 | [link](https://wandb.ai/marksmans/dlm/runs/b2a7f2e4) |
| arm/1233M-30f/pre-train | 3624 | 1,271,060,224 | [link](https://wandb.ai/marksmans/dlm/runs/317e3d54) |
| arm/1233M-60f/pre-train | 7249 | 1,271,060,224 | [link](https://wandb.ai/marksmans/dlm/runs/a576010e) |
| arm/1233M-6f/pre-train | 724 | 1,271,060,224 | [link](https://wandb.ai/marksmans/dlm/runs/81b529e3) |
| arm/170M-100f/pre-train | 73072 | 208,531,200 | [link](https://wandb.ai/marksmans/dlm/runs/96a4f141) |
| arm/170M-10f/pre-train | 7307 | 208,531,200 | [link](https://wandb.ai/marksmans/dlm/runs/a681c70c) |
| arm/170M-30f/pre-train | 21921 | 208,531,200 | [link](https://wandb.ai/marksmans/dlm/runs/733111c7) |
| arm/170M-60f/pre-train | 43843 | 208,531,200 | [link](https://wandb.ai/marksmans/dlm/runs/546e1184) |
| arm/170M-6f/pre-train | 4384 | 208,531,200 | [link](https://wandb.ai/marksmans/dlm/runs/2eaee3cb) |
| arm/336M-100f/pre-train | 40144 | 380,281,856 | [link](https://wandb.ai/marksmans/dlm/runs/19b8f681) |
| arm/336M-10f/pre-train | 4014 | 380,281,856 | [link](https://wandb.ai/marksmans/dlm/runs/72b6e66a) |
| arm/336M-30f/pre-train | 12043 | 380,281,856 | [link](https://wandb.ai/marksmans/dlm/runs/4bbbee37) |
| arm/336M-60f/pre-train | 24086 | 380,281,856 | [link](https://wandb.ai/marksmans/dlm/runs/b825f0c3) |
| arm/336M-6f/pre-train | 2408 | 380,281,856 | [link](https://wandb.ai/marksmans/dlm/runs/af6cc85a) |
| arm/472M-100f/pre-train | 29462 | 524,499,200 | [link](https://wandb.ai/marksmans/dlm/runs/4bd0b799) |
| arm/472M-10f/pre-train | 2946 | 524,499,200 | [link](https://wandb.ai/marksmans/dlm/runs/e444698b) |
| arm/472M-30f/pre-train | 8838 | 524,499,200 | [link](https://wandb.ai/marksmans/dlm/runs/988cad3a) |
| arm/472M-60f/pre-train | 17677 | 524,499,200 | [link](https://wandb.ai/marksmans/dlm/runs/6ea8aa24) |
| arm/472M-6f/pre-train | 1767 | 524,499,200 | [link](https://wandb.ai/marksmans/dlm/runs/e8adf0f3) |
| arm/629M-100f/pre-train | 22588 | 671,970,560 | [link](https://wandb.ai/marksmans/dlm/runs/812425e3) |
| arm/629M-10f/pre-train | 2258 | 671,970,560 | [link](https://wandb.ai/marksmans/dlm/runs/61027ca4) |
| arm/629M-30f/pre-train | 6776 | 671,970,560 | [link](https://wandb.ai/marksmans/dlm/runs/93963428) |
| arm/629M-60f/pre-train | 13553 | 671,970,560 | [link](https://wandb.ai/marksmans/dlm/runs/178adad8) |
| arm/629M-6f/pre-train | 1355 | 671,970,560 | [link](https://wandb.ai/marksmans/dlm/runs/7a5b5d2d) |
| arm/66M-10f/pre-train | 16241 | 102,495,360 | [link](https://wandb.ai/marksmans/dlm/runs/0fd0e998) |
| arm/66M-30f/pre-train | 48724 | 102,495,360 | [link](https://wandb.ai/marksmans/dlm/runs/080db9bb) |
| arm/66M-60f/pre-train | 97449 | 102,495,360 | [link](https://wandb.ai/marksmans/dlm/runs/f0c93293) |
| arm/66M-6f/pre-train | 9744 | 102,495,360 | [link](https://wandb.ai/marksmans/dlm/runs/79ab5ffb) |
| arm/944M-100f/pre-train | 15467 | 983,314,944 | [link](https://wandb.ai/marksmans/dlm/runs/8a660db5) |
| arm/944M-10f/pre-train | 1546 | 983,314,944 | [link](https://wandb.ai/marksmans/dlm/runs/59ad9a8e) |
| arm/944M-30f/pre-train | 4640 | 983,314,944 | [link](https://wandb.ai/marksmans/dlm/runs/c2b04352) |
| arm/944M-60f/pre-train | 9280 | 983,314,944 | [link](https://wandb.ai/marksmans/dlm/runs/db7d6e67) |
| arm/944M-6f/pre-train | 928 | 983,314,944 | [link](https://wandb.ai/marksmans/dlm/runs/abd2629e) |
| e2d2-w32/113M-100f/pre-train | 93162 | 171,371,520 | [link](https://wandb.ai/marksmans/dlm/runs/11815232) |
| e2d2-w32/113M-10f/pre-train | 9316 | 171,371,520 | [link](https://wandb.ai/marksmans/dlm/runs/82e0b025) |
| e2d2-w32/113M-30f/pre-train | 27948 | 171,371,520 | [link](https://wandb.ai/marksmans/dlm/runs/cc9416e1) |
| e2d2-w32/113M-60f/pre-train | 55897 | 171,371,520 | [link](https://wandb.ai/marksmans/dlm/runs/50ae295a) |
| e2d2-w32/113M-6f/pre-train | 5589 | 171,371,520 | [link](https://wandb.ai/marksmans/dlm/runs/08e38255) |
| e2d2-w32/1233M-100f/pre-train | 10568 | 1,454,139,904 | [link](https://wandb.ai/marksmans/dlm/runs/c5f45d1c) |
| e2d2-w32/1233M-10f/pre-train | 1056 | 1,454,139,904 | [link](https://wandb.ai/marksmans/dlm/runs/4efb0469) |
| e2d2-w32/1233M-30f/pre-train | 3170 | 1,454,139,904 | [link](https://wandb.ai/marksmans/dlm/runs/00a9bd59) |
| e2d2-w32/1233M-60f/pre-train | 6341 | 1,454,139,904 | [link](https://wandb.ai/marksmans/dlm/runs/be3db3fc) |
| e2d2-w32/1233M-6f/pre-train | 634 | 1,454,139,904 | [link](https://wandb.ai/marksmans/dlm/runs/b575898c) |
| e2d2-w32/170M-100f/pre-train | 65485 | 233,316,096 | [link](https://wandb.ai/marksmans/dlm/runs/01826f73) |
| e2d2-w32/170M-10f/pre-train | 6548 | 233,316,096 | [link](https://wandb.ai/marksmans/dlm/runs/28fd45dc) |
| e2d2-w32/170M-30f/pre-train | 19645 | 233,316,096 | [link](https://wandb.ai/marksmans/dlm/runs/40a92d11) |
| e2d2-w32/170M-60f/pre-train | 39291 | 233,316,096 | [link](https://wandb.ai/marksmans/dlm/runs/f74f7cee) |
| e2d2-w32/170M-6f/pre-train | 3929 | 233,316,096 | [link](https://wandb.ai/marksmans/dlm/runs/a184ee0e) |
| e2d2-w32/336M-100f/pre-train | 35628 | 427,485,184 | [link](https://wandb.ai/marksmans/dlm/runs/ffae6288) |
| e2d2-w32/336M-10f/pre-train | 3562 | 427,485,184 | [link](https://wandb.ai/marksmans/dlm/runs/d33eb9bc) |
| e2d2-w32/336M-30f/pre-train | 10688 | 427,485,184 | [link](https://wandb.ai/marksmans/dlm/runs/0f87ad27) |
| e2d2-w32/336M-60f/pre-train | 21377 | 427,485,184 | [link](https://wandb.ai/marksmans/dlm/runs/b3974e90) |
| e2d2-w32/336M-6f/pre-train | 2137 | 427,485,184 | [link](https://wandb.ai/marksmans/dlm/runs/5b007327) |
| e2d2-w32/472M-100f/pre-train | 26099 | 593,332,480 | [link](https://wandb.ai/marksmans/dlm/runs/e91a392d) |
| e2d2-w32/472M-10f/pre-train | 2609 | 593,332,480 | [link](https://wandb.ai/marksmans/dlm/runs/da69085d) |
| e2d2-w32/472M-30f/pre-train | 7829 | 593,332,480 | [link](https://wandb.ai/marksmans/dlm/runs/9f693d22) |
| e2d2-w32/472M-60f/pre-train | 15659 | 593,332,480 | [link](https://wandb.ai/marksmans/dlm/runs/e4a3dbce) |
| e2d2-w32/472M-6f/pre-train | 1565 | 593,332,480 | [link](https://wandb.ai/marksmans/dlm/runs/0a0cd474) |
| e2d2-w32/629M-100f/pre-train | 19844 | 765,386,240 | [link](https://wandb.ai/marksmans/dlm/runs/80c2d366) |
| e2d2-w32/629M-10f/pre-train | 1984 | 765,386,240 | [link](https://wandb.ai/marksmans/dlm/runs/44bc4b32) |
| e2d2-w32/629M-30f/pre-train | 5953 | 765,386,240 | [link](https://wandb.ai/marksmans/dlm/runs/63c6beaf) |
| e2d2-w32/629M-60f/pre-train | 11906 | 765,386,240 | [link](https://wandb.ai/marksmans/dlm/runs/2e6e3a99) |
| e2d2-w32/629M-6f/pre-train | 1190 | 765,386,240 | [link](https://wandb.ai/marksmans/dlm/runs/73e4645d) |
| e2d2-w32/66M-10f/pre-train | 14731 | 112,332,160 | [link](https://wandb.ai/marksmans/dlm/runs/a6db8e59) |
| e2d2-w32/66M-30f/pre-train | 44195 | 112,332,160 | [link](https://wandb.ai/marksmans/dlm/runs/73f36145) |
| e2d2-w32/66M-60f/pre-train | 88391 | 112,332,160 | [link](https://wandb.ai/marksmans/dlm/runs/054ceaa1) |
| e2d2-w32/66M-6f/pre-train | 8839 | 112,332,160 | [link](https://wandb.ai/marksmans/dlm/runs/026ef903) |
| e2d2-w32/944M-100f/pre-train | 13663 | 1,117,827,072 | [link](https://wandb.ai/marksmans/dlm/runs/f6acf2c2) |
| e2d2-w32/944M-10f/pre-train | 1366 | 1,117,827,072 | [link](https://wandb.ai/marksmans/dlm/runs/78871b0b) |
| e2d2-w32/944M-30f/pre-train | 4099 | 1,117,827,072 | [link](https://wandb.ai/marksmans/dlm/runs/d6ee574a) |
| e2d2-w32/944M-60f/pre-train | 8198 | 1,117,827,072 | [link](https://wandb.ai/marksmans/dlm/runs/d320542f) |
| e2d2-w32/944M-6f/pre-train | 819 | 1,117,827,072 | [link](https://wandb.ai/marksmans/dlm/runs/ef92ff23) |
| e2d2-w4/113M-100f/pre-train | 95282 | 169,601,280 | [link](https://wandb.ai/marksmans/dlm/runs/cfad7e10) |
| e2d2-w4/113M-10f/pre-train | 9528 | 169,601,280 | [link](https://wandb.ai/marksmans/dlm/runs/b175d13e) |
| e2d2-w4/113M-30f/pre-train | 28584 | 169,601,280 | [link](https://wandb.ai/marksmans/dlm/runs/38c38fa0) |
| e2d2-w4/113M-60f/pre-train | 57169 | 169,601,280 | [link](https://wandb.ai/marksmans/dlm/runs/7b440a0d) |
| e2d2-w4/113M-6f/pre-train | 5716 | 169,601,280 | [link](https://wandb.ai/marksmans/dlm/runs/b6d49dc6) |
| e2d2-w4/1233M-100f/pre-train | 10855 | 1,415,597,568 | [link](https://wandb.ai/marksmans/dlm/runs/92cac08a) |
| e2d2-w4/1233M-10f/pre-train | 1085 | 1,415,597,568 | [link](https://wandb.ai/marksmans/dlm/runs/c7b1cb57) |
| e2d2-w4/1233M-30f/pre-train | 3256 | 1,415,597,568 | [link](https://wandb.ai/marksmans/dlm/runs/1572ae1c) |
| e2d2-w4/1233M-60f/pre-train | 6513 | 1,415,597,568 | [link](https://wandb.ai/marksmans/dlm/runs/def9a88a) |
| e2d2-w4/1233M-6f/pre-train | 651 | 1,415,597,568 | [link](https://wandb.ai/marksmans/dlm/runs/04af57ba) |
| e2d2-w4/170M-100f/pre-train | 66561 | 229,775,616 | [link](https://wandb.ai/marksmans/dlm/runs/c88f2d5a) |
| e2d2-w4/170M-10f/pre-train | 6656 | 229,775,616 | [link](https://wandb.ai/marksmans/dlm/runs/a25be5c8) |
| e2d2-w4/170M-30f/pre-train | 19968 | 229,775,616 | [link](https://wandb.ai/marksmans/dlm/runs/691471a3) |
| e2d2-w4/170M-60f/pre-train | 39937 | 229,775,616 | [link](https://wandb.ai/marksmans/dlm/runs/44fb1856) |
| e2d2-w4/170M-6f/pre-train | 3993 | 229,775,616 | [link](https://wandb.ai/marksmans/dlm/runs/79cf6f07) |
| e2d2-w4/336M-100f/pre-train | 36185 | 421,191,680 | [link](https://wandb.ai/marksmans/dlm/runs/01fe245f) |
| e2d2-w4/336M-10f/pre-train | 3618 | 421,191,680 | [link](https://wandb.ai/marksmans/dlm/runs/2e211bc7) |
| e2d2-w4/336M-30f/pre-train | 10855 | 421,191,680 | [link](https://wandb.ai/marksmans/dlm/runs/9724fc66) |
| e2d2-w4/336M-60f/pre-train | 21711 | 421,191,680 | [link](https://wandb.ai/marksmans/dlm/runs/e8f1ce3b) |
| e2d2-w4/336M-6f/pre-train | 2171 | 421,191,680 | [link](https://wandb.ai/marksmans/dlm/runs/82c18687) |
| e2d2-w4/472M-100f/pre-train | 26556 | 583,499,520 | [link](https://wandb.ai/marksmans/dlm/runs/2dcdde1e) |
| e2d2-w4/472M-10f/pre-train | 2655 | 583,499,520 | [link](https://wandb.ai/marksmans/dlm/runs/bbad7a6c) |
| e2d2-w4/472M-30f/pre-train | 7966 | 583,499,520 | [link](https://wandb.ai/marksmans/dlm/runs/6fb95b8e) |
| e2d2-w4/472M-60f/pre-train | 15933 | 583,499,520 | [link](https://wandb.ai/marksmans/dlm/runs/e0eea608) |
| e2d2-w4/472M-6f/pre-train | 1593 | 583,499,520 | [link](https://wandb.ai/marksmans/dlm/runs/97eab534) |
| e2d2-w4/629M-100f/pre-train | 20366 | 745,720,320 | [link](https://wandb.ai/marksmans/dlm/runs/bb3818dc) |
| e2d2-w4/629M-10f/pre-train | 2036 | 745,720,320 | [link](https://wandb.ai/marksmans/dlm/runs/90a96dc0) |
| e2d2-w4/629M-30f/pre-train | 6109 | 745,720,320 | [link](https://wandb.ai/marksmans/dlm/runs/b47c5e70) |
| e2d2-w4/629M-60f/pre-train | 12219 | 745,720,320 | [link](https://wandb.ai/marksmans/dlm/runs/b948e3c9) |
| e2d2-w4/629M-6f/pre-train | 1221 | 745,720,320 | [link](https://wandb.ai/marksmans/dlm/runs/77adc886) |
| e2d2-w4/66M-10f/pre-train | 15100 | 109,873,280 | [link](https://wandb.ai/marksmans/dlm/runs/2b7d661a) |
| e2d2-w4/66M-30f/pre-train | 45302 | 109,873,280 | [link](https://wandb.ai/marksmans/dlm/runs/3c7e5536) |
| e2d2-w4/66M-60f/pre-train | 90605 | 109,873,280 | [link](https://wandb.ai/marksmans/dlm/runs/0aabab12) |
| e2d2-w4/66M-6f/pre-train | 9060 | 109,873,280 | [link](https://wandb.ai/marksmans/dlm/runs/31e595df) |
| e2d2-w4/944M-100f/pre-train | 13930 | 1,096,588,800 | [link](https://wandb.ai/marksmans/dlm/runs/6bcba193) |
| e2d2-w4/944M-10f/pre-train | 1393 | 1,096,588,800 | [link](https://wandb.ai/marksmans/dlm/runs/f6e3fad2) |
| e2d2-w4/944M-30f/pre-train | 4179 | 1,096,588,800 | [link](https://wandb.ai/marksmans/dlm/runs/b35236a1) |
| e2d2-w4/944M-60f/pre-train | 8358 | 1,096,588,800 | [link](https://wandb.ai/marksmans/dlm/runs/9bee5c12) |
| e2d2-w4/944M-6f/pre-train | 835 | 1,096,588,800 | [link](https://wandb.ai/marksmans/dlm/runs/90486546) |
| maskarm/113M-100f/pre-train | 104137 | 155,437,824 | [link](https://wandb.ai/marksmans/dlm/runs/dcbba724) |
| maskarm/113M-10f/pre-train | 10413 | 155,437,824 | [link](https://wandb.ai/marksmans/dlm/runs/fed5724d) |
| maskarm/113M-30f/pre-train | 31241 | 155,437,824 | [link](https://wandb.ai/marksmans/dlm/runs/ccfd0b74) |
| maskarm/113M-60f/pre-train | 62482 | 155,437,824 | [link](https://wandb.ai/marksmans/dlm/runs/f53a9e0c) |
| maskarm/113M-6f/pre-train | 6248 | 155,437,824 | [link](https://wandb.ai/marksmans/dlm/runs/706c0614) |
| maskarm/1233M-100f/pre-train | 12083 | 1,271,060,224 | [link](https://wandb.ai/marksmans/dlm/runs/25c96e77) |
| maskarm/1233M-10f/pre-train | 1208 | 1,271,060,224 | [link](https://wandb.ai/marksmans/dlm/runs/5fd0e102) |
| maskarm/1233M-30f/pre-train | 3624 | 1,271,060,224 | [link](https://wandb.ai/marksmans/dlm/runs/d46f44f6) |
| maskarm/1233M-60f/pre-train | 7249 | 1,271,060,224 | [link](https://wandb.ai/marksmans/dlm/runs/61a5a9c8) |
| maskarm/1233M-6f/pre-train | 724 | 1,271,060,224 | [link](https://wandb.ai/marksmans/dlm/runs/e2aad288) |
| maskarm/170M-100f/pre-train | 73072 | 208,531,200 | [link](https://wandb.ai/marksmans/dlm/runs/03692e1c) |
| maskarm/170M-10f/pre-train | 7307 | 208,531,200 | [link](https://wandb.ai/marksmans/dlm/runs/d6fa6544) |
| maskarm/170M-30f/pre-train | 21921 | 208,531,200 | [link](https://wandb.ai/marksmans/dlm/runs/b5f6f351) |
| maskarm/170M-60f/pre-train | 43843 | 208,531,200 | [link](https://wandb.ai/marksmans/dlm/runs/2facff2f) |
| maskarm/170M-6f/pre-train | 4384 | 208,531,200 | [link](https://wandb.ai/marksmans/dlm/runs/9ed41f79) |
| maskarm/336M-100f/pre-train | 40144 | 380,281,856 | [link](https://wandb.ai/marksmans/dlm/runs/b0b8b338) |
| maskarm/336M-10f/pre-train | 4014 | 380,281,856 | [link](https://wandb.ai/marksmans/dlm/runs/2fd46801) |
| maskarm/336M-30f/pre-train | 12043 | 380,281,856 | [link](https://wandb.ai/marksmans/dlm/runs/0c08d46e) |
| maskarm/336M-60f/pre-train | 24086 | 380,281,856 | [link](https://wandb.ai/marksmans/dlm/runs/0c974c6c) |
| maskarm/336M-6f/pre-train | 2408 | 380,281,856 | [link](https://wandb.ai/marksmans/dlm/runs/0bfe33b1) |
| maskarm/472M-100f/pre-train | 29462 | 524,499,200 | [link](https://wandb.ai/marksmans/dlm/runs/bae81779) |
| maskarm/472M-10f/pre-train | 2946 | 524,499,200 | [link](https://wandb.ai/marksmans/dlm/runs/1ab970e1) |
| maskarm/472M-30f/pre-train | 8838 | 524,499,200 | [link](https://wandb.ai/marksmans/dlm/runs/175abc06) |
| maskarm/472M-60f/pre-train | 17677 | 524,499,200 | [link](https://wandb.ai/marksmans/dlm/runs/8b5db358) |
| maskarm/472M-6f/pre-train | 1767 | 524,499,200 | [link](https://wandb.ai/marksmans/dlm/runs/48230244) |
| maskarm/629M-100f/pre-train | 22588 | 671,970,560 | [link](https://wandb.ai/marksmans/dlm/runs/a1fed9ff) |
| maskarm/629M-10f/pre-train | 2258 | 671,970,560 | [link](https://wandb.ai/marksmans/dlm/runs/5114f700) |
| maskarm/629M-30f/pre-train | 6776 | 671,970,560 | [link](https://wandb.ai/marksmans/dlm/runs/3c2a812f) |
| maskarm/629M-60f/pre-train | 13553 | 671,970,560 | [link](https://wandb.ai/marksmans/dlm/runs/fe50c118) |
| maskarm/629M-6f/pre-train | 1355 | 671,970,560 | [link](https://wandb.ai/marksmans/dlm/runs/cbe5e716) |
| maskarm/66M-10f/pre-train | 16241 | 102,495,360 | [link](https://wandb.ai/marksmans/dlm/runs/e9951cdc) |
| maskarm/66M-30f/pre-train | 48724 | 102,495,360 | [link](https://wandb.ai/marksmans/dlm/runs/2ca77f8a) |
| maskarm/66M-60f/pre-train | 97449 | 102,495,360 | [link](https://wandb.ai/marksmans/dlm/runs/f73b8bfe) |
| maskarm/66M-6f/pre-train | 9744 | 102,495,360 | [link](https://wandb.ai/marksmans/dlm/runs/3d72daf7) |
| maskarm/944M-100f/pre-train | 15467 | 983,314,944 | [link](https://wandb.ai/marksmans/dlm/runs/cc51834f) |
| maskarm/944M-10f/pre-train | 1546 | 983,314,944 | [link](https://wandb.ai/marksmans/dlm/runs/208dab87) |
| maskarm/944M-30f/pre-train | 4640 | 983,314,944 | [link](https://wandb.ai/marksmans/dlm/runs/23e134ba) |
| maskarm/944M-60f/pre-train | 9280 | 983,314,944 | [link](https://wandb.ai/marksmans/dlm/runs/01863cb5) |
| maskarm/944M-6f/pre-train | 928 | 983,314,944 | [link](https://wandb.ai/marksmans/dlm/runs/e84b2d5a) |