Instructions to use EARAI/EARAI0.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use EARAI/EARAI0.1 with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download EARAI/EARAI0.1 --local-dir EARAI0.1
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
|
Download assets/evaluation.svg from EARAI/EARAI0.1: direct link, hf CLI and curl.
- Browser
- Download file 4.62 kB
-
https://huggingface.co/EARAI/EARAI0.1/resolve/main/assets/evaluation.svg
- Command line
-
hf download hf://EARAI/EARAI0.1/assets/evaluation.svg
-
curl -L -o evaluation.svg https://huggingface.co/EARAI/EARAI0.1/resolve/main/assets/evaluation.svg
4.62 kB
| <svg xmlns="http://www.w3.org/2000/svg" width="1600" height="900" viewBox="0 0 1600 900" role="img" aria-labelledby="title desc"> | |
| <title id="title">EARAI 0.1 HumanEval research evaluation</title> | |
| <desc id="desc">EARAI 0.1 on the full HumanEval set: 147 of 164 tasks fit the output budget, 123 parsed as Python, none passed the tests. On a matched sample of 20 HumanEval tasks, solved counts were: Qwen2.5-0.5B 7 of 20, Qwen2.5-Coder-0.5B 3 of 20, TinyStarCoderPy 1 of 20, EARAI 0.1 0 of 20.</desc> | |
| <rect width="1600" height="900" fill="#0a0a0a"/> | |
| <text x="100" y="96" fill="#8a8a8a" font-family="Arial, Helvetica, sans-serif" font-size="20" letter-spacing="6">EARAI / RESEARCH EVALUATION</text> | |
| <text x="100" y="182" fill="#ffffff" font-family="Arial, Helvetica, sans-serif" font-size="64" font-weight="bold">HumanEval</text> | |
| <text x="100" y="224" fill="#9a9a9a" font-family="Arial, Helvetica, sans-serif" font-size="23">frozen checkpoint 87,776 路 166.9M parameters</text> | |
| <line x1="100" y1="260" x2="1500" y2="260" stroke="#262626"/> | |
| <text x="100" y="314" fill="#7a7a7a" font-family="Arial, Helvetica, sans-serif" font-size="19" letter-spacing="4">FULL HUMANEVAL 路 164 TASKS</text> | |
| <text x="100" y="374" fill="#e6e6e6" font-family="Arial, Helvetica, sans-serif" font-size="25">Tasks within output budget</text> | |
| <rect x="560" y="361" width="840" height="8" rx="4" fill="#1c1c1c"/> | |
| <rect x="560" y="361" width="752" height="8" rx="4" fill="#ffffff"/> | |
| <text x="1500" y="380" fill="#ffffff" font-family="Arial, Helvetica, sans-serif" font-size="25" text-anchor="end">147 / 164</text> | |
| <text x="100" y="434" fill="#e6e6e6" font-family="Arial, Helvetica, sans-serif" font-size="25">Python syntax accepted</text> | |
| <rect x="560" y="421" width="840" height="8" rx="4" fill="#1c1c1c"/> | |
| <rect x="560" y="421" width="630" height="8" rx="4" fill="#cccccc"/> | |
| <text x="1500" y="440" fill="#ffffff" font-family="Arial, Helvetica, sans-serif" font-size="25" text-anchor="end">123 / 164</text> | |
| <text x="100" y="494" fill="#e6e6e6" font-family="Arial, Helvetica, sans-serif" font-size="25">Passed execution tests</text> | |
| <rect x="560" y="481" width="840" height="8" rx="4" fill="#1c1c1c"/> | |
| <text x="1500" y="500" fill="#8a8a8a" font-family="Arial, Helvetica, sans-serif" font-size="25" text-anchor="end">0 / 164</text> | |
| <line x1="100" y1="537" x2="1500" y2="537" stroke="#262626"/> | |
| <text x="100" y="588" fill="#7a7a7a" font-family="Arial, Helvetica, sans-serif" font-size="19" letter-spacing="4">MATCHED SAMPLE 路 20 TASKS 路 SOLVED</text> | |
| <text x="100" y="618" fill="#6a6a6a" font-family="Arial, Helvetica, sans-serif" font-size="17">Same forced prefix 路 768-token budget 路 isolated container. Bars show solved tasks out of 20.</text> | |
| <text x="100" y="678" fill="#e6e6e6" font-family="Arial, Helvetica, sans-serif" font-size="22">Qwen2.5-0.5B (494M)</text> | |
| <rect x="560" y="666" width="840" height="10" rx="5" fill="#1c1c1c"/> | |
| <rect x="560" y="666" width="294" height="10" rx="5" fill="#ffffff"/> | |
| <text x="1500" y="684" fill="#ffffff" font-family="Arial, Helvetica, sans-serif" font-size="22" text-anchor="end">7 / 20</text> | |
| <text x="100" y="728" fill="#e6e6e6" font-family="Arial, Helvetica, sans-serif" font-size="22">Qwen2.5-Coder-0.5B (494M)</text> | |
| <rect x="560" y="716" width="840" height="10" rx="5" fill="#1c1c1c"/> | |
| <rect x="560" y="716" width="126" height="10" rx="5" fill="#b8b8b8"/> | |
| <text x="1500" y="734" fill="#e6e6e6" font-family="Arial, Helvetica, sans-serif" font-size="22" text-anchor="end">3 / 20</text> | |
| <text x="100" y="778" fill="#e6e6e6" font-family="Arial, Helvetica, sans-serif" font-size="22">TinyStarCoderPy (164M)</text> | |
| <rect x="560" y="766" width="840" height="10" rx="5" fill="#1c1c1c"/> | |
| <rect x="560" y="766" width="42" height="10" rx="5" fill="#6e6e6e"/> | |
| <text x="1500" y="784" fill="#b0b0b0" font-family="Arial, Helvetica, sans-serif" font-size="22" text-anchor="end">1 / 20</text> | |
| <text x="100" y="828" fill="#ffffff" font-family="Arial, Helvetica, sans-serif" font-size="22" font-weight="bold">EARAI 0.1 (167M)</text> | |
| <rect x="560" y="816" width="840" height="10" rx="5" fill="#1c1c1c"/> | |
| <polygon points="544,814 544,826 555,820" fill="#ffffff"/> | |
| <text x="1500" y="834" fill="#8a8a8a" font-family="Arial, Helvetica, sans-serif" font-size="22" text-anchor="end">0 / 20</text> | |
| <text x="100" y="874" fill="#6a6a6a" font-family="Arial, Helvetica, sans-serif" font-size="16">Different tokenizers and interfaces 路 Small matched sample, not a general ranking 路 Syntax includes the supplied function prefix.</text> | |
| </svg> |