Instructions to use EARAI/EARAI0.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use EARAI/EARAI0.1 with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download EARAI/EARAI0.1 --local-dir EARAI0.1
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
File size: 6,035 Bytes
f255843 2100fa6 f255843 2100fa6 21de506 2100fa6 66781ee 2100fa6 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | ---
language:
- en
tags:
- code
- code-generation
- research
- mlx
- experimental
---

# EARAI 0.1
**RESEARCH PREVIEW · 166.9M PARAMETERS · CHECKPOINT 87,776**
EARAI 0.1 is an early research model for code generation. It explores a structured representation of programming context alongside a generative decoder. This release shares the weights and tokenizer; the research implementation is not included.
> **Status:** the model is experimental and is not ready for autonomous programming or production use. Its current execution-based benchmark results are reported below, including failures.
---
## Evaluation

The release was frozen before evaluation. One deterministic candidate was generated per task and tested in a network-isolated container. No external model or tool was used to repair the candidate.
| Evaluation | Tasks | Passed | Other observations |
|---|---:|---:|---|
| [HumanEval](https://github.com/openai/human-eval), full set | 164 | **0 / 164** | 17 prompts exceeded the total output budget; 123 generated programs parsed as Python |
| [Sanitized MBPP](https://github.com/google-research/google-research/tree/master/mbpp), sampled | 30 | **0 / 30** | 0 raw outputs parsed as Python in this sample |
For HumanEval, the public function prefix was supplied to the decoder and counted toward the 768-token output budget. Tasks that did not fit are counted as not solved. Both models' HumanEval continuations were cut at the same top-level stop patterns before execution. The syntax count includes the supplied prefix and does **not** imply the function is correct.
The MBPP figure is a deterministic random sample from the sanitized dataset, **not** the standard complete MBPP test split. The prompt explicitly requested Python and included one visible test. The raw output was executed without code extraction. Full task-level outcomes and the dataset checksums are provided in [`benchmark_outcomes.jsonl`](benchmark_outcomes.jsonl) and [`metrics.json`](metrics.json).
### Matched small-model comparison
On the **same 20 HumanEval tasks**, this frozen checkpoint solved **0 / 20**. For reference, under the same protocol (forced prefix, 768-token budget, isolated container): [Qwen2.5-0.5B](https://huggingface.co/Qwen/Qwen2.5-0.5B) (494M) solved **7 / 20**, [Qwen2.5-Coder-0.5B](https://huggingface.co/Qwen/Qwen2.5-Coder-0.5B) (494M) solved **3 / 20**, and [TinyStarCoderPy](https://huggingface.co/bigcode/tiny_starcoder_py) (164M) solved **1 / 20** under the local test. On the same 10 sampled MBPP tasks, all four solved 0 / 10 with the raw-output procedure. This is a small comparison, not a leaderboard or a claim that the models are generally equivalent. Their tokenizers and model interfaces differ, and the same numeric token budget does not imply identical amounts of text. The market-model runs are reproducible with `research/bench_humaneval_market.py`; per-task outcomes are in [`market_eval/`](market_eval/).
As a check on the executor, 10 sampled reference implementations from each benchmark passed their original tests. This check does not rule out every evaluation issue or training-data contamination.
## Data and training
The model was trained from scratch on code and task data derived from open-source sources, with examples in Python, C, C++, C#, Go, Swift and JavaScript. The training pipeline also included a small set of execution-checked repair examples and teacher-generated hints. Language coverage in training is **not** a verified claim of programming ability in each language.
This checkpoint was produced on an Apple M5 Mac with 16 GB unified memory. The training context ceiling was 8,192 graph nodes; graph nodes are **not** tokens. A 256K-token context window or reliable use of distant project context has not been established.
## Files and use
| File | Contents |
|---|---|
| [`model.safetensors`](model.safetensors) | Frozen model weights at step 87,776 |
| [`earai_tokenizer.json`](earai_tokenizer.json) | EARAI tokenizer data |
| [`release_info.json`](release_info.json) | Parameter count, checkpoint and weight checksum |
| [`metrics.json`](metrics.json) | Evaluation configuration and aggregate results |
| [`benchmark_outcomes.jsonl`](benchmark_outcomes.jsonl) | Task IDs and outcomes; no benchmark reference solutions |
**Loading limitation:** this is a custom MLX research checkpoint, not a standard Hugging Face Transformers model. The tokenizer is also in an EARAI-specific format. The inference implementation is not included in this release, so `AutoModel` / `AutoTokenizer` cannot load these files as a ready-to-use application.
The weights are public and can be downloaded or copied. No model-weight license is specified in this card.
## Limitations
- The model frequently produces incorrect, repetitive or incomplete code.
- The current results do not establish independent project-level programming ability.
- Long output and long-context use remain constrained and require further testing.
- Public-code training data can include errors, duplication or licensing constraints; the data filters are not a proof of semantic quality or absence of sensitive material.
- Benchmark results depend on prompt format, tokenization, decoding and execution rules. The matched 20-task comparison is too small to establish a general ranking.
---
### Кратко по-русски
EARAI 0.1 — исследовательская модель для генерации кода. В полном HumanEval эта зафиксированная версия решила **0 из 164** задач; 17 заданий не поместились в лимит. Модель пока не готова для самостоятельной разработки. В репозитории опубликованы веса и токенизатор, но исходная реализация и готовый загрузчик не включены.
|