Instructions to use EARAI/EARAI0.1 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use EARAI/EARAI0.1 with MLX:
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download EARAI/EARAI0.1 --local-dir EARAI0.1
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Atomic Chat
EARAI 0.1
RESEARCH PREVIEW · 166.9M PARAMETERS · CHECKPOINT 87,776
EARAI 0.1 is an early research model for code generation. It explores a structured representation of programming context alongside a generative decoder. This release shares the weights and tokenizer; the research implementation is not included.
Status: the model is experimental and is not ready for autonomous programming or production use. Its current execution-based benchmark results are reported below, including failures.
Evaluation
The release was frozen before evaluation. One deterministic candidate was generated per task and tested in a network-isolated container. No external model or tool was used to repair the candidate.
| Evaluation | Tasks | Passed | Other observations |
|---|---|---|---|
| HumanEval, full set | 164 | 0 / 164 | 17 prompts exceeded the total output budget; 123 generated programs parsed as Python |
| Sanitized MBPP, sampled | 30 | 0 / 30 | 0 raw outputs parsed as Python in this sample |
For HumanEval, the public function prefix was supplied to the decoder and counted toward the 768-token output budget. Tasks that did not fit are counted as not solved. Both models' HumanEval continuations were cut at the same top-level stop patterns before execution. The syntax count includes the supplied prefix and does not imply the function is correct.
The MBPP figure is a deterministic random sample from the sanitized dataset, not the standard complete MBPP test split. The prompt explicitly requested Python and included one visible test. The raw output was executed without code extraction. Full task-level outcomes and the dataset checksums are provided in benchmark_outcomes.jsonl and metrics.json.
Matched small-model comparison
On the same 20 HumanEval tasks, this frozen checkpoint solved 0 / 20. For reference, under the same protocol (forced prefix, 768-token budget, isolated container): Qwen2.5-0.5B (494M) solved 7 / 20, Qwen2.5-Coder-0.5B (494M) solved 3 / 20, and TinyStarCoderPy (164M) solved 1 / 20 under the local test. On the same 10 sampled MBPP tasks, all four solved 0 / 10 with the raw-output procedure. This is a small comparison, not a leaderboard or a claim that the models are generally equivalent. Their tokenizers and model interfaces differ, and the same numeric token budget does not imply identical amounts of text. The market-model runs are reproducible with research/bench_humaneval_market.py; per-task outcomes are in market_eval/.
As a check on the executor, 10 sampled reference implementations from each benchmark passed their original tests. This check does not rule out every evaluation issue or training-data contamination.
Data and training
The model was trained from scratch on code and task data derived from open-source sources, with examples in Python, C, C++, C#, Go, Swift and JavaScript. The training pipeline also included a small set of execution-checked repair examples and teacher-generated hints. Language coverage in training is not a verified claim of programming ability in each language.
This checkpoint was produced on an Apple M5 Mac with 16 GB unified memory. The training context ceiling was 8,192 graph nodes; graph nodes are not tokens. A 256K-token context window or reliable use of distant project context has not been established.
Files and use
| File | Contents |
|---|---|
model.safetensors |
Frozen model weights at step 87,776 |
earai_tokenizer.json |
EARAI tokenizer data |
release_info.json |
Parameter count, checkpoint and weight checksum |
metrics.json |
Evaluation configuration and aggregate results |
benchmark_outcomes.jsonl |
Task IDs and outcomes; no benchmark reference solutions |
Loading limitation: this is a custom MLX research checkpoint, not a standard Hugging Face Transformers model. The tokenizer is also in an EARAI-specific format. The inference implementation is not included in this release, so AutoModel / AutoTokenizer cannot load these files as a ready-to-use application.
The weights are public and can be downloaded or copied. No model-weight license is specified in this card.
Limitations
- The model frequently produces incorrect, repetitive or incomplete code.
- The current results do not establish independent project-level programming ability.
- Long output and long-context use remain constrained and require further testing.
- Public-code training data can include errors, duplication or licensing constraints; the data filters are not a proof of semantic quality or absence of sensitive material.
- Benchmark results depend on prompt format, tokenization, decoding and execution rules. The matched 20-task comparison is too small to establish a general ranking.
Кратко по-русски
EARAI 0.1 — исследовательская модель для генерации кода. В полном HumanEval эта зафиксированная версия решила 0 из 164 задач; 17 заданий не поместились в лимит. Модель пока не готова для самостоятельной разработки. В репозитории опубликованы веса и токенизатор, но исходная реализация и готовый загрузчик не включены.
Quantized
# Download the model from the Hub pip install huggingface_hub[hf_xet] hf download EARAI/EARAI0.1 --local-dir EARAI0.1