Instructions to use EzioDevio/ArduinoLLM-7B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use EzioDevio/ArduinoLLM-7B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="EzioDevio/ArduinoLLM-7B")
messages = [
    {"role": "user", "content": "Who are you?"},
]
pipe(messages)

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("EzioDevio/ArduinoLLM-7B")
model = AutoModelForCausalLM.from_pretrained("EzioDevio/ArduinoLLM-7B", device_map="auto")
messages = [
    {"role": "user", "content": "Who are you?"},
]
inputs = tokenizer.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=256)
print(tokenizer.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps Settings

vLLM

How to use EzioDevio/ArduinoLLM-7B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "EzioDevio/ArduinoLLM-7B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EzioDevio/ArduinoLLM-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/EzioDevio/ArduinoLLM-7B

SGLang

How to use EzioDevio/ArduinoLLM-7B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "EzioDevio/ArduinoLLM-7B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EzioDevio/ArduinoLLM-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "EzioDevio/ArduinoLLM-7B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "EzioDevio/ArduinoLLM-7B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Docker Model Runner
How to use EzioDevio/ArduinoLLM-7B with Docker Model Runner:
```
docker model run hf.co/EzioDevio/ArduinoLLM-7B
```

EzioDevio commited on 17 days ago

Commit

a67baf3

verified ·

1 Parent(s): 86fd8f6

Upload folder using huggingface_hub

Browse files

Files changed (1) hide show

README.md +34 -51

README.md CHANGED Viewed

@@ -1,56 +1,42 @@
 ---
 license: apache-2.0
-base_model: unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit
 tags:
   - arduino
   - esp32
   - raspberry-pi
   - micropython
   - circuitpython
-  - embedded
   - lora
-  - unsloth
 language:
   - en
-pipeline_tag: text-generation
 ---
-# Arduino/Embedded Wiring & Code Assistant (Qwen2.5-Coder-7B LoRA)
-A LoRA fine-tune of [Qwen2.5-Coder-7B-Instruct](https://huggingface.co/Qwen/Qwen2.5-Coder-7B-Instruct), specialized for generating wiring instructions and working code for embedded/maker projects.
-Given a plain-language request like *"wire a BME280 to an ESP32 over I2C"*, it produces a wiring table, an ASCII wiring diagram, complete working code, and a short explanation — across Arduino C++, Raspberry Pi Python, MicroPython, and CircuitPython, on 7 boards (ESP32, Raspberry Pi, Raspberry Pi Pico, Arduino Uno, ESP8266, Teensy 4.0, M5Stack Core2).
-**Full training pipeline, dataset, and evaluation harness:** [GitHub repo](https://github.com/YOUR-USERNAME/arduino-embedded-llm)
-## Benchmark (12-question held-out eval, deterministic decoding)
-| Metric | This model (7B, 373 examples) | Earlier 1.5B baseline (65 examples) |
-|---|---|---|
-| Format compliance | **83%** | 58% |
-| Runtime contamination-free | **100%** | 83% |
-| Code syntax valid | **~100%*** | 92% |
-| Speed (RTX 4070 8GB laptop) | 12.0 tok/s | 13.7 tok/s |
-| Speed (data-center GPU) | 73.2 tok/s | — |
-\* Two apparent syntax failures were confirmed to be 900-token generation-cap truncation, not real errors.
-"Runtime contamination" = wrong-framework API leaking into an answer (e.g. MicroPython's `machine.Pin` appearing in Arduino code, or Raspberry Pi's physical-pin numbering convention appearing in an ESP32 answer). Eliminating this was the primary goal of moving from the 1.5B to the 7B model plus a larger, validated dataset.
-We also tested a LoRA rank-32 variant: no measurable quality improvement over rank-16 on any metric, so this release ships the more efficient rank-16 adapter.
-## ⚠️ Known limitations
-This is a v0.1 release trained on 373 examples — small by any standard. Manual testing (6 hand-picked questions, not from the eval set) found a clear, honest capability boundary:
-**Reliable (3/3 tested):** straightforward single-component wiring closely resembling the training distribution — e.g. "wire a BME280 to an ESP32 over I2C," "read a DHT22 on a Raspberry Pi," "wire an SSD1306 OLED to an ESP32 in MicroPython." All three produced correct pins, correct libraries, and internally consistent answers.
-**Unreliable (0/3 tested):** novel multi-component combinations and protocol-level troubleshooting outside the training distribution. Three distinct hallucination patterns were found:
-- Internal inconsistencies within one answer (a wiring diagram wire contradicting its own table)
-- A fabricated, non-functional low-level procedure for an I2C address conflict (invented register names/values presented with confidence)
-- An invented sensor capability not physically possible on the requested hardware (a "CO2 estimate" from a sensor that cannot measure CO2)
-**Always verify wiring against the actual component datasheet before connecting real hardware.** Incorrect voltage or pin assignments can damage components. Treat outputs as a first draft, not a final instruction set.
 ## Usage
@@ -58,34 +44,31 @@ This is a v0.1 release trained on 373 examples — small by any standard. Manual
 from unsloth import FastLanguageModel
 model, tokenizer = FastLanguageModel.from_pretrained(
-    "unsloth/Qwen2.5-Coder-7B-Instruct-bnb-4bit",
     max_seq_length=2048,
     load_in_4bit=True,
 )
-model.load_adapter("YOUR-USERNAME/arduino-embedded-qwen2.5-coder-7b")
 FastLanguageModel.for_inference(model)
-messages = [{"role": "user", "content": "How do I wire a BME280 to an ESP32 over I2C?"}]
-inputs = tokenizer.apply_chat_template(
-    messages, tokenize=True, add_generation_prompt=True, return_tensors="pt"
-).to("cuda")
-outputs = model.generate(input_ids=inputs, max_new_tokens=1300, do_sample=False)
-print(tokenizer.decode(outputs[0][inputs.shape[1]:], skip_special_tokens=True))
 ```
-Minimum hardware: ~6GB VRAM for inference (tested at 8GB on an RTX 4070 laptop). A 0.5B distilled variant for CPU/Raspberry Pi Zero 2W deployment is planned.
-## Training details
-- **Base model:** Qwen2.5-Coder-7B-Instruct (4-bit, via Unsloth)
-- **Method:** LoRA, rank 16, alpha 16, all attention + MLP projections
-- **Dataset:** 373 examples, synthetically generated and validated against an automated contamination checker before merging into training data
-- **Epochs:** 3
-## License
-Apache 2.0, inherited from the base model. Free for commercial use, modification, and redistribution.
-## Citation / Acknowledgments
-Built on [Qwen2.5-Coder](https://github.com/QwenLM/Qwen2.5-Coder) (Alibaba Qwen team) using [Unsloth](https://github.com/unslothai/unsloth).

 ---
 license: apache-2.0
+base_model: Qwen/Qwen2.5-Coder-7B-Instruct
 tags:
   - arduino
   - esp32
   - raspberry-pi
   - micropython
   - circuitpython
+  - embedded-systems
+  - code-generation
   - lora
+  - continued-pretraining
 language:
   - en
+library_name: transformers
 ---
+# ArduinoLLM-7B (v2)
+A Qwen2.5-Coder-7B specialist for embedded systems wiring and code generation, covering Arduino C++, Raspberry Pi Python, MicroPython, and CircuitPython across 9 boards.
+Given a component and a board, it generates a wiring table, an ASCII wiring diagram, working code, and a short explanation of the key design decision — in one consistent format.
+## What's new in v2
+v1 was trained entirely on LLM-generated synthetic examples. v2 adds a genuine second training stage: **continued pretraining on real library source code** (~500M tokens from 240,634 real files across the Arduino core libraries, `micropython-lib`, and the Adafruit CircuitPython Bundle), followed by re-running instruction fine-tuning on top.
+**Result**: format compliance improved from 83% to 100% on a 12-question held-out benchmark, and three specific, previously-documented failure modes (an internally-inconsistent wiring diagram, a fabricated I2C conflict-resolution procedure, and an invented sensor capability) no longer reproduce.
+| Metric | v1 | v2 |
+|---|---|---|
+| Format compliance | 83% | **100%** |
+| Contamination-free | 100% | 100% |
+| Correct runtime API | 92% | 92% |
+| Syntax valid | ~100% | 100% |
+| Speed (RTX 5090) | 73.2 tok/s | 37.5 tok/s |
+v2 currently runs slower than v1 — a known tradeoff from the local re-merge process, not a quality issue. A faster-quantized export may follow.
 ## Usage
 from unsloth import FastLanguageModel
 model, tokenizer = FastLanguageModel.from_pretrained(
+    model_name="EzioDevio/ArduinoLLM-7B",
     max_seq_length=2048,
     load_in_4bit=True,
 )
 FastLanguageModel.for_inference(model)
+prompt = "How do I wire a BME280 sensor to an ESP32 over I2C, with Arduino code?"
+inputs = tokenizer(prompt, return_tensors="pt").to("cuda")
+outputs = model.generate(**inputs, max_new_tokens=1300)
+print(tokenizer.decode(outputs[0], skip_special_tokens=True))
 ```
+## Training data
+- **Synthetic**: 1,607 unique wiring/code examples, generated via the Anthropic API, Google's Gemini API, and a locally-hosted Qwen2.5-Coder-32B, validated for format compliance and runtime contamination.
+- **Real corpus**: ~500M tokens (of a 2.59B-token total) from real Arduino/MicroPython/CircuitPython library source code.
+OpenAI and DeepSeek were not used, since both explicitly prohibit using their API output to train competing models.
+## Limitations
+- Most reliable on well-represented single-component combinations.
+- v2 used only ~19% of the available real-code corpus.
+- Always verify wiring against the component's actual datasheet before connecting power.
+## Full pipeline and dataset
+See [github.com/EzioDEVio/ArduinoLLM-7B](https://github.com/EzioDEVio/ArduinoLLM-7B) for the complete training pipeline, dataset, and evaluation scripts.