Add HuggingFace metadata tags

66fd6b1 verified about 1 month ago

9.26 kB

---
language:
  - en
  - es
tags:
  - iot
  - smart-home
  - home-assistant
  - tool-calling
  - agent
  - cybersecurity
  - devops
  - ollama
  - inference-pipeline
  - autonomous-agent
  - raspberry-pi
  - qwen3-coder
license: mit
library_name: transformers
pipeline_tag: text-generation
---
# TokioAI-Coder IoT Package

**Turn a 30B open-weight model into a fully autonomous IoT/smart-home agent.**

This package contains everything needed to deploy TokioAI-Coder as an autonomous IoT controller:
- The complete **TokioAI Engine** -- inference pipeline that wraps any Ollama model and adds reliable tool calling
- A production-tested **IoT system prompt** with a full Home Assistant cookbook
- **12 native tools** for system administration, file operations, network diagnostics, and robot vision
- **Server-side investigation** -- the engine autonomously chains 8+ tool calls to answer a single question

## What It Does

You say: *"is Home Assistant running?"*

The engine:
1. Checks `systemctl status home-assistant` -- not found
2. Checks `docker ps | grep home` -- not found
3. Checks `ss -tlnp | grep 8123` -- port listening
4. Checks `curl -s http://localhost:8123/api/` -- HA API responds
5. Answers: "Yes, HA is running on port 8123 via Docker"

All 5 steps happen **in a single API call**. The client sees only the final answer.

You say: *"play some blues"*

The engine:
1. Calls HA API to send `play_media` to the Alexa
2. Waits 3 seconds
3. Queries the media player state to verify the music actually changed
4. Reports: "Now playing: Blues Essentials on Jarvis"

## Architecture

```
                     TokioAI Engine (OpenAI-compatible API)
                     ====================================
User Query ──> Router ──> Ollama Model ──> Verifier ──> Tool Executor ──>
                 |              |              |              |
              Intent        System         Retry          Local
            Classifier     Prompt +       Loop (2x)     Execution
            (0ms regex)   Few-Shot                    (read-only safe)
                 |              |              |              |
                 v              v              v              v
              tool_call     Guided         Validates      Feeds result
              / text /      Generation     JSON +         back to model
              decision      (schema-       correct        for next step
                            constrained)   tool names
                                                              |
                                                              v
                                                    Investigation Loop
                                                    (up to 8 rounds,
                                                     90s timeout)
                                                              |
                                                              v
                                                      Final Answer
```

## Components

### 1. TokioAI Engine (`engine/`)
The core inference pipeline (3,774 lines of Python):

| Module | Lines | Purpose |
|---|---|---|
| `server.py` | 864 | OpenAI-compatible API with server-side investigation |
| `schemas.py` | 497 | Tool definitions, JSON validation, parameter schemas |
| `pipeline.py` | 476 | Main orchestrator: Router -> Memory -> Model -> Verifier |
| `router.py` | 382 | Intent classifier (regex) + system prompt builder |
| `memory.py` | 353 | Adaptive few-shot memory (learns from successes) |
| `executor.py` | 331 | Safe server-side tool execution |
| `guided.py` | 228 | JSON schema constraints for Ollama |
| `verifier.py` | 186 | Output validation + retry logic |

### 2. IoT System Prompt (`prompts/iot_system_prompt.md`)
A production-tested system prompt that includes:
- Complete Home Assistant API cookbook (media, lights, switches, sensors, vacuum, TTS)
- Alexa voice control via `alexa_media` integration
- Robot vision via moondream (1.4B vision model)
- Investigation playbooks for multi-step problem solving
- Anti-hallucination rules (never simulate tool execution)
- Verification protocol (always confirm state changes actually happened)

### 3. Tool Definitions (`tools/`)
12 native tools in OpenAI function-calling format:
- `execute_local` -- Run shell commands
- `execute_raspi` / `execute_gcp` / `execute_router` -- Remote execution
- `read_file` / `write_file` / `edit_file` -- File operations
- `search_files` -- grep across codebases
- `diagnose` -- System health checks
- `robot_vision` -- Camera capture + AI analysis
- `memory` -- Persistent memory across sessions
- `task` -- Task tracking for multi-session projects

## Quick Start

### Prerequisites
- Python 3.8+
- [Ollama](https://ollama.ai/) with a model (recommended: `qwen3-coder` 30B MoE)
- Home Assistant instance (for IoT features)

### Install

```bash
# Clone this repo
git clone https://huggingface.co/MrMoz33/tokioai-coder-iot
cd tokioai-coder-iot
# Install dependencies
pip install flask requests
# Pull a model
ollama pull qwen3-coder  # 30B MoE (3B active), recommended
# or: ollama pull qwen2.5-coder:7b  # lighter alternative
```

### Configure Home Assistant

Set your HA token:
```bash
export HA_TOKEN="your-long-lived-access-token"
export HA_URL="http://localhost:8123"  # or your HA URL
```

### Start

```bash
# Start the engine
python -m engine serve --model qwen3-coder --port 8080
# Use it (any OpenAI-compatible client works)
curl http://localhost:8080/v1/chat/completions \
  -H "Content-Type: application/json" \
  -d '{
    "model": "tokioai-engine",
    "messages": [{"role": "user", "content": "is home assistant running?"}]
  }'
```

### With TokioAI CLI

```bash
pip install git+https://github.com/TokioAI/tokioai.git
export TOKIOAI_ENGINE_HOST=http://localhost:8080
tokioai  # Interactive agent with full tool calling
```

## Customization

### Adding Your Own Home Assistant Entities

Edit `prompts/iot_system_prompt.md` and add your entities to the cookbook section:

```markdown
LIGHTS: light.living_room, light.bedroom, light.kitchen
SWITCHES: switch.fan, switch.heater
SENSORS: sensor.temperature, sensor.humidity
MEDIA: media_player.living_room_speaker
```

### Adding Custom Tools

Edit `engine/schemas.py` to add new tool definitions:

```python
{
    "type": "function",
    "function": {
        "name": "my_custom_tool",
        "description": "What this tool does",
        "parameters": {
            "type": "object",
            "properties": {
                "param1": {"type": "string", "description": "First param"}
            },
            "required": ["param1"]
        }
    }
}
```

Then add the executor in `engine/executor.py`.

### Adapting for Different Smart Home Platforms

The system prompt is platform-agnostic in concept. To use with:
- **Google Home**: Replace HA API calls with Google Home API
- **Apple HomeKit**: Use HomeKit Accessory Protocol
- **MQTT Direct**: Replace curl commands with `mosquitto_pub/sub`

The engine and tool pipeline remain identical -- only the system prompt changes.

## Performance

Tested on Raspberry Pi 5 (8GB) + L4 GPU (via SSH tunnel):

| Metric | Value |
|---|---|
| Model | Qwen3-Coder 30B (3B active MoE) |
| GPU VRAM | 19.4 GB (L4 24GB) |
| Response time | 3-8s (simple), 15-45s (investigation) |
| Investigation depth | Up to 8 rounds per query |
| Tool calling accuracy | 94%+ with engine pipeline |
| Uptime | 30+ days continuous (systemd service) |

### Cost Comparison

| Approach | Monthly Cost | Latency | Privacy |
|---|---|---|---|
| GPT-4o API | $50-200 | 2-5s | Data sent to cloud |
| Claude API | $50-200 | 2-5s | Data sent to cloud |
| **TokioAI-Coder (self-hosted)** | **$0** | **3-8s** | **100% local** |

After initial GPU investment ($0.50-1.50/hr for L4 on GCP, or free on local hardware), all inference is free and private.

## Model Compatibility

| Model | Params | Tool Calling | IoT Quality | Notes |
|---|---|---|---|---|
| qwen3-coder | 30B (3B active) | Excellent | Excellent | Recommended. MoE = fast inference |
| qwen2.5-coder:7b | 7B | Good | Good | Budget option |
| llama3.1:8b | 8B | Good | Fair | General purpose |
| deepseek-coder-v2:16b | 16B | Good | Good | Strong code understanding |
| mistral:7b | 7B | OK | Fair | Needs guided gen |

The engine's guided generation + retry logic compensates for weaker models.

## Related Projects

- [TokioAI CLI](https://github.com/TokioAI/tokioai) -- Multi-provider AI CLI with native tool calling
- [TokioAI Fine-Tuning Guide](https://github.com/daletoniris/tokioai-fine-tuning-guide) -- Train your own tool-calling model
- [TokioAI Fine-Tuning Paper](https://github.com/daletoniris/tokioai-fine-tuning-paper) -- Research paper on LLM fine-tuning for tool use
- [TokioNav](https://github.com/daletoniris/tokionav) -- AI-controlled autonomous robot navigation

## License

MIT

## Credits

Built by Daniel Dieser ([daletoniris](https://github.com/daletoniris)) and the TokioAI team.

The TokioAI Engine was developed to make small open models competitive with GPT-4 and Claude for autonomous tool calling -- specifically for IoT, cybersecurity, and DevOps tasks. The key insight: you don't need a bigger model, you need a smarter pipeline around it.

	---
	language:
	- en
	- es
	tags:
	- iot
	- smart-home
	- home-assistant
	- tool-calling
	- agent
	- cybersecurity
	- devops
	- ollama
	- inference-pipeline
	- autonomous-agent
	- raspberry-pi
	- qwen3-coder
	license: mit
	library_name: transformers
	pipeline_tag: text-generation
	---

	# TokioAI-Coder IoT Package

	Turn a 30B open-weight model into a fully autonomous IoT/smart-home agent.

	This package contains everything needed to deploy TokioAI-Coder as an autonomous IoT controller:
	- The complete TokioAI Engine -- inference pipeline that wraps any Ollama model and adds reliable tool calling
	- A production-tested IoT system prompt with a full Home Assistant cookbook
	- 12 native tools for system administration, file operations, network diagnostics, and robot vision
	- Server-side investigation -- the engine autonomously chains 8+ tool calls to answer a single question

	## What It Does

	You say: "is Home Assistant running?"

	The engine:
	1. Checks `systemctl status home-assistant` -- not found
	2. Checks `docker ps \| grep home` -- not found
	3. Checks `ss -tlnp \| grep 8123` -- port listening
	4. Checks `curl -s http://localhost:8123/api/` -- HA API responds
	5. Answers: "Yes, HA is running on port 8123 via Docker"

	All 5 steps happen in a single API call. The client sees only the final answer.

	You say: "play some blues"

	The engine:
	1. Calls HA API to send `play_media` to the Alexa
	2. Waits 3 seconds
	3. Queries the media player state to verify the music actually changed
	4. Reports: "Now playing: Blues Essentials on Jarvis"

	## Architecture

	```
	TokioAI Engine (OpenAI-compatible API)
	====================================
	User Query ──> Router ──> Ollama Model ──> Verifier ──> Tool Executor ──>
	\| \| \| \|
	Intent System Retry Local
	Classifier Prompt + Loop (2x) Execution
	(0ms regex) Few-Shot (read-only safe)
	\| \| \| \|
	v v v v
	tool_call Guided Validates Feeds result
	/ text / Generation JSON + back to model
	decision (schema- correct for next step
	constrained) tool names
	\|
	v
	Investigation Loop
	(up to 8 rounds,
	90s timeout)
	\|
	v
	Final Answer
	```

	## Components

	### 1. TokioAI Engine (`engine/`)
	The core inference pipeline (3,774 lines of Python):

	\| Module \| Lines \| Purpose \|
	\|---\|---\|---\|
	\| `server.py` \| 864 \| OpenAI-compatible API with server-side investigation \|
	\| `schemas.py` \| 497 \| Tool definitions, JSON validation, parameter schemas \|
	\| `pipeline.py` \| 476 \| Main orchestrator: Router -> Memory -> Model -> Verifier \|
	\| `router.py` \| 382 \| Intent classifier (regex) + system prompt builder \|
	\| `memory.py` \| 353 \| Adaptive few-shot memory (learns from successes) \|
	\| `executor.py` \| 331 \| Safe server-side tool execution \|
	\| `guided.py` \| 228 \| JSON schema constraints for Ollama \|
	\| `verifier.py` \| 186 \| Output validation + retry logic \|

	### 2. IoT System Prompt (`prompts/iot_system_prompt.md`)
	A production-tested system prompt that includes:
	- Complete Home Assistant API cookbook (media, lights, switches, sensors, vacuum, TTS)
	- Alexa voice control via `alexa_media` integration
	- Robot vision via moondream (1.4B vision model)
	- Investigation playbooks for multi-step problem solving
	- Anti-hallucination rules (never simulate tool execution)
	- Verification protocol (always confirm state changes actually happened)

	### 3. Tool Definitions (`tools/`)
	12 native tools in OpenAI function-calling format:
	- `execute_local` -- Run shell commands
	- `execute_raspi` / `execute_gcp` / `execute_router` -- Remote execution
	- `read_file` / `write_file` / `edit_file` -- File operations
	- `search_files` -- grep across codebases
	- `diagnose` -- System health checks
	- `robot_vision` -- Camera capture + AI analysis
	- `memory` -- Persistent memory across sessions
	- `task` -- Task tracking for multi-session projects

	## Quick Start

	### Prerequisites
	- Python 3.8+
	- [Ollama](https://ollama.ai/) with a model (recommended: `qwen3-coder` 30B MoE)
	- Home Assistant instance (for IoT features)

	### Install

	```bash
	# Clone this repo
	git clone https://huggingface.co/MrMoz33/tokioai-coder-iot
	cd tokioai-coder-iot

	# Install dependencies
	pip install flask requests

	# Pull a model
	ollama pull qwen3-coder # 30B MoE (3B active), recommended
	# or: ollama pull qwen2.5-coder:7b # lighter alternative
	```

	### Configure Home Assistant

	Set your HA token:
	```bash
	export HA_TOKEN="your-long-lived-access-token"
	export HA_URL="http://localhost:8123" # or your HA URL
	```

	### Start

	```bash
	# Start the engine
	python -m engine serve --model qwen3-coder --port 8080

	# Use it (any OpenAI-compatible client works)
	curl http://localhost:8080/v1/chat/completions \
	-H "Content-Type: application/json" \
	-d '{
	"model": "tokioai-engine",
	"messages": [{"role": "user", "content": "is home assistant running?"}]
	}'
	```

	### With TokioAI CLI

	```bash
	pip install git+https://github.com/TokioAI/tokioai.git
	export TOKIOAI_ENGINE_HOST=http://localhost:8080
	tokioai # Interactive agent with full tool calling
	```

	## Customization

	### Adding Your Own Home Assistant Entities

	Edit `prompts/iot_system_prompt.md` and add your entities to the cookbook section:

	```markdown
	LIGHTS: light.living_room, light.bedroom, light.kitchen
	SWITCHES: switch.fan, switch.heater
	SENSORS: sensor.temperature, sensor.humidity
	MEDIA: media_player.living_room_speaker
	```

	### Adding Custom Tools

	Edit `engine/schemas.py` to add new tool definitions:

	```python
	{
	"type": "function",
	"function": {
	"name": "my_custom_tool",
	"description": "What this tool does",
	"parameters": {
	"type": "object",
	"properties": {
	"param1": {"type": "string", "description": "First param"}
	},
	"required": ["param1"]
	}
	}
	}
	```

	Then add the executor in `engine/executor.py`.

	### Adapting for Different Smart Home Platforms

	The system prompt is platform-agnostic in concept. To use with:
	- Google Home: Replace HA API calls with Google Home API
	- Apple HomeKit: Use HomeKit Accessory Protocol
	- MQTT Direct: Replace curl commands with `mosquitto_pub/sub`

	The engine and tool pipeline remain identical -- only the system prompt changes.

	## Performance

	Tested on Raspberry Pi 5 (8GB) + L4 GPU (via SSH tunnel):

	\| Metric \| Value \|
	\|---\|---\|
	\| Model \| Qwen3-Coder 30B (3B active MoE) \|
	\| GPU VRAM \| 19.4 GB (L4 24GB) \|
	\| Response time \| 3-8s (simple), 15-45s (investigation) \|
	\| Investigation depth \| Up to 8 rounds per query \|
	\| Tool calling accuracy \| 94%+ with engine pipeline \|
	\| Uptime \| 30+ days continuous (systemd service) \|

	### Cost Comparison

	\| Approach \| Monthly Cost \| Latency \| Privacy \|
	\|---\|---\|---\|---\|
	\| GPT-4o API \| $50-200 \| 2-5s \| Data sent to cloud \|
	\| Claude API \| $50-200 \| 2-5s \| Data sent to cloud \|
	\| TokioAI-Coder (self-hosted) \| $0 \| 3-8s \| 100% local \|

	After initial GPU investment ($0.50-1.50/hr for L4 on GCP, or free on local hardware), all inference is free and private.

	## Model Compatibility

	\| Model \| Params \| Tool Calling \| IoT Quality \| Notes \|
	\|---\|---\|---\|---\|---\|
	\| qwen3-coder \| 30B (3B active) \| Excellent \| Excellent \| Recommended. MoE = fast inference \|
	\| qwen2.5-coder:7b \| 7B \| Good \| Good \| Budget option \|
	\| llama3.1:8b \| 8B \| Good \| Fair \| General purpose \|
	\| deepseek-coder-v2:16b \| 16B \| Good \| Good \| Strong code understanding \|
	\| mistral:7b \| 7B \| OK \| Fair \| Needs guided gen \|

	The engine's guided generation + retry logic compensates for weaker models.

	## Related Projects

	- [TokioAI CLI](https://github.com/TokioAI/tokioai) -- Multi-provider AI CLI with native tool calling
	- [TokioAI Fine-Tuning Guide](https://github.com/daletoniris/tokioai-fine-tuning-guide) -- Train your own tool-calling model
	- [TokioAI Fine-Tuning Paper](https://github.com/daletoniris/tokioai-fine-tuning-paper) -- Research paper on LLM fine-tuning for tool use
	- [TokioNav](https://github.com/daletoniris/tokionav) -- AI-controlled autonomous robot navigation

	## License

	MIT

	## Credits

	Built by Daniel Dieser ([daletoniris](https://github.com/daletoniris)) and the TokioAI team.

	The TokioAI Engine was developed to make small open models competitive with GPT-4 and Claude for autonomous tool calling -- specifically for IoT, cybersecurity, and DevOps tasks. The key insight: you don't need a bigger model, you need a smarter pipeline around it.