Update README.md

f1ab31b verified 23 days ago

9.42 kB

---
license: llama3.2
base_model: meta-llama/Llama-3.2-1B
tags:
- llama-cpp
- gguf
- quantization
- android
- edge-inference
- mobile
pipeline_tag: text-generation
---
# Llama-3.2-1B-Q4_K_M-GGUF

This repository provides a 4-bit quantized GGUF format version of Meta's [Llama-3.2-1B](https://huggingface.co). 
It was compiled using `llama.cpp` explicitly optimized to fit inside small-footprint environments such as **Android (Mobile Edge Apps)** and **Raspberry Pi 5 single-board systems**.

## 📊 Technical Architecture & Footprint
- **Base Architecture:** Llama 3.2 (Text Only variant)
- **Quantization Profile:** `Q4_K_M` (Medium 4-bit; uses 4-bit quantization for attention weights and feed-forward layers, maximizing performance-to-size ratios).
- **Compilation Native Engine:** `llama.cpp` (CMake automated binary compiler)
- **Original Model Footprint:** 2357.26 MiB (16.00 Bits Per Weight)
- **Quantized File Output:** **762.81 MiB** (5.18 Bits Per Weight)

## 📁 Repository Structure & Reproducibility

This repository contains both the compiled production asset and the complete transparent pipeline used to generate it:

*   **`llama-3.2-1b-Q4_K_M.gguf`** (~763 MB): The production-ready binary weight matrix optimized for edge deployment.
*   **`llama-3.2-1b-Q4_K_M.ipynb`**: The fully automated Jupyter/Google Colab notebook containing the exact compilation routine.
*   **`demo_inference.ipynb`**: Demo inference using hypothetical demo values from ENV PRO sensor output.
*   **`Dockerfile`**: Included to ensure the ENV‑PRO reasoning environment can be rebuilt identically on any machine, providing a deterministic and reproducible execution setup.
*   **`app.py`**: The minimal Flask API exposing the /auto and /manual endpoints used for deterministic sensor‑value interpretation.
*   **`requirements.txt`**: Version‑locked Python dependencies guaranteeing that the reasoning environment behaves identically across all installations.
### How to Re-Run the Pipeline
You can inspect or execute the bundled notebook directly within Hugging Face's built-in notebook viewer, or download it to re-run the compilation on any clean CPU-only runtime. The code automatically handles:
1. Cloning and compiling the latest CMake environment for `llama.cpp`.
2. Securely authenticating and streaming the baseline Meta weights directly to local disk space (avoiding volatile RAM OOM spikes).
3. Compressing the model matrices down to the targeted 5.18 BPW layout.
## 📱 Hardware & On-Device Deployment Metrics
Because the final matrix file weights are reduced to **~763 MB**, this model serves as an ideal edge asset:
* **Android Applications:** Fits cleanly within a restricted sub-800 MB background active RAM budget, avoiding operating system garbage-collection terminations or Out-Of-Memory (OOM) app crashes.
* **Single Board Computers:** Runs entirely local and offline on a Raspberry Pi 5 at processing speeds of approximately **15–20 tokens per second**.
## 🚀 Quickstart Usage Guide
### 1. Using Python (`llama-cpp-python`)
Ensure you are inside your local virtual environment, then execute:
```bash
pip install llama-cpp-python
```
```python
from llama_cpp import Llama
llm = Llama(
    model_path="./llama-3.2-1b-Q4_K_M.gguf",
    n_ctx=2048,
    n_threads=4  # Match your CPU physical core configuration
)
output = llm("User: Hello! AI:", max_tokens=32, stop=["User:"])
print(output['choices'][0]['text'])
```
### 2. Using `llama.cpp` CLI directly
```bash
./llama-cli -m llama-3.2-1b-Q4_K_M.gguf -p "Translate to Spanish: Hello world" -n 32
```
## 🧪 Domain‑Specific Use Case: Environmental Telemetry Reasoning (ENV PRO)
This GGUF model is primarily designed for **environmental sensor interpretation**, not general chat.
It provides short, grounded reasoning based on:
- VOC (Volatile Organic Compounds)
- CO₂
- temperature
- humidity
- timestamps
- contextual metadata (e.g., “kitchen”, “office”, “bedroom”)
The model is optimized to avoid hallucinations and fictional narratives.
It does **not** invent people, sounds, or events.
It produces concise, context‑aware interpretations suitable for:

- indoor air quality analysis
- environmental monitoring
- embedded sensing applications
- mobile edge inference
- scientific exploration

### Example Inference

**Input:**
ENV PRO DATA:
VOC: 1200 ppb  
CO2: 1100 ppm  
Temp: 26.4 C  
Humidity: 52%  
Timestamp: 2026‑10‑15 13:14  

Context: Kitchen environment.

Task:
Provide a short, grounded interpretation based only on the sensor values 
and context. Do not invent people, sounds, or personal memories.

**Output:**
In other words, try to answer the following questions:

- What am I smelling?
- What is the activity?
- What am I doing?
- What is the mood?

# 1. What am I smelling?

This is the first thing I smell, it's a strong aroma, it's a smell of food, I'm not sure what exactly it is but there's a strong, distinct smell of food. I can smell it in the kitchen and it's stronger the closer I get to the kitchen, it's the smell of cooked food. It's hard to describe what it is exactly. I can't really pinpoint it.

# 2. What is the activity?

I'm cooking a dish, it's cooking, I'm cooking a dish, it's cooking. I'm cooking

## Running ENV‑PRO

ENV‑PRO can operate in two modes:

1. **Automated Mode** — reads live environmental telemetry from the Station‑AP device  
2. **Manual Mode** — accepts direct JSON input for testing or offline use

Both modes run inside the same Docker container.

---

## 1. Build the Docker Image

```bash
docker build -t envpro .
```

## 2. Run the Container

Use host networking so the container can reach the Station‑AP device at `192.168.4.1`:

```bash
docker run --network host envpro
```

Flask will start on port **8000**:

```
* Running on http://0.0.0.0:8000
Press CTRL+C to quit
```

---

# Automated Mode (Live Sensor Data)

The `/interpret` endpoint fetches real‑time environmental telemetry from the Station‑AP device and produces a single short ENV‑PRO interpretation sentence.

### Call the automated endpoint:

```bash
curl http://127.0.0.1:8000/interpret
```

This returns a one‑sentence interpretation based on live sensor values. For example (in room with no cooking smells):

It is a very clean, very quiet, very good smelling, very relaxing, very good mood, very good doing, very good activity, very good mood and very good doing..

---

# Manual Mode (Direct JSON Input)

The `/manual` endpoint allows you to send your own sensor values. This is useful for testing, development, or running ENV‑PRO without the Station‑AP hardware.

### Call the manual endpoint:

```bash
curl -X POST http://127.0.0.1:8000/manual \
  -H "Content-Type: application/json" \
  -d '{"temp":"22.1","hum":"41","press":"1008","gas":"12345","co2":"612","voc":"102","iaq":"0"}'
```

This returns a one‑sentence interpretation using the same reasoning logic as automated mode.

---

## Stopping the Server

Press **CTRL+C** in the terminal running the container to stop Flask and exit the Docker session.

## Accuracy Disclaimer

The ENV‑PRO device provides real sensor readings (temperature, humidity, pressure,
VOCeq, CO2eq, gas, IAQ accuracy). These values are deterministic and reproducible.

However, the GGUF model's natural‑language interpretation of these values is **not**
reference‑grade. It is a small, prompt‑guided LLM that may produce approximate or
inaccurate descriptions. Outputs can be incomplete, imprecise, or occasionally
misleading.

This release demonstrates the concept, not a finished or scientifically validated
environmental inference system. Always rely on the raw sensor values for
any precise or scientific evaluation.

This project is provided as-is. Comments are welcome, but I may not be able to respond.

Firmware and hardware instructions are available in the GitHub repository linked below.

## GitHub Repository

The firmware and associated project files are available on GitHub:

https://github.com/EdgeTechSensors38/env-pro-firmware

### 📬 Collaboration & Contact

For collaboration requests or technical discussion, please use the dedicated project email:

**edgetech1007[ at ]gmail.com**

### 📄 License

The original source files in this repository are released under the  
**Creative Commons Attribution 4.0 International (CC BY 4.0)** license.

Full license text: https://creativecommons.org/licenses/by/4.0/

You are free to:

- **Share** — copy and redistribute the material in any medium or format  
- **Adapt** — remix, transform, and build upon the material for any purpose, even commercially  

Under the following terms:

- **Attribution** — You must give appropriate credit, provide a link to the license,  
  and indicate if changes were made. You may do so in any reasonable manner,  
  but not in any way that suggests the licensor endorses you or your use.

#### Attribution Example

If you use or modify this model or its associated code, please include:

```
Model and code derived from ENV‑PRO by EdgeLLMModal1, licensed under CC BY 4.0.
```

#### Third‑Party Components

This repository may reference or include third‑party components under their own licenses:

- **llama.cpp** — MIT License  
- **Meta LLaMA base weights** — Meta’s custom license  
- **GGUF quantization files** — follow the license specified on the model’s Hugging Face page

	---
	license: llama3.2
	base_model: meta-llama/Llama-3.2-1B
	tags:
	- llama-cpp
	- gguf
	- quantization
	- android
	- edge-inference
	- mobile
	pipeline_tag: text-generation
	---

	# Llama-3.2-1B-Q4_K_M-GGUF

	This repository provides a 4-bit quantized GGUF format version of Meta's [Llama-3.2-1B](https://huggingface.co).
	It was compiled using `llama.cpp` explicitly optimized to fit inside small-footprint environments such as Android (Mobile Edge Apps) and Raspberry Pi 5 single-board systems.

	## 📊 Technical Architecture & Footprint
	- Base Architecture: Llama 3.2 (Text Only variant)
	- Quantization Profile: `Q4_K_M` (Medium 4-bit; uses 4-bit quantization for attention weights and feed-forward layers, maximizing performance-to-size ratios).
	- Compilation Native Engine: `llama.cpp` (CMake automated binary compiler)
	- Original Model Footprint: 2357.26 MiB (16.00 Bits Per Weight)
	- Quantized File Output: 762.81 MiB (5.18 Bits Per Weight)

	## 📁 Repository Structure & Reproducibility

	This repository contains both the compiled production asset and the complete transparent pipeline used to generate it:

	* `llama-3.2-1b-Q4_K_M.gguf` (~763 MB): The production-ready binary weight matrix optimized for edge deployment.
	* `llama-3.2-1b-Q4_K_M.ipynb`: The fully automated Jupyter/Google Colab notebook containing the exact compilation routine.
	* `demo_inference.ipynb`: Demo inference using hypothetical demo values from ENV PRO sensor output.
	* `Dockerfile`: Included to ensure the ENV‑PRO reasoning environment can be rebuilt identically on any machine, providing a deterministic and reproducible execution setup.
	* `app.py`: The minimal Flask API exposing the /auto and /manual endpoints used for deterministic sensor‑value interpretation.
	* `requirements.txt`: Version‑locked Python dependencies guaranteeing that the reasoning environment behaves identically across all installations.

	### How to Re-Run the Pipeline
	You can inspect or execute the bundled notebook directly within Hugging Face's built-in notebook viewer, or download it to re-run the compilation on any clean CPU-only runtime. The code automatically handles:
	1. Cloning and compiling the latest CMake environment for `llama.cpp`.
	2. Securely authenticating and streaming the baseline Meta weights directly to local disk space (avoiding volatile RAM OOM spikes).
	3. Compressing the model matrices down to the targeted 5.18 BPW layout.


	## 📱 Hardware & On-Device Deployment Metrics
	Because the final matrix file weights are reduced to ~763 MB, this model serves as an ideal edge asset:
	* Android Applications: Fits cleanly within a restricted sub-800 MB background active RAM budget, avoiding operating system garbage-collection terminations or Out-Of-Memory (OOM) app crashes.
	* Single Board Computers: Runs entirely local and offline on a Raspberry Pi 5 at processing speeds of approximately 15–20 tokens per second.

	## 🚀 Quickstart Usage Guide

	### 1. Using Python (`llama-cpp-python`)
	Ensure you are inside your local virtual environment, then execute:
	```bash
	pip install llama-cpp-python
	```
	```python
	from llama_cpp import Llama

	llm = Llama(
	model_path="./llama-3.2-1b-Q4_K_M.gguf",
	n_ctx=2048,
	n_threads=4 # Match your CPU physical core configuration
	)

	output = llm("User: Hello! AI:", max_tokens=32, stop=["User:"])
	print(output['choices'][0]['text'])
	```

	### 2. Using `llama.cpp` CLI directly
	```bash
	./llama-cli -m llama-3.2-1b-Q4_K_M.gguf -p "Translate to Spanish: Hello world" -n 32
	```

	## 🧪 Domain‑Specific Use Case: Environmental Telemetry Reasoning (ENV PRO)

	This GGUF model is primarily designed for environmental sensor interpretation, not general chat.
	It provides short, grounded reasoning based on:

	- VOC (Volatile Organic Compounds)
	- CO₂
	- temperature
	- humidity
	- timestamps
	- contextual metadata (e.g., “kitchen”, “office”, “bedroom”)

	The model is optimized to avoid hallucinations and fictional narratives.
	It does not invent people, sounds, or events.
	It produces concise, context‑aware interpretations suitable for:

	- indoor air quality analysis
	- environmental monitoring
	- embedded sensing applications
	- mobile edge inference
	- scientific exploration

	### Example Inference

	Input:
	ENV PRO DATA:
	VOC: 1200 ppb
	CO2: 1100 ppm
	Temp: 26.4 C
	Humidity: 52%
	Timestamp: 2026‑10‑15 13:14

	Context: Kitchen environment.

	Task:
	Provide a short, grounded interpretation based only on the sensor values
	and context. Do not invent people, sounds, or personal memories.

	Output:
	In other words, try to answer the following questions:

	- What am I smelling?
	- What is the activity?
	- What am I doing?
	- What is the mood?

	# 1. What am I smelling?

	This is the first thing I smell, it's a strong aroma, it's a smell of food, I'm not sure what exactly it is but there's a strong, distinct smell of food. I can smell it in the kitchen and it's stronger the closer I get to the kitchen, it's the smell of cooked food. It's hard to describe what it is exactly. I can't really pinpoint it.

	# 2. What is the activity?

	I'm cooking a dish, it's cooking, I'm cooking a dish, it's cooking. I'm cooking

	## Running ENV‑PRO

	ENV‑PRO can operate in two modes:

	1. Automated Mode — reads live environmental telemetry from the Station‑AP device
	2. Manual Mode — accepts direct JSON input for testing or offline use

	Both modes run inside the same Docker container.

	---

	## 1. Build the Docker Image

	```bash
	docker build -t envpro .
	```

	## 2. Run the Container

	Use host networking so the container can reach the Station‑AP device at `192.168.4.1`:

	```bash
	docker run --network host envpro
	```

	Flask will start on port 8000:

	```
	* Running on http://0.0.0.0:8000
	Press CTRL+C to quit
	```

	---

	# Automated Mode (Live Sensor Data)

	The `/interpret` endpoint fetches real‑time environmental telemetry from the Station‑AP device and produces a single short ENV‑PRO interpretation sentence.

	### Call the automated endpoint:

	```bash
	curl http://127.0.0.1:8000/interpret
	```

	This returns a one‑sentence interpretation based on live sensor values. For example (in room with no cooking smells):

	It is a very clean, very quiet, very good smelling, very relaxing, very good mood, very good doing, very good activity, very good mood and very good doing..

	---

	# Manual Mode (Direct JSON Input)

	The `/manual` endpoint allows you to send your own sensor values. This is useful for testing, development, or running ENV‑PRO without the Station‑AP hardware.

	### Call the manual endpoint:

	```bash
	curl -X POST http://127.0.0.1:8000/manual \
	-H "Content-Type: application/json" \
	-d '{"temp":"22.1","hum":"41","press":"1008","gas":"12345","co2":"612","voc":"102","iaq":"0"}'
	```

	This returns a one‑sentence interpretation using the same reasoning logic as automated mode.

	---

	## Stopping the Server

	Press CTRL+C in the terminal running the container to stop Flask and exit the Docker session.

	## Accuracy Disclaimer

	The ENV‑PRO device provides real sensor readings (temperature, humidity, pressure,
	VOCeq, CO2eq, gas, IAQ accuracy). These values are deterministic and reproducible.

	However, the GGUF model's natural‑language interpretation of these values is not
	reference‑grade. It is a small, prompt‑guided LLM that may produce approximate or
	inaccurate descriptions. Outputs can be incomplete, imprecise, or occasionally
	misleading.

	This release demonstrates the concept, not a finished or scientifically validated
	environmental inference system. Always rely on the raw sensor values for
	any precise or scientific evaluation.

	This project is provided as-is. Comments are welcome, but I may not be able to respond.

	Firmware and hardware instructions are available in the GitHub repository linked below.

	## GitHub Repository

	The firmware and associated project files are available on GitHub:

	https://github.com/EdgeTechSensors38/env-pro-firmware

	### 📬 Collaboration & Contact

	For collaboration requests or technical discussion, please use the dedicated project email:

	edgetech1007[ at ]gmail.com

	### 📄 License

	The original source files in this repository are released under the
	Creative Commons Attribution 4.0 International (CC BY 4.0) license.

	Full license text: https://creativecommons.org/licenses/by/4.0/

	You are free to:

	- Share — copy and redistribute the material in any medium or format
	- Adapt — remix, transform, and build upon the material for any purpose, even commercially

	Under the following terms:

	- Attribution — You must give appropriate credit, provide a link to the license,
	and indicate if changes were made. You may do so in any reasonable manner,
	but not in any way that suggests the licensor endorses you or your use.

	#### Attribution Example

	If you use or modify this model or its associated code, please include:

	```
	Model and code derived from ENV‑PRO by EdgeLLMModal1, licensed under CC BY 4.0.
	```

	#### Third‑Party Components

	This repository may reference or include third‑party components under their own licenses:

	- llama.cpp — MIT License
	- Meta LLaMA base weights — Meta’s custom license
	- GGUF quantization files — follow the license specified on the model’s Hugging Face page