Instructions to use EdgeLLMModal1/env-pro-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use EdgeLLMModal1/env-pro-gguf with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf EdgeLLMModal1/env-pro-gguf:Q4_K_M # Run inference directly in the terminal: llama cli -hf EdgeLLMModal1/env-pro-gguf:Q4_K_M
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf EdgeLLMModal1/env-pro-gguf:Q4_K_M # Run inference directly in the terminal: llama cli -hf EdgeLLMModal1/env-pro-gguf:Q4_K_M
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf EdgeLLMModal1/env-pro-gguf:Q4_K_M # Run inference directly in the terminal: ./build/bin/llama cli -hf EdgeLLMModal1/env-pro-gguf:Q4_K_M
Use Docker
docker model run hf.co/EdgeLLMModal1/env-pro-gguf:Q4_K_M
- Llama new
- LM Studio
- Jan
- vLLM
How to use EdgeLLMModal1/env-pro-gguf with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "EdgeLLMModal1/env-pro-gguf" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "EdgeLLMModal1/env-pro-gguf", "prompt": "Once upon a time,", "max_tokens": 512, "temperature": 0.5 }'Use Docker
docker model run hf.co/EdgeLLMModal1/env-pro-gguf:Q4_K_M
- Ollama
How to use EdgeLLMModal1/env-pro-gguf with Ollama:
ollama run hf.co/EdgeLLMModal1/env-pro-gguf:Q4_K_M
- Unsloth Desktop
- Docker Model Runner
How to use EdgeLLMModal1/env-pro-gguf with Docker Model Runner:
docker model run hf.co/EdgeLLMModal1/env-pro-gguf:Q4_K_M
- Lemonade
How to use EdgeLLMModal1/env-pro-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull EdgeLLMModal1/env-pro-gguf:Q4_K_M
Run and chat with the model
lemonade run user.env-pro-gguf-Q4_K_M
List all available models
lemonade list
- Atomic Chat
Download app.py from EdgeLLMModal1/env-pro-gguf: direct link, hf CLI and curl.
- Browser
- Download file 3.48 kB
-
https://huggingface.co/EdgeLLMModal1/env-pro-gguf/resolve/main/app.py
- Command line
-
hf download hf://EdgeLLMModal1/env-pro-gguf/app.py
-
curl -L -o app.py https://huggingface.co/EdgeLLMModal1/env-pro-gguf/resolve/main/app.py
3.48 kB
| from flask import Flask, request | |
| from llama_cpp import Llama | |
| import requests | |
| from bs4 import BeautifulSoup | |
| ENV_URL = "http://192.168.4.1/" # your Station AP page | |
| llm = Llama(model_path="model/llama-3.2-1b-Q4_K_M.gguf") | |
| app = Flask(__name__) | |
| def fetch_envpro_values(): | |
| r = requests.get(ENV_URL, timeout=3) | |
| soup = BeautifulSoup(r.text, "html.parser") | |
| text = soup.get_text() | |
| def extract(label): | |
| start = text.find(label) | |
| if start == -1: | |
| return None | |
| value = text[start+len(label):].split("\n")[0].strip() | |
| return value | |
| return { | |
| "temp": extract("Temp:"), | |
| "hum": extract("Hum:"), | |
| "press": extract("Press:"), | |
| "gas": extract("Gas:"), | |
| "co2": extract("CO2eq:"), | |
| "voc": extract("VOCeq:"), | |
| "iaq": extract("IAQ accuracy:") | |
| } | |
| def interpret(): | |
| data = fetch_envpro_values() | |
| prompt = ( | |
| "ENV PRO DATA:\n" | |
| f"Temp: {data['temp']}\n" | |
| f"Hum: {data['hum']}\n" | |
| f"Press: {data['press']}\n" | |
| f"Gas: {data['gas']}\n" | |
| f"CO2eq: {data['co2']}\n" | |
| f"VOCeq: {data['voc']}\n" | |
| f"IAQ accuracy: {data['iaq']}\n\n" | |
| "Context: Indoor environment. Do not assume cooking unless the sensor values indicate it.\n" | |
| "Instruction: Base your interpretation ONLY on the ENV PRO DATA values above.\n" | |
| "Do NOT invent activities.\n" | |
| "Do NOT infer smells that are not supported by the data.\n" | |
| "The environment may be a bedroom, living room, hallway, kitchen or any indoor space.\n\n" | |
| "Task: Output exactly ONE short sentence.\n" | |
| "Rules:\n" | |
| "1. The sentence MUST start with one of these words: smell, activity, mood, doing\n" | |
| "2. The sentence MUST NOT exceed 100 characters.\n" | |
| "3. Do NOT output examples.\n" | |
| "4. Do NOT output lists.\n" | |
| "5. Do NOT explain anything.\n" | |
| "6. Output exactly ONE short sentence.\n\n" | |
| "Now output the sentence:\n" | |
| ) | |
| out = llm(prompt, max_tokens=160) | |
| return out["choices"][0]["text"] | |
| # ⭐ NEW: Manual input mode (works inside Docker Desktop) | |
| def manual(): | |
| data = request.json | |
| prompt = ( | |
| "ENV PRO DATA:\n" | |
| f"Temp: {data['temp']}\n" | |
| f"Hum: {data['hum']}\n" | |
| f"Press: {data['press']}\n" | |
| f"Gas: {data['gas']}\n" | |
| f"CO2eq: {data['co2']}\n" | |
| f"VOCeq: {data['voc']}\n" | |
| f"IAQ accuracy: {data['iaq']}\n\n" | |
| "Context: Indoor environment. Do not assume cooking unless the sensor values indicate it.\n" | |
| "Instruction: Base your interpretation ONLY on the ENV PRO DATA values above.\n" | |
| "Do NOT invent activities.\n" | |
| "Do NOT infer smells that are not supported by the data.\n" | |
| "The environment may be a bedroom, living room, hallway, kitchen or any indoor space.\n\n" | |
| "Task: Output exactly ONE short sentence.\n" | |
| "Rules:\n" | |
| "1. The sentence MUST start with one of these words: smell, activity, mood, doing\n" | |
| "2. The sentence MUST NOT exceed 100 characters.\n" | |
| "3. Do NOT output examples.\n" | |
| "4. Do NOT output lists.\n" | |
| "5. Do NOT explain anything.\n" | |
| "6. Output exactly ONE short sentence.\n\n" | |
| "Now output the sentence:\n" | |
| ) | |
| out = llm(prompt, max_tokens=160) | |
| return out["choices"][0]["text"] | |
| app.run(host="0.0.0.0", port=8000) | |