Text Generation
GGUF
Japanese
English
llama.cpp
Mixture of Experts
expert-pruning
intel-mac
cpu
local-agent
imatrix
conversational
Instructions to use miutti/intel-mac-local-llm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use miutti/intel-mac-local-llm with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf miutti/intel-mac-local-llm:UD-Q2_K_XL # Run inference directly in the terminal: llama cli -hf miutti/intel-mac-local-llm:UD-Q2_K_XL
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf miutti/intel-mac-local-llm:UD-Q2_K_XL # Run inference directly in the terminal: llama cli -hf miutti/intel-mac-local-llm:UD-Q2_K_XL
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf miutti/intel-mac-local-llm:UD-Q2_K_XL # Run inference directly in the terminal: ./llama-cli -hf miutti/intel-mac-local-llm:UD-Q2_K_XL
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf miutti/intel-mac-local-llm:UD-Q2_K_XL # Run inference directly in the terminal: ./build/bin/llama-cli -hf miutti/intel-mac-local-llm:UD-Q2_K_XL
Use Docker
docker model run hf.co/miutti/intel-mac-local-llm:UD-Q2_K_XL
- LM Studio
- Jan
- vLLM
How to use miutti/intel-mac-local-llm with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "miutti/intel-mac-local-llm" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "miutti/intel-mac-local-llm", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/miutti/intel-mac-local-llm:UD-Q2_K_XL
- Ollama
How to use miutti/intel-mac-local-llm with Ollama:
ollama run hf.co/miutti/intel-mac-local-llm:UD-Q2_K_XL
- Unsloth Desktop
- Pi
How to use miutti/intel-mac-local-llm with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf miutti/intel-mac-local-llm:UD-Q2_K_XL
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "miutti/intel-mac-local-llm:UD-Q2_K_XL" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use miutti/intel-mac-local-llm with Docker Model Runner:
docker model run hf.co/miutti/intel-mac-local-llm:UD-Q2_K_XL
- Lemonade
How to use miutti/intel-mac-local-llm with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull miutti/intel-mac-local-llm:UD-Q2_K_XL
Run and chat with the model
lemonade run user.intel-mac-local-llm-UD-Q2_K_XL
List all available models
lemonade list
- Hermes Agent
How to use miutti/intel-mac-local-llm with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf miutti/intel-mac-local-llm:UD-Q2_K_XL
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default miutti/intel-mac-local-llm:UD-Q2_K_XL
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use miutti/intel-mac-local-llm with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf miutti/intel-mac-local-llm:UD-Q2_K_XL
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "miutti/intel-mac-local-llm:UD-Q2_K_XL" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download source/kernel/tests/test_chat_seiri.py from miutti/intel-mac-local-llm: direct link, hf CLI and curl.
- Browser
- Download file 6.59 kB
-
https://huggingface.co/miutti/intel-mac-local-llm/resolve/main/source/kernel/tests/test_chat_seiri.py
- Command line
-
hf download hf://miutti/intel-mac-local-llm/source/kernel/tests/test_chat_seiri.py
-
curl -L -o test_chat_seiri.py https://huggingface.co/miutti/intel-mac-local-llm/resolve/main/source/kernel/tests/test_chat_seiri.py
6.59 kB
| #!/usr/bin/env python3 | |
| """会話整理案のローカル生成・検証・実行。""" | |
| from __future__ import annotations | |
| import json | |
| from pathlib import Path | |
| import sys | |
| import tempfile | |
| import unittest | |
| from unittest.mock import patch | |
| HERE = Path(__file__).resolve().parents[1] | |
| sys.path.insert(0, str(HERE)) | |
| import chats # noqa: E402 | |
| class _Response: | |
| def __init__(self, data): | |
| self.data = json.dumps(data, ensure_ascii=False).encode("utf-8") | |
| def __enter__(self): | |
| return self | |
| def __exit__(self, *_args): | |
| return False | |
| def read(self): | |
| return self.data | |
| class ChatSeiriTests(unittest.TestCase): | |
| def setUp(self): | |
| self.tmp = tempfile.TemporaryDirectory() | |
| root = Path(self.tmp.name) | |
| self.old = chats.DIR, chats.DB, chats.TITLE_EXAMPLES, chats.SEIRI_OBOE, chats._READY | |
| chats.DIR = str(root / "legacy") | |
| chats.DB = str(root / "chats.sqlite3") | |
| chats.TITLE_EXAMPLES = str(root / "title_examples.json") | |
| chats.SEIRI_OBOE = str(root / "seiri_oboe.json") | |
| chats._READY = False | |
| def tearDown(self): | |
| chats.DIR, chats.DB, chats.TITLE_EXAMPLES, chats.SEIRI_OBOE, chats._READY = self.old | |
| self.tmp.cleanup() | |
| def _chat(self, title, first, updated=None): | |
| conversation = chats.create(title) | |
| chats.add_turn(conversation["id"], "user", first) | |
| if updated is not None: | |
| with chats._db() as conn: | |
| conn.execute("UPDATE conversations SET updated=? WHERE id=?", (updated, conversation["id"])) | |
| return chats.load(conversation["id"]) | |
| def test_qwen_json_is_read_after_think_tag_and_code_fence(self): | |
| payload = {"choices": [{"message": {"content": '<think>考え中</think>```json\n{"組":[],"ゴミ箱":[],"しまう":[]}\n```'}}]} | |
| with patch("urllib.request.urlopen", side_effect=[_Response([]), _Response(payload)]): | |
| answer = chats._ask_seiri_qwen([]) | |
| self.assertEqual(answer, {"組": [], "ゴミ箱": [], "しまう": []}) | |
| def test_broken_json_returns_none(self): | |
| payload = {"choices": [{"message": {"content": '```json\n{"組": ['}}]} | |
| with patch("urllib.request.urlopen", side_effect=[_Response([]), _Response(payload)]): | |
| self.assertIsNone(chats._ask_seiri_qwen([])) | |
| def test_nonexistent_ids_are_ignored_and_recent_chat_cannot_be_trashed(self): | |
| old = self._chat("古い会話", "内容", chats._now() - 9 * 86400) | |
| recent = self._chat("最近の会話", "別の内容") | |
| answer = {"組": [], "ゴミ箱": [ | |
| {"id": "not-real", "理由": "存在しない"}, {"id": recent["id"], "理由": "最近"}, | |
| {"id": old["id"], "理由": "不要"}], "しまう": []} | |
| proposals = chats._seiri_qwen_proposals([old, recent], answer) | |
| self.assertEqual([x["id"] for x in proposals], [old["id"]]) | |
| self.assertEqual(proposals[0]["出どころ"], "Qwen3.6") | |
| def test_recent_chat_is_not_archived_by_qwen(self): | |
| recent = self._chat("今の作業", "続きをお願い", updated=chats._now() - 86400) | |
| old = self._chat("前の作業", "もう終わった", updated=chats._now() - 5 * 86400) | |
| answer = {"組": [], "ゴミ箱": [], "しまう": [{"id": recent["id"], "理由": "短い"}, {"id": old["id"], "理由": "終わった"}]} | |
| with patch.object(chats, "_ask_seiri_qwen", return_value=answer): | |
| proposals = [p for p in chats.seiri_an() if p.get("出どころ") == "Qwen3.6"] | |
| self.assertEqual([(p["id"], p["区分"]) for p in proposals], [(old["id"], "しまう")]) | |
| def test_archived_chat_is_not_archived_again(self): | |
| done = self._chat("終わった作業", "もう済んだ", updated=chats._now() - 9 * 86400) | |
| chats.archive(done["id"], True) | |
| answer = {"組": [], "ゴミ箱": [], "しまう": [{"id": done["id"], "理由": "既にしまった"}]} | |
| with patch.object(chats, "_ask_seiri_qwen", return_value=answer): | |
| self.assertFalse([p for p in chats.seiri_an() if p.get("出どころ") == "Qwen3.6"]) | |
| def test_timeout_falls_back_to_rules_only(self): | |
| chat = self._chat("試験: 古い会話", "古い試験", chats._now() - 8 * 86400) | |
| with patch.object(chats, "_ask_seiri_qwen", return_value=None): | |
| proposals = chats.seiri_an() | |
| self.assertEqual([(p["id"], p["区分"], p["出どころ"]) for p in proposals], | |
| [(chat["id"], "ゴミ箱", "規則")]) | |
| def test_qwen_group_name_reuses_existing_name_and_limits_fields(self): | |
| first = self._chat("旅の計画", "旅の計画を相談") | |
| second = self._chat("旅の荷物", "旅の荷物を相談") | |
| existing = self._chat("既存", "既存会話") | |
| chats.set_group(existing["id"], "旅行") | |
| answer = {"組": [{"名": "旅行メモ", "ids": [first["id"], second["id"], "missing"], "理由": "理由" * 30}], | |
| "ゴミ箱": [], "しまう": []} | |
| proposals = chats._seiri_qwen_proposals([first, second], answer) | |
| self.assertEqual({p["組"] for p in proposals}, {"旅行"}) | |
| self.assertTrue(all(len(p["理由"]) <= 40 for p in proposals)) | |
| self.assertTrue(all(len(p["組"]) <= 12 for p in proposals)) | |
| def test_rejected_proposal_is_not_shown_again(self): | |
| chat = self._chat("分類候補", "相談") | |
| Path(chats.SEIRI_OBOE).write_text(json.dumps({"外した": [{"id": chat["id"], "区分": "しまう"}], "組名": []})) | |
| answer = {"組": [], "ゴミ箱": [], "しまう": [{"id": chat["id"], "理由": "しまう"}]} | |
| self.assertEqual(chats._seiri_qwen_proposals([chat], answer), []) | |
| def test_seiri_suru_executes_remembered_plan_without_calling_qwen_again(self): | |
| first = self._chat("旅行の計画", "一つ目") | |
| second = self._chat("旅行の荷物", "二つ目") | |
| answer = {"組": [{"名": "旅行", "ids": [first["id"], second["id"]], "理由": "近い話"}], | |
| "ゴミ箱": [], "しまう": []} | |
| with patch.object(chats, "_ask_seiri_qwen", return_value=answer): | |
| proposals = chats.seiri_an() | |
| with patch.object(chats, "_ask_seiri_qwen", side_effect=AssertionError("再問い合わせした")): | |
| done = chats.seiri_suru([{**p, "checked": True} for p in proposals]) | |
| self.assertEqual(done, 2) | |
| self.assertEqual(chats.load(first["id"])["組"], "旅行") | |
| self.assertEqual(chats.load(second["id"])["組"], "旅行") | |
| if __name__ == "__main__": | |
| unittest.main() | |