Instructions to use miutti/intel-mac-local-llm with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use miutti/intel-mac-local-llm with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf miutti/intel-mac-local-llm:UD-Q2_K_XL
# Run inference directly in the terminal:
llama cli -hf miutti/intel-mac-local-llm:UD-Q2_K_XL

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf miutti/intel-mac-local-llm:UD-Q2_K_XL
# Run inference directly in the terminal:
llama cli -hf miutti/intel-mac-local-llm:UD-Q2_K_XL

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf miutti/intel-mac-local-llm:UD-Q2_K_XL
# Run inference directly in the terminal:
./llama-cli -hf miutti/intel-mac-local-llm:UD-Q2_K_XL

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf miutti/intel-mac-local-llm:UD-Q2_K_XL
# Run inference directly in the terminal:
./build/bin/llama-cli -hf miutti/intel-mac-local-llm:UD-Q2_K_XL

Use Docker

docker model run hf.co/miutti/intel-mac-local-llm:UD-Q2_K_XL

Llama new
LM Studio
Jan

vLLM

How to use miutti/intel-mac-local-llm with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "miutti/intel-mac-local-llm"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "miutti/intel-mac-local-llm",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/miutti/intel-mac-local-llm:UD-Q2_K_XL

Ollama
How to use miutti/intel-mac-local-llm with Ollama:
```
ollama run hf.co/miutti/intel-mac-local-llm:UD-Q2_K_XL
```
Unsloth Desktop

How to use miutti/intel-mac-local-llm with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf miutti/intel-mac-local-llm:UD-Q2_K_XL

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "miutti/intel-mac-local-llm:UD-Q2_K_XL"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use miutti/intel-mac-local-llm with Docker Model Runner:
```
docker model run hf.co/miutti/intel-mac-local-llm:UD-Q2_K_XL
```

Lemonade

How to use miutti/intel-mac-local-llm with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull miutti/intel-mac-local-llm:UD-Q2_K_XL

Run and chat with the model

lemonade run user.intel-mac-local-llm-UD-Q2_K_XL

List all available models

lemonade list

Hermes Agent

How to use miutti/intel-mac-local-llm with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf miutti/intel-mac-local-llm:UD-Q2_K_XL

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default miutti/intel-mac-local-llm:UD-Q2_K_XL

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use miutti/intel-mac-local-llm with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf miutti/intel-mac-local-llm:UD-Q2_K_XL

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "miutti/intel-mac-local-llm:UD-Q2_K_XL" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

intel-mac-local-llm / source /kernel /council_bench.py

miutti

Source: kernel, desktop app, tools (snapshot of the GitHub repo)

df41178 verified 23 days ago

Raw History Blame Contribute Delete

2.94 kB

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
council_bench.py -- 「長く考えると、どれだけ解けるようになるか」を測る
  ひとりで探すと、どの手から試すかで当たり外れが出る。
  試せる通り数（＝考える時間）を絞って、
  ひとり と 5人の合議 で、解ける数がどう変わるかを見る。
"""
import os, sys, io, time, contextlib
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
import kernel, council

CASES = [
    {"動作": "移動", "場所": "Desktop", "種類": "画像", "時期": "去年"},
    {"動作": "移動", "場所": "Downloads", "種類": "PDF", "時期": "去年"},
    {"動作": "数える", "場所": "Desktop", "種類": "画像", "時期": "去年"},
    {"動作": "一覧",  "場所": "Downloads", "種類": "PDF"},
    {"動作": "重複",  "場所": "Desktop"},
    {"動作": "大きさ", "場所": "Desktop", "種類": "画像"},
    {"動作": "移動", "場所": "Desktop", "種類": "書類", "行き先": "しょるい"},
    {"動作": "数える", "場所": "Desktop", "除く": "画像"},
    {"動作": "一覧",  "場所": "Desktop", "種類": "画像", "並べ方": "大きい順"},
    {"動作": "移動", "場所": "Desktop"},
]


def solves(fn, budget, depth):
    # 実績（policy.json）が残っていると、あとの回ほど有利になって
    # 比べものにならない。毎回まっさらから測る
    pj = os.path.join(os.path.dirname(os.path.abspath(__file__)), "policy.json")
    if os.path.exists(pj):
        os.remove(pj)
    kernel._policy.cache_clear()
    ok, t0 = 0, time.time()
    for slots in CASES:
        with contextlib.redirect_stdout(io.StringIO()):
            kernel.make_demo()
            try:
                plan = fn(slots, budget, depth)
            except Exception:
                plan = None
        ok += bool(plan)
    return ok, (time.time() - t0) * 1000


def one(slots, budget, depth):
    p, _st = kernel.solve(slots, verbose=False, budget=budget, max_depth=depth)
    return p


def many(slots, budget, depth):
    p, _st, _lg = council.deliberate(slots, per_view=3, budget=budget,
                                     max_depth=depth, verbose=False)
    return p


if __name__ == "__main__":
    n = len(CASES)
    print(f"  問題 {n} 問。「試せる通り数」を絞って、解ける数を見る\n")
    print(f"  {'考える量':<12}{'ひとり':<14}{'6人で合議':<14}{'ひとり・6倍考える':<14}")
    print("  " + "─" * 62)
    for budget in (8, 12, 20, 40, 80, 300):
        a, ta = solves(one, budget, 6)
        b, tb = solves(many, budget, 6)
        c, tc = solves(one, budget * 6, 6)
        print(f"  {budget:>4}通りまで   {a}/{n} ({ta:5.0f}ms)   "
              f"{b}/{n} ({tb:5.0f}ms)   {c}/{n} ({tc:5.0f}ms)")
    print("\n  ※ どれも掛け算は0回。変えているのは考える時間だけ")

	#!/usr/bin/env python3
	# -- coding: utf-8 --
	"""
	council_bench.py -- 「長く考えると、どれだけ解けるようになるか」を測る

	ひとりで探すと、どの手から試すかで当たり外れが出る。
	試せる通り数（＝考える時間）を絞って、
	ひとりと 5人の合議で、解ける数がどう変わるかを見る。
	"""
	import os, sys, io, time, contextlib
	sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
	import kernel, council

	CASES = [
	{"動作": "移動", "場所": "Desktop", "種類": "画像", "時期": "去年"},
	{"動作": "移動", "場所": "Downloads", "種類": "PDF", "時期": "去年"},
	{"動作": "数える", "場所": "Desktop", "種類": "画像", "時期": "去年"},
	{"動作": "一覧", "場所": "Downloads", "種類": "PDF"},
	{"動作": "重複", "場所": "Desktop"},
	{"動作": "大きさ", "場所": "Desktop", "種類": "画像"},
	{"動作": "移動", "場所": "Desktop", "種類": "書類", "行き先": "しょるい"},
	{"動作": "数える", "場所": "Desktop", "除く": "画像"},
	{"動作": "一覧", "場所": "Desktop", "種類": "画像", "並べ方": "大きい順"},
	{"動作": "移動", "場所": "Desktop"},
	]


	def solves(fn, budget, depth):
	# 実績（policy.json）が残っていると、あとの回ほど有利になって
	# 比べものにならない。毎回まっさらから測る
	pj = os.path.join(os.path.dirname(os.path.abspath(__file__)), "policy.json")
	if os.path.exists(pj):
	os.remove(pj)
	kernel._policy.cache_clear()
	ok, t0 = 0, time.time()
	for slots in CASES:
	with contextlib.redirect_stdout(io.StringIO()):
	kernel.make_demo()
	try:
	plan = fn(slots, budget, depth)
	except Exception:
	plan = None
	ok += bool(plan)
	return ok, (time.time() - t0) * 1000


	def one(slots, budget, depth):
	p, _st = kernel.solve(slots, verbose=False, budget=budget, max_depth=depth)
	return p


	def many(slots, budget, depth):
	p, _st, _lg = council.deliberate(slots, per_view=3, budget=budget,
	max_depth=depth, verbose=False)
	return p


	if __name__ == "__main__":
	n = len(CASES)
	print(f" 問題 {n} 問。「試せる通り数」を絞って、解ける数を見る\n")
	print(f" {'考える量':<12}{'ひとり':<14}{'6人で合議':<14}{'ひとり・6倍考える':<14}")
	print(" " + "─" * 62)
	for budget in (8, 12, 20, 40, 80, 300):
	a, ta = solves(one, budget, 6)
	b, tb = solves(many, budget, 6)
	c, tc = solves(one, budget * 6, 6)
	print(f" {budget:>4}通りまで {a}/{n} ({ta:5.0f}ms) "
	f"{b}/{n} ({tb:5.0f}ms) {c}/{n} ({tc:5.0f}ms)")
	print("\n ※ どれも掛け算は0回。変えているのは考える時間だけ")