Instructions to use WeeAIResearch/Phoenix-69B-A6.6B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use WeeAIResearch/Phoenix-69B-A6.6B with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf WeeAIResearch/Phoenix-69B-A6.6B
# Run inference directly in the terminal:
llama cli -hf WeeAIResearch/Phoenix-69B-A6.6B

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf WeeAIResearch/Phoenix-69B-A6.6B
# Run inference directly in the terminal:
llama cli -hf WeeAIResearch/Phoenix-69B-A6.6B

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf WeeAIResearch/Phoenix-69B-A6.6B
# Run inference directly in the terminal:
./build/bin/llama cli -hf WeeAIResearch/Phoenix-69B-A6.6B

Use Docker

docker model run hf.co/WeeAIResearch/Phoenix-69B-A6.6B

Llama new
LM Studio
Jan

vLLM

How to use WeeAIResearch/Phoenix-69B-A6.6B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "WeeAIResearch/Phoenix-69B-A6.6B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "WeeAIResearch/Phoenix-69B-A6.6B",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/WeeAIResearch/Phoenix-69B-A6.6B

Ollama
How to use WeeAIResearch/Phoenix-69B-A6.6B with Ollama:
```
ollama run hf.co/WeeAIResearch/Phoenix-69B-A6.6B
```
Unsloth Desktop

How to use WeeAIResearch/Phoenix-69B-A6.6B with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf WeeAIResearch/Phoenix-69B-A6.6B

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "WeeAIResearch/Phoenix-69B-A6.6B"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use WeeAIResearch/Phoenix-69B-A6.6B with Docker Model Runner:
```
docker model run hf.co/WeeAIResearch/Phoenix-69B-A6.6B
```

Lemonade

How to use WeeAIResearch/Phoenix-69B-A6.6B with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull WeeAIResearch/Phoenix-69B-A6.6B

Run and chat with the model

lemonade run user.Phoenix-69B-A6.6B-{{QUANT_TAG}}

List all available models

lemonade list

Hermes Agent

How to use WeeAIResearch/Phoenix-69B-A6.6B with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf WeeAIResearch/Phoenix-69B-A6.6B

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default WeeAIResearch/Phoenix-69B-A6.6B

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use WeeAIResearch/Phoenix-69B-A6.6B with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf WeeAIResearch/Phoenix-69B-A6.6B

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "WeeAIResearch/Phoenix-69B-A6.6B" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Phoenix-69B-A6.6B / runtime /make_expert_map.py

WeeAIResearch

Release Phoenix-69B-A6.6B: v21 coding-restored GGUF

7428366 verified 8 days ago

Raw History Blame Contribute Delete

1.86 kB

"""Build a machine-local exact-expert map from GGUF tensor metadata."""
from pathlib import Path
import argparse,struct,sys

def main():
    p=argparse.ArgumentParser();p.add_argument('model',type=Path);p.add_argument('output',type=Path);p.add_argument('--llama-source',type=Path,default=Path('llama.cpp'));a=p.parse_args()
    sys.path.insert(0,str(a.llama_source.resolve()/'gguf-py'))
    import gguf
    model=a.model.resolve();st=model.stat();reader=gguf.GGUFReader(model);ts={t.name:t for t in reader.tensors}
    payload=bytearray(b'WEEPF001'+struct.pack('<QQQQI',st.st_size,st.st_dev,st.st_ino,st.st_mtime_ns,48));count=0
    for layer in range(48):
        experts=[]
        for tier in range(4):
            suffix='' if tier==0 else f'_tier{tier}'
            names=[f'blk.{layer}.ffn_{kind}_exps{suffix}.weight' for kind in ('up','gate','down')]
            if names[0] not in ts:
                assert all(n not in ts for n in names);continue
            tensors=[ts[n] for n in names];n=int(tensors[0].shape[2])
            assert all(len(t.shape)==3 and int(t.shape[2])==n and t.n_bytes%n==0 for t in tensors)
            for e in range(n):
                ranges=[]
                for t in tensors:
                    length=int(t.n_bytes)//n;offset=int(t.data_offset)+e*length
                    assert 0<length<=8*2**20 and 0<=offset<offset+length<=st.st_size
                    ranges.append((offset,length))
                experts.append(ranges)
        assert len(experts)==int(ts[f'blk.{layer}.ffn_gate_inp.weight'].shape[1]);count+=len(experts)
        payload+=struct.pack('<I',len(experts))
        for expert in experts:
            for offset,length in expert:payload+=struct.pack('<QQ',offset,length)
    a.output.write_bytes(payload);print(f'Mapped {count} expert slots into {a.output}')

if __name__=='__main__':main()

	"""Build a machine-local exact-expert map from GGUF tensor metadata."""
	from pathlib import Path
	import argparse,struct,sys

	def main():
	p=argparse.ArgumentParser();p.add_argument('model',type=Path);p.add_argument('output',type=Path);p.add_argument('--llama-source',type=Path,default=Path('llama.cpp'));a=p.parse_args()
	sys.path.insert(0,str(a.llama_source.resolve()/'gguf-py'))
	import gguf
	model=a.model.resolve();st=model.stat();reader=gguf.GGUFReader(model);ts={t.name:t for t in reader.tensors}
	payload=bytearray(b'WEEPF001'+struct.pack('<QQQQI',st.st_size,st.st_dev,st.st_ino,st.st_mtime_ns,48));count=0
	for layer in range(48):
	experts=[]
	for tier in range(4):
	suffix='' if tier==0 else f'_tier{tier}'
	names=[f'blk.{layer}.ffn_{kind}_exps{suffix}.weight' for kind in ('up','gate','down')]
	if names[0] not in ts:
	assert all(n not in ts for n in names);continue
	tensors=[ts[n] for n in names];n=int(tensors[0].shape[2])
	assert all(len(t.shape)==3 and int(t.shape[2])==n and t.n_bytes%n==0 for t in tensors)
	for e in range(n):
	ranges=[]
	for t in tensors:
	length=int(t.n_bytes)//n;offset=int(t.data_offset)+e*length
	assert 0<length<=82*20 and 0<=offset<offset+length<=st.st_size
	ranges.append((offset,length))
	experts.append(ranges)
	assert len(experts)==int(ts[f'blk.{layer}.ffn_gate_inp.weight'].shape[1]);count+=len(experts)
	payload+=struct.pack('<I',len(experts))
	for expert in experts:
	for offset,length in expert:payload+=struct.pack('<QQ',offset,length)
	a.output.write_bytes(payload);print(f'Mapped {count} expert slots into {a.output}')

	if __name__=='__main__':main()