Instructions to use appvoid/palmer-003 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use appvoid/palmer-003 with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("text-generation", model="appvoid/palmer-003")

# pip install -U transformers accelerate
# Load model directly
from transformers import AutoTokenizer, AutoModelForCausalLM

tokenizer = AutoTokenizer.from_pretrained("appvoid/palmer-003")
model = AutoModelForCausalLM.from_pretrained("appvoid/palmer-003", device_map="auto")

Notebooks
Google Colab
Kaggle
Local Apps Settings

llama.cpp

How to use appvoid/palmer-003 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf appvoid/palmer-003
# Run inference directly in the terminal:
llama cli -hf appvoid/palmer-003

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf appvoid/palmer-003
# Run inference directly in the terminal:
llama cli -hf appvoid/palmer-003

Use pre-built binary

# Download pre-built binary from:
# https://github.com/ggerganov/llama.cpp/releases
# Start a local OpenAI-compatible server with a web UI:
./llama-server -hf appvoid/palmer-003
# Run inference directly in the terminal:
./llama-cli -hf appvoid/palmer-003

Build from source code

git clone https://github.com/ggerganov/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-server llama-cli
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama-server -hf appvoid/palmer-003
# Run inference directly in the terminal:
./build/bin/llama-cli -hf appvoid/palmer-003

Use Docker

docker model run hf.co/appvoid/palmer-003

LM Studio
Jan

vLLM

How to use appvoid/palmer-003 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "appvoid/palmer-003"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "appvoid/palmer-003",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker

docker model run hf.co/appvoid/palmer-003

SGLang

How to use appvoid/palmer-003 with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "appvoid/palmer-003" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "appvoid/palmer-003",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "appvoid/palmer-003" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "appvoid/palmer-003",
		"messages": [
			{
				"role": "user",
				"content": "What is the capital of France?"
			}
		]
	}'

Ollama
How to use appvoid/palmer-003 with Ollama:
```
ollama run hf.co/appvoid/palmer-003
```
Unsloth Desktop
Docker Model Runner
How to use appvoid/palmer-003 with Docker Model Runner:
```
docker model run hf.co/appvoid/palmer-003
```

Lemonade

How to use appvoid/palmer-003 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull appvoid/palmer-003

Run and chat with the model

lemonade run user.palmer-003-{{QUANT_TAG}}

List all available models

lemonade list

Atomic Chat

appvoid commited on Jan 27, 2024

Commit

06ff562

verified ·

1 Parent(s): e5ffce7

Update README.md

Browse files

Files changed (1) hide show

README.md +3 -11

README.md CHANGED Viewed

@@ -22,18 +22,10 @@ qwen      | 0.4536 | 0.3490 | 0.3320 | 0.5876 | 0.7307 | 0.5896 | 0.5070 | 1.8B
 This work constitutes, given its compactness, an advancement towards SMLs, easily empowering edge devices such as mobile phones, raspberry pis and automated software/robots. Aditionally, palmer-003 follows the same philosophy as palmer-002.5 to become a more powerful model with more data instead of less.
 ```
-prompt: Reality is but
-output: a dream,
-And the dreams we make are our reality.
-The world is a canvas, painted by our minds,
-And we can make it a masterpiece.
-So let us create, let us dream,
-And let our imagination run wild.
-For in our imagination lies our power,
-To create a world that is truly our own.
 ```
 You can support me [through kofi](https://ko-fi.com/appvoid), or you can download graphite widget app.

 This work constitutes, given its compactness, an advancement towards SMLs, easily empowering edge devices such as mobile phones, raspberry pis and automated software/robots. Aditionally, palmer-003 follows the same philosophy as palmer-002.5 to become a more powerful model with more data instead of less.
 ```
+prompt: Never give up.
+output: Never give up. Keep pushing forward. Remember, you are not alone. You have the strength and the support of your family and friends.
+As you continue on your journey, remember to take care of yourself. Eat well, exercise regularly, and get enough sleep. Stay positive and focused on your goals. Remember, you are capable of achieving anything you set your mind to.
 ```
 You can support me [through kofi](https://ko-fi.com/appvoid), or you can download graphite widget app.