Text Generation
Transformers
Safetensors
GGUF
llama
mergekit
Merge
text-generation-inference
conversational
Instructions to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- Transformers
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with Transformers:
# Use a pipeline as a high-level helper from transformers import pipeline pipe = pipeline("text-generation", model="voidful/Llama-3.1-TAIDE-R1-8B-Chat")# pip install -U transformers accelerate # Load model directly from transformers import AutoTokenizer, AutoModelForCausalLM tokenizer = AutoTokenizer.from_pretrained("voidful/Llama-3.1-TAIDE-R1-8B-Chat") model = AutoModelForCausalLM.from_pretrained("voidful/Llama-3.1-TAIDE-R1-8B-Chat", device_map="auto") - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat # Run inference directly in the terminal: llama cli -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat # Run inference directly in the terminal: llama cli -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat
Use pre-built binary
# Download pre-built binary from: # https://github.com/ggerganov/llama.cpp/releases # Start a local OpenAI-compatible server with a web UI: ./llama-server -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat # Run inference directly in the terminal: ./llama-cli -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat
Build from source code
git clone https://github.com/ggerganov/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-server llama-cli # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama-server -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat # Run inference directly in the terminal: ./build/bin/llama-cli -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat
Use Docker
docker model run hf.co/voidful/Llama-3.1-TAIDE-R1-8B-Chat
- LM Studio
- Jan
- vLLM
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with vLLM:
Install from pip and serve model
# Install vLLM from pip: pip install vllm # Start the vLLM server: vllm serve "voidful/Llama-3.1-TAIDE-R1-8B-Chat" # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "voidful/Llama-3.1-TAIDE-R1-8B-Chat", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker
docker model run hf.co/voidful/Llama-3.1-TAIDE-R1-8B-Chat
- SGLang
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with SGLang:
Install from pip and serve model
# Install SGLang from pip: pip install sglang # Start the SGLang server: python3 -m sglang.launch_server \ --model-path "voidful/Llama-3.1-TAIDE-R1-8B-Chat" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "voidful/Llama-3.1-TAIDE-R1-8B-Chat", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }'Use Docker images
docker run --gpus all \ --shm-size 32g \ -p 30000:30000 \ -v ~/.cache/huggingface:/root/.cache/huggingface \ --env "HF_TOKEN=<secret>" \ --ipc=host \ lmsysorg/sglang:latest \ python3 -m sglang.launch_server \ --model-path "voidful/Llama-3.1-TAIDE-R1-8B-Chat" \ --host 0.0.0.0 \ --port 30000 # Call the server using curl (OpenAI-compatible API): curl -X POST "http://localhost:30000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "voidful/Llama-3.1-TAIDE-R1-8B-Chat", "messages": [ { "role": "user", "content": "What is the capital of France?" } ] }' - Ollama
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with Ollama:
ollama run hf.co/voidful/Llama-3.1-TAIDE-R1-8B-Chat
- Unsloth Desktop
- Pi
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "voidful/Llama-3.1-TAIDE-R1-8B-Chat" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with Docker Model Runner:
docker model run hf.co/voidful/Llama-3.1-TAIDE-R1-8B-Chat
- Lemonade
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull voidful/Llama-3.1-TAIDE-R1-8B-Chat
Run and chat with the model
lemonade run user.Llama-3.1-TAIDE-R1-8B-Chat-{{QUANT_TAG}}List all available models
lemonade list
- Hermes Agent
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default voidful/Llama-3.1-TAIDE-R1-8B-Chat
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use voidful/Llama-3.1-TAIDE-R1-8B-Chat with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf voidful/Llama-3.1-TAIDE-R1-8B-Chat
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "voidful/Llama-3.1-TAIDE-R1-8B-Chat" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Update template
Browse files
template
CHANGED
|
@@ -10,4 +10,56 @@ SYSTEM """
|
|
| 10 |
You first think about the reasoning process in the mind and then provide the user with the answer while reasoning step by step, and putting the final answer within \\boxed{}.
|
| 11 |
The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags, respectively, i.e.,
|
| 12 |
<think> reasoning process here </think><answer> answer here </answer>.
|
| 13 |
-
"""
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 10 |
You first think about the reasoning process in the mind and then provide the user with the answer while reasoning step by step, and putting the final answer within \\boxed{}.
|
| 11 |
The reasoning process and answer are enclosed within <think> </think> and <answer> </answer> tags, respectively, i.e.,
|
| 12 |
<think> reasoning process here </think><answer> answer here </answer>.
|
| 13 |
+
"""
|
| 14 |
+
|
| 15 |
+
TEMPLATE """{{- if or .System .Tools }}<|start_header_id|>system<|end_header_id|>
|
| 16 |
+
{{- if .System }}
|
| 17 |
+
|
| 18 |
+
{{ .System }}
|
| 19 |
+
{{- end }}
|
| 20 |
+
{{- if .Tools }}
|
| 21 |
+
|
| 22 |
+
Cutting Knowledge Date: December 2023
|
| 23 |
+
|
| 24 |
+
When you receive a tool call response, use the output to format an answer to the orginal user question.
|
| 25 |
+
|
| 26 |
+
You are a helpful assistant with tool calling capabilities.
|
| 27 |
+
{{- end }}<|eot_id|>
|
| 28 |
+
{{- end }}
|
| 29 |
+
{{- range $i, $_ := .Messages }}
|
| 30 |
+
{{- $last := eq (len (slice $.Messages $i)) 1 }}
|
| 31 |
+
{{- if eq .Role "user" }}<|start_header_id|>user<|end_header_id|><think>
|
| 32 |
+
{{- if and $.Tools $last }}
|
| 33 |
+
|
| 34 |
+
Given the following functions, please respond with a JSON for a function call with its proper arguments that best answers the given prompt.
|
| 35 |
+
|
| 36 |
+
Respond in the format {"name": function name, "parameters": dictionary of argument name and its value}. Do not use variables.
|
| 37 |
+
|
| 38 |
+
{{ range $.Tools }}
|
| 39 |
+
{{- . }}
|
| 40 |
+
{{ end }}
|
| 41 |
+
Question: {{ .Content }}<|eot_id|>
|
| 42 |
+
{{- else }}
|
| 43 |
+
|
| 44 |
+
{{ .Content }}<|eot_id|>
|
| 45 |
+
{{- end }}{{ if $last }}<|start_header_id|>assistant<|end_header_id|>
|
| 46 |
+
|
| 47 |
+
{{ end }}
|
| 48 |
+
{{- else if eq .Role "assistant" }}<|start_header_id|>assistant<|end_header_id|>
|
| 49 |
+
{{- if .ToolCalls }}
|
| 50 |
+
{{ range .ToolCalls }}
|
| 51 |
+
{"name": "{{ .Function.Name }}", "parameters": {{ .Function.Arguments }}}{{ end }}
|
| 52 |
+
{{- else }}
|
| 53 |
+
|
| 54 |
+
{{ .Content }}
|
| 55 |
+
{{- end }}{{ if not $last }}<|eot_id|>{{ end }}
|
| 56 |
+
{{- else if eq .Role "tool" }}<|start_header_id|>ipython<|end_header_id|>
|
| 57 |
+
|
| 58 |
+
{{ .Content }}<|eot_id|>{{ if $last }}<|start_header_id|>assistant<|end_header_id|>
|
| 59 |
+
|
| 60 |
+
{{ end }}
|
| 61 |
+
{{- end }}
|
| 62 |
+
{{- end }}"""
|
| 63 |
+
PARAMETER stop <|start_header_id|>
|
| 64 |
+
PARAMETER stop <|end_header_id|>
|
| 65 |
+
PARAMETER stop <|eot_id|>
|