Instructions to use VladHong/llama-cpp-K2-FAST2 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use VladHong/llama-cpp-K2-FAST2 with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf VladHong/llama-cpp-K2-FAST2 # Run inference directly in the terminal: llama cli -hf VladHong/llama-cpp-K2-FAST2
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf VladHong/llama-cpp-K2-FAST2 # Run inference directly in the terminal: llama cli -hf VladHong/llama-cpp-K2-FAST2
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf VladHong/llama-cpp-K2-FAST2 # Run inference directly in the terminal: ./build/bin/llama cli -hf VladHong/llama-cpp-K2-FAST2
Use Docker
docker model run hf.co/VladHong/llama-cpp-K2-FAST2
- Llama new
- LM Studio
- Jan
- Ollama
How to use VladHong/llama-cpp-K2-FAST2 with Ollama:
ollama run hf.co/VladHong/llama-cpp-K2-FAST2
- Unsloth Desktop
- Docker Model Runner
How to use VladHong/llama-cpp-K2-FAST2 with Docker Model Runner:
docker model run hf.co/VladHong/llama-cpp-K2-FAST2
- Lemonade
How to use VladHong/llama-cpp-K2-FAST2 with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull VladHong/llama-cpp-K2-FAST2
Run and chat with the model
lemonade run user.llama-cpp-K2-FAST2-{{QUANT_TAG}}List all available models
lemonade list
- Atomic Chat
|
Download examples/deprecation-warning/README.md from VladHong/llama-cpp-K2-FAST2: direct link, hf CLI and curl.
- Browser
- Download file 1.68 kB
-
https://huggingface.co/VladHong/llama-cpp-K2-FAST2/resolve/main/examples/deprecation-warning/README.md
- Command line
-
hf download hf://VladHong/llama-cpp-K2-FAST2/examples/deprecation-warning/README.md
-
curl -L -o README.md https://huggingface.co/VladHong/llama-cpp-K2-FAST2/resolve/main/examples/deprecation-warning/README.md
1.68 kB
| # Migration notice for binary filenames | |
| > [!IMPORTANT] | |
| [2024 Jun 12] Binaries have been renamed w/ a `llama-` prefix. `main` is now `llama-cli`, `server` is `llama-server`, etc (https://github.com/ggml-org/llama.cpp/pull/7809) | |
| This migration was important, but it is a breaking change that may not always be immediately obvious to users. | |
| Please update all scripts and workflows to use the new binary names. | |
| | Old Filename | New Filename | | |
| | ---- | ---- | | |
| | main | llama-cli | | |
| | server | llama-server | | |
| | llama-bench | llama-bench | | |
| | embedding | llama-embedding | | |
| | quantize | llama-quantize | | |
| | tokenize | llama-tokenize | | |
| | export-lora | llama-export-lora | | |
| | libllava.a | libllava.a | | |
| | baby-llama | llama-baby-llama | | |
| | batched | llama-batched | | |
| | batched-bench | llama-batched-bench | | |
| | benchmark-matmult | llama-benchmark-matmult | | |
| | convert-llama2c-to-ggml | llama-convert-llama2c-to-ggml | | |
| | eval-callback | llama-eval-callback | | |
| | gbnf-validator | llama-gbnf-validator | | |
| | gguf | llama-gguf | | |
| | gguf-split | llama-gguf-split | | |
| | gritlm | llama-gritlm | | |
| | imatrix | llama-imatrix | | |
| | infill | llama-infill | | |
| | llava-cli | llama-llava-cli | | |
| | lookahead | llama-lookahead | | |
| | lookup | llama-lookup | | |
| | lookup-create | llama-lookup-create | | |
| | lookup-merge | llama-lookup-merge | | |
| | lookup-stats | llama-lookup-stats | | |
| | parallel | llama-parallel | | |
| | passkey | llama-passkey | | |
| | perplexity | llama-perplexity | | |
| | q8dot | llama-q8dot | | |
| | quantize-stats | llama-quantize-stats | | |
| | retrieval | llama-retrieval | | |
| | save-load-state | llama-save-load-state | | |
| | simple | llama-simple | | |
| | speculative | llama-speculative | | |
| | vdot | llama-vdot | | |
| | tests/test-c.o | tests/test-c.o | | |