Instructions to use bartscheffer/deepkit-code-reranker-gguf with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- llama.cpp
How to use bartscheffer/deepkit-code-reranker-gguf with llama.cpp:
Install (macOS, Linux)
curl -LsSf https://llama.app/install.sh | sh # Start a local OpenAI-compatible server with a web UI: llama serve -hf bartscheffer/deepkit-code-reranker-gguf:Q8_0 # Run inference directly in the terminal: llama cli -hf bartscheffer/deepkit-code-reranker-gguf:Q8_0
Install from WinGet (Windows)
winget install llama.cpp # Start a local OpenAI-compatible server with a web UI: llama serve -hf bartscheffer/deepkit-code-reranker-gguf:Q8_0 # Run inference directly in the terminal: llama cli -hf bartscheffer/deepkit-code-reranker-gguf:Q8_0
Build from source code
git clone https://github.com/ggml-org/llama.cpp.git cd llama.cpp cmake -B build cmake --build build -j --target llama-app # Start a local OpenAI-compatible server with a web UI: ./build/bin/llama serve -hf bartscheffer/deepkit-code-reranker-gguf:Q8_0 # Run inference directly in the terminal: ./build/bin/llama cli -hf bartscheffer/deepkit-code-reranker-gguf:Q8_0
Use Docker
docker model run hf.co/bartscheffer/deepkit-code-reranker-gguf:Q8_0
- Llama new
- LM Studio
- Jan
- Ollama
How to use bartscheffer/deepkit-code-reranker-gguf with Ollama:
ollama run hf.co/bartscheffer/deepkit-code-reranker-gguf:Q8_0
- Unsloth Desktop
- Pi
How to use bartscheffer/deepkit-code-reranker-gguf with Pi:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf bartscheffer/deepkit-code-reranker-gguf:Q8_0
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "llama-cpp": { "baseUrl": "http://localhost:9931/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "bartscheffer/deepkit-code-reranker-gguf:Q8_0" } ] } } }Run Pi
# Start Pi in your project directory: pi
- Docker Model Runner
How to use bartscheffer/deepkit-code-reranker-gguf with Docker Model Runner:
docker model run hf.co/bartscheffer/deepkit-code-reranker-gguf:Q8_0
- Lemonade
How to use bartscheffer/deepkit-code-reranker-gguf with Lemonade:
Pull the model
# Download Lemonade from https://lemonade-server.ai/ lemonade pull bartscheffer/deepkit-code-reranker-gguf:Q8_0
Run and chat with the model
lemonade run user.deepkit-code-reranker-gguf-Q8_0
List all available models
lemonade list
- Hermes Agent
How to use bartscheffer/deepkit-code-reranker-gguf with Hermes Agent:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf bartscheffer/deepkit-code-reranker-gguf:Q8_0
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:9931/v1 hermes config set model.default bartscheffer/deepkit-code-reranker-gguf:Q8_0
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use bartscheffer/deepkit-code-reranker-gguf with OpenClaw:
Start the llama.cpp server
# Install llama.cpp: brew install llama.cpp # Start a local OpenAI-compatible server: llama serve -hf bartscheffer/deepkit-code-reranker-gguf:Q8_0
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:9931/v1 \ --custom-model-id "bartscheffer/deepkit-code-reranker-gguf:Q8_0" \ --custom-provider-id llama-cpp \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Download artifact-manifest.json from bartscheffer/deepkit-code-reranker-gguf: direct link, hf CLI and curl.
- Browser
- Download file 2.84 kB
-
https://huggingface.co/bartscheffer/deepkit-code-reranker-gguf/resolve/main/artifact-manifest.json
- Command line
-
hf download hf://bartscheffer/deepkit-code-reranker-gguf/artifact-manifest.json
-
curl -L -o artifact-manifest.json https://huggingface.co/bartscheffer/deepkit-code-reranker-gguf/resolve/main/artifact-manifest.json
2.84 kB
| { | |
| "model_source": "Qwen/Qwen3-Reranker-0.6B", | |
| "model_revision": "e61197ed45024b0ed8a2d74b80b4d909f1255473", | |
| "llama_cpp_revision": "e79e4bf660e19f2ad851e06c6913f7a8c5852621", | |
| "rust_binding_revision": "63e549708237b16b39018288582655539e0a9d5b", | |
| "license": "Apache-2.0 (upstream model); converter MIT", | |
| "python": "3.12.14", | |
| "artifacts": [ | |
| { | |
| "file": "Qwen3-Reranker-0.6B-Q8_0.gguf", | |
| "size": 639153344, | |
| "sha256": "4fb2a26728ecea8ea30dc396ac553af14970329849f0dd4b23ae06340f37715a", | |
| "metadata": { | |
| "general.architecture": "qwen3", | |
| "general.file_type": 7, | |
| "qwen3.pooling_type": 4, | |
| "qwen3.classifier.output_labels": [ | |
| "yes", | |
| "no" | |
| ], | |
| "tokenizer.chat_template.rerank": "<|im_start|>system\nJudge whether the Document meets the requirements based on the Query and the Instruct provided. Note that the answer can only be \"yes\" or \"no\".<|im_end|>\n<|im_start|>user\n<Instruct>: Given a web search query, retrieve relevant passages that answer the query\n<Query>: {query}\n<Document>: {document}<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n" | |
| }, | |
| "tensor_count": 311, | |
| "classifier_type": "Q8_0", | |
| "classifier_token_ids": [ | |
| 9693, | |
| 2152 | |
| ], | |
| "classifier_row_cosines": [ | |
| 0.9999851584434509, | |
| 0.9999845027923584 | |
| ], | |
| "classifier_max_abs_error": 0.0003905296325683594, | |
| "cpu_smoke_scores": [ | |
| 0.9998122, | |
| 1.7702505e-05, | |
| 0.0056796884 | |
| ] | |
| }, | |
| { | |
| "file": "Qwen3-Reranker-0.6B-f16.gguf", | |
| "size": 1197634304, | |
| "sha256": "a3182982cf5ff9d3c40a75b3c4120e297769be0da333cf49b4ad86755c9535e5", | |
| "metadata": { | |
| "general.architecture": "qwen3", | |
| "general.file_type": 1, | |
| "qwen3.pooling_type": 4, | |
| "qwen3.classifier.output_labels": [ | |
| "yes", | |
| "no" | |
| ], | |
| "tokenizer.chat_template.rerank": "<|im_start|>system\nJudge whether the Document meets the requirements based on the Query and the Instruct provided. Note that the answer can only be \"yes\" or \"no\".<|im_end|>\n<|im_start|>user\n<Instruct>: Given a web search query, retrieve relevant passages that answer the query\n<Query>: {query}\n<Document>: {document}<|im_end|>\n<|im_start|>assistant\n<think>\n\n</think>\n\n" | |
| }, | |
| "tensor_count": 311, | |
| "classifier_type": "F16", | |
| "classifier_token_ids": [ | |
| 9693, | |
| 2152 | |
| ], | |
| "classifier_row_cosines": [ | |
| 1.0, | |
| 1.0 | |
| ], | |
| "classifier_max_abs_error": 0.0, | |
| "cpu_smoke_scores": [ | |
| 0.9997133, | |
| 1.6217742e-05, | |
| 0.005983466 | |
| ] | |
| } | |
| ], | |
| "q8_repeat_byte_identical": true, | |
| "requirements_lock_sha256": "928aafba3a9677549fe2f1e159e1d72d5241d4202fbdc82e76a41668a5d44d6f" | |
| } | |