bartscheffer's picture
Add verified Qwen3 reranker Q8_0 and f16 conversions
90736d4 verified
|
Raw History Blame Contribute Delete
1.97 kB
---
license: apache-2.0
base_model: Qwen/Qwen3-Reranker-0.6B
library_name: gguf
pipeline_tag: text-ranking
tags:
- gguf
- reranker
- deepkit-code
---
# Deepkit Code — Qwen3 Reranker 0.6B GGUF
Q8_0 and f16 conversions of the official Qwen3-Reranker-0.6B model, prepared for
[Deepkit Code](https://github.com/bartscheffer/deepkit-code). This repository hosts
model artifacts; it does not contain the Deepkit Code executable.
## Provenance
- Official model revision: `e61197ed45024b0ed8a2d74b80b4d909f1255473`.
- Unmodified official llama.cpp converter: `e79e4bf660e19f2ad851e06c6913f7a8c5852621`.
- Formats: Q8_0 and f16. Exact sizes and SHA-256 are in `artifact-manifest.json`.
- Model license: Apache-2.0. The original model is developed by Qwen; this is a
project-maintained conversion, not an official Qwen GGUF release.
[Conversion scripts and dependency lock](https://github.com/bartscheffer/deepkit-code/tree/a5762f2/scripts/models)
record the preparation workflow. All upstream files were verified against pinned
Git-blob/LFS hashes before offline conversion. Repeating Q8_0 conversion produced
byte-identical output in the recorded environment.
## Runtime contract
These are rerankers, not embedding models. Both contain `cls.output.weight`, Rank
pooling, classifier labels `[yes, no]`, and `tokenizer.chat_template.rerank`.
Use the stored rerank template for the query/document pair. The recorded native
Qwen3 Rank implementation already applies softmax: the first pooled value is
P(yes). Do not apply a second softmax or L2-normalize the two probabilities.
Both artifacts pass actual CPU three-pair checks: a relevant answer scores above
0.999 and an irrelevant answer below 0.00002. Classifier rows were also compared
against the original yes/no token weights. Details are in the manifest. These are
artifact smoke tests, not a retrieval benchmark or a throughput guarantee.
Deepkit Code's production search integration is still in development.