ONNX
quantized
retrieval
reranker
embeddings
DevNexon commited on
Commit
0b3a005
·
verified ·
1 Parent(s): 0d058cf

Add files using upload-large-folder tool

Browse files
Files changed (2) hide show
  1. README.md +3 -8
  2. inference.py +2 -2
README.md CHANGED
@@ -1,6 +1,5 @@
1
  ---
2
  license: cc-by-4.0
3
- library_name: onnx
4
  tags:
5
  - onnx
6
  - quantized
@@ -9,11 +8,6 @@ tags:
9
  - embeddings
10
  datasets:
11
  - nynefold/FreshRet-Small
12
- base_model:
13
- - perplexity-ai/pplx-embed-v1-0.6b
14
- - codefuse-ai/F2LLM-v2-1.7B
15
- - Qwen/Qwen3-Reranker-0.6B
16
- - zeroentropy/zerank-1-small
17
  ---
18
  # retrieval-quant: quantized ONNX retrievers
19
 
@@ -28,7 +22,8 @@ Repos: [`nynefold/pplx-embed-v1-0.6b-ONNX`](https://huggingface.co/nynefold/pplx
28
  [`nynefold/zerank-1-small-ONNX`](https://huggingface.co/nynefold/zerank-1-small-ONNX), and the evaluation set
29
  [`nynefold/FreshRet-Small`](https://huggingface.co/datasets/nynefold/FreshRet-Small). `pipeline_example.py` in this repo
30
  retrieves with pplx-embed and reranks with zerank, using `inference.py` (ONNX Runtime, `tokenizers`
31
- and NumPy only).
 
32
 
33
  ## Results in short
34
 
@@ -223,7 +218,7 @@ Base model [`Qwen/Qwen3-Reranker-0.6B`](https://huggingface.co/Qwen/Qwen3-Rerank
223
 
224
  ### zerank-1-small
225
 
226
- Base model [`zeroentropy/zerank-1-small`](https://huggingface.co/zeroentropy/zerank-1-small), FP32 6885 MB. Files: [`nynefold/zerank-1-small-ONNX`](https://huggingface.co/nynefold/zerank-1-small-ONNX).
227
 
228
  | variant | MB | Spearman mean (gate 0.98) | FreshRet Small nDCG@10 vs FP32 (gate -1.0 pt) | verdict |
229
  |---|---:|---:|---|---|
 
1
  ---
2
  license: cc-by-4.0
 
3
  tags:
4
  - onnx
5
  - quantized
 
8
  - embeddings
9
  datasets:
10
  - nynefold/FreshRet-Small
 
 
 
 
 
11
  ---
12
  # retrieval-quant: quantized ONNX retrievers
13
 
 
22
  [`nynefold/zerank-1-small-ONNX`](https://huggingface.co/nynefold/zerank-1-small-ONNX), and the evaluation set
23
  [`nynefold/FreshRet-Small`](https://huggingface.co/datasets/nynefold/FreshRet-Small). `pipeline_example.py` in this repo
24
  retrieves with pplx-embed and reranks with zerank, using `inference.py` (ONNX Runtime, `tokenizers`
25
+ and NumPy only). All six repos are in the collection
26
+ [retrieval-quant: quantized ONNX retrievers](https://huggingface.co/collections/nynefold/retrieval-quant-quantized-onnx-retrievers-6adb9c3a7f6012982d819511).
27
 
28
  ## Results in short
29
 
 
218
 
219
  ### zerank-1-small
220
 
221
+ Base model [`zeroentropy/zerank-1-small-reranker`](https://huggingface.co/zeroentropy/zerank-1-small-reranker), FP32 6885 MB. Files: [`nynefold/zerank-1-small-ONNX`](https://huggingface.co/nynefold/zerank-1-small-ONNX).
222
 
223
  | variant | MB | Spearman mean (gate 0.98) | FreshRet Small nDCG@10 vs FP32 (gate -1.0 pt) | verdict |
224
  |---|---:|---:|---|---|
inference.py CHANGED
@@ -13,7 +13,7 @@ Needs only `onnxruntime`, `tokenizers` and `numpy` (plus `huggingface_hub` to do
13
  print(rr.rerank("how do I reset the router?", ["Hold the reset button ...", "The router has ..."]))
14
 
15
  `load` takes a repo id or a local folder. All model-specific details (prompts, truncation, output
16
- transform) come from `inference_config.json`, which is part of the contract: the graphs expect
17
  exactly these inputs.
18
  """
19
  import json
@@ -24,7 +24,7 @@ import numpy as np
24
  import onnxruntime as ort
25
  from tokenizers import Tokenizer
26
 
27
- CONFIG = "inference_config.json"
28
 
29
 
30
  def _local(repo_or_dir, variant):
 
13
  print(rr.rerank("how do I reset the router?", ["Hold the reset button ...", "The router has ..."]))
14
 
15
  `load` takes a repo id or a local folder. All model-specific details (prompts, truncation, output
16
+ transform) come from `config.json`, which is part of the contract: the graphs expect
17
  exactly these inputs.
18
  """
19
  import json
 
24
  import onnxruntime as ort
25
  from tokenizers import Tokenizer
26
 
27
+ CONFIG = "config.json" # also the file the Hub counts downloads by
28
 
29
 
30
  def _local(repo_or_dir, variant):