CreeJee's picture
Add ONNX exports (fp32, fp16) of nlpai-lab/KURE-Reranker-nano
90b87fb verified
|
Raw History Blame Contribute Delete
2.7 kB
---
license: apache-2.0
base_model: nlpai-lab/KURE-Reranker-nano
library_name: transformers.js
pipeline_tag: text-ranking
language:
- ko
- en
tags:
- onnx
- cross-encoder
- reranker
- modernbert
---
# KURE-Reranker-nano (ONNX)
ONNX weights of [nlpai-lab/KURE-Reranker-nano](https://huggingface.co/nlpai-lab/KURE-Reranker-nano), a Korean and English cross-encoder reranker (ModernBERT, one relevance logit per query and passage pair), for ONNX Runtime and [Transformers.js](https://huggingface.co/docs/transformers.js).
The original repository publishes only `model.safetensors`. This repository adds ONNX exports of the same weights. The model, its training and its evaluation are the work of NLP & AI Lab, Korea University; see the original model card for them.
## Files
| File | Precision | Size |
|---|---|---|
| `onnx/model.onnx` | fp32 | 598 MB |
| `onnx/model_fp16.onnx` | fp16 weights and activations, fp32 inputs and outputs | 299 MB |
`config.json`, `tokenizer.json`, `tokenizer_config.json` and `special_tokens_map.json` come from the export. The tokenizer is the original's. `config.json` differs only in the recorded Transformers version and dtype field.
## How the files were made
From revision `8514b9e113f3b27c441ed9c2b50f81d80388b2fd` of the original:
```sh
optimum-cli export onnx --model nlpai-lab/KURE-Reranker-nano --task text-classification out
```
The fp16 file converts the fp32 export with ONNX Runtime's converter:
```python
import onnx
from onnxruntime.transformers.float16 import convert_float_to_float16
model = onnx.load("onnx/model.onnx")
onnx.save(convert_float_to_float16(model, keep_io_types=True), "onnx/model_fp16.onnx")
```
No weights were trained or changed beyond that precision conversion.
## Usage with Transformers.js
```js
import { AutoModelForSequenceClassification, AutoTokenizer } from "@huggingface/transformers";
const id = "CreeJee/KURE-Reranker-nano-ONNX";
const tokenizer = await AutoTokenizer.from_pretrained(id);
const model = await AutoModelForSequenceClassification.from_pretrained(id, {
dtype: "fp16",
device: "webgpu",
});
const query = "๋Œ€ํ•œ๋ฏผ๊ตญ์˜ ์ˆ˜๋„๋Š” ์–ด๋””์ธ๊ฐ€?";
const passages = ["์„œ์šธ์€ ๋Œ€ํ•œ๋ฏผ๊ตญ์˜ ์ˆ˜๋„์ด๋‹ค.", "๋ถ€์‚ฐ์€ ํ•ญ๊ตฌ ๋„์‹œ์ด๋‹ค."];
const inputs = tokenizer(
passages.map(() => query),
{ text_pair: passages, padding: true, truncation: true, max_length: 256 },
);
const { logits } = await model(inputs);
// One logit per pair; higher is more relevant.
console.log(Array.from(logits.data));
```
## License
Apache-2.0, as the original. The ONNX files are a modified form of the original weights: they were converted to ONNX, and `model_fp16.onnx` to half precision.