CreeJee's picture
Add ONNX exports (fp32, fp16) of nlpai-lab/KURE-Reranker-nano
90b87fb verified
|
Raw History Blame Contribute Delete
2.7 kB
metadata
license: apache-2.0
base_model: nlpai-lab/KURE-Reranker-nano
library_name: transformers.js
pipeline_tag: text-ranking
language:
  - ko
  - en
tags:
  - onnx
  - cross-encoder
  - reranker
  - modernbert

KURE-Reranker-nano (ONNX)

ONNX weights of nlpai-lab/KURE-Reranker-nano, a Korean and English cross-encoder reranker (ModernBERT, one relevance logit per query and passage pair), for ONNX Runtime and Transformers.js.

The original repository publishes only model.safetensors. This repository adds ONNX exports of the same weights. The model, its training and its evaluation are the work of NLP & AI Lab, Korea University; see the original model card for them.

Files

File Precision Size
onnx/model.onnx fp32 598 MB
onnx/model_fp16.onnx fp16 weights and activations, fp32 inputs and outputs 299 MB

config.json, tokenizer.json, tokenizer_config.json and special_tokens_map.json come from the export. The tokenizer is the original's. config.json differs only in the recorded Transformers version and dtype field.

How the files were made

From revision 8514b9e113f3b27c441ed9c2b50f81d80388b2fd of the original:

optimum-cli export onnx --model nlpai-lab/KURE-Reranker-nano --task text-classification out

The fp16 file converts the fp32 export with ONNX Runtime's converter:

import onnx
from onnxruntime.transformers.float16 import convert_float_to_float16

model = onnx.load("onnx/model.onnx")
onnx.save(convert_float_to_float16(model, keep_io_types=True), "onnx/model_fp16.onnx")

No weights were trained or changed beyond that precision conversion.

Usage with Transformers.js

import { AutoModelForSequenceClassification, AutoTokenizer } from "@huggingface/transformers";

const id = "CreeJee/KURE-Reranker-nano-ONNX";
const tokenizer = await AutoTokenizer.from_pretrained(id);
const model = await AutoModelForSequenceClassification.from_pretrained(id, {
  dtype: "fp16",
  device: "webgpu",
});

const query = "๋Œ€ํ•œ๋ฏผ๊ตญ์˜ ์ˆ˜๋„๋Š” ์–ด๋””์ธ๊ฐ€?";
const passages = ["์„œ์šธ์€ ๋Œ€ํ•œ๋ฏผ๊ตญ์˜ ์ˆ˜๋„์ด๋‹ค.", "๋ถ€์‚ฐ์€ ํ•ญ๊ตฌ ๋„์‹œ์ด๋‹ค."];
const inputs = tokenizer(
  passages.map(() => query),
  { text_pair: passages, padding: true, truncation: true, max_length: 256 },
);
const { logits } = await model(inputs);
// One logit per pair; higher is more relevant.
console.log(Array.from(logits.data));

License

Apache-2.0, as the original. The ONNX files are a modified form of the original weights: they were converted to ONNX, and model_fp16.onnx to half precision.