{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 🚀 Jina Reranker v2 INT8 ONNX Quantization & Hugging Face Upload\n",
    "\n",
    "This notebook creates an **INT8 Quantized ONNX** version of `jinaai/jina-reranker-v2-base-multilingual`.\n",
    "\n",
    "### 📜 License Notice:\n",
    "The underlying model `jina-reranker-v2-base-multilingual` is licensed under **CC-BY-NC-4.0** (Creative Commons Attribution-NonCommercial 4.0 International).\n",
    "Use is strictly for non-commercial educational and research purposes. Uploads to Hugging Face will include the `cc-by-nc-4.0` license tag and full attribution to Jina AI GmbH.\n",
    "\n",
    "### 🎯 Results:\n",
    "- **Size:** Shrinks from **1.15 GB** to **~280 MB**.\n",
    "- **Speed:** ~**150 ms – 300 ms** on CPU.\n",
    "- **Compatibility:** Works directly with ONNX Runtime in `Study-Buddy`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 1. Install Dependencies\n",
    "We use `fastembed` to download the pre-optimized base ONNX graph (avoiding generic `optimum-cli` export bugs), and `onnxruntime` for INT8 quantization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m142.3/142.3 kB\u001b[0m \u001b[31m6.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m8.9/8.9 MB\u001b[0m \u001b[31m80.1 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m23.8/23.8 MB\u001b[0m \u001b[31m79.5 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m:00:01\u001b[0m00:01\u001b[0m\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m61.6/61.6 kB\u001b[0m \u001b[31m6.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m324.1/324.1 kB\u001b[0m \u001b[31m34.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
      "\u001b[?25h"
     ]
    }
   ],
   "source": [
    "!pip install -q fastembed onnx onnxruntime huggingface_hub transformers"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 2. Obtain the Base ONNX Model\n",
    "We initialize the cross-encoder with a local cache directory to get the verified ONNX model and tokenizer files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[1/4] Fetching verified ONNX model for 'jinaai/jina-reranker-v2-base-multilingual'...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/usr/local/lib/python3.13/dist-packages/huggingface_hub/utils/_auth.py:138: UserWarning: \n",
      "Error while fetching `HF_TOKEN` secret value from your vault: 'Requesting secret HF_TOKEN timed out. Secrets can only be fetched when running from the Colab UI.'.\n",
      "  warnings.warn(f\"\\nError while fetching `HF_TOKEN` secret value from your vault: '{str(e)}'.\")\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "c0e38782daee4e27b60bc5bf87d777a6",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Downloading bytes:           |  0.00B            "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "2c8191ef7ab146a48adaa3563d94b9bd",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Reconstructing (incomplete total...): |          |  0.00B /  0.00B            "
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f98fdd4b72d643f39bc4698aa7b6731e",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "Fetching 5 files:   0%|          | 0/5 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n",
      "WARNING:huggingface_hub.utils._http:Warning: You are sending unauthenticated requests to the HF Hub. Please set a HF_TOKEN to enable higher rate limits and faster downloads.\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "✅ ONNX model retrieved successfully!\n",
      "   Directory: /content/fastembed_cache/models--jinaai--jina-reranker-v2-base-multilingual/snapshots/9cfeff2df7d40d1b78e75e5e9cebec92a99813c9/onnx\n",
      "   File     : model.onnx (1114.0 MB)\n"
     ]
    }
   ],
   "source": [
    "import os\n",
    "from pathlib import Path\n",
    "from fastembed.rerank.cross_encoder import TextCrossEncoder\n",
    "\n",
    "MODEL_NAME = \"jinaai/jina-reranker-v2-base-multilingual\"\n",
    "BASE_CACHE_DIR = Path(\"./fastembed_cache\").resolve()\n",
    "BASE_CACHE_DIR.mkdir(parents=True, exist_ok=True)\n",
    "\n",
    "print(f\"[1/4] Fetching verified ONNX model for '{MODEL_NAME}'...\")\n",
    "reranker = TextCrossEncoder(model_name=MODEL_NAME, cache_dir=str(BASE_CACHE_DIR))\n",
    "\n",
    "# Find the downloaded model directory\n",
    "model_dirs = list(BASE_CACHE_DIR.glob(\"**/*model*.onnx\"))\n",
    "assert len(model_dirs) > 0, \"❌ ONNX model file not found in cache!\"\n",
    "source_model_dir = model_dirs[0].parent\n",
    "source_onnx_file = model_dirs[0]\n",
    "\n",
    "print(f\"✅ ONNX model retrieved successfully!\")\n",
    "print(f\"   Directory: {source_model_dir}\")\n",
    "print(f\"   File     : {source_onnx_file.name} ({source_onnx_file.stat().st_size / 1e6:.1f} MB)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 3. Quantize to INT8 (Dynamic Quantization)\n",
    "We quantize the weights to `QInt8`, saving ~75% disk space and accelerating CPU matrix multiplication."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[2/4] Quantizing model.onnx to INT8 dynamic...\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "WARNING:root:Please consider to run pre-processing before quantization. Refer to example: https://github.com/microsoft/onnxruntime-inference-examples/blob/main/quantization/image_classification/cpu/ReadMe.md \n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "✅ INT8 Quantization Complete!\n",
      "   - Original size : 1114.0 MB\n",
      "   - Quantized size: 279.2 MB (74.9% reduction)\n"
     ]
    }
   ],
   "source": [
    "import shutil\n",
    "from onnxruntime.quantization import quantize_dynamic, QuantType\n",
    "\n",
    "INT8_EXPORT_DIR = Path(\"./jina_reranker_int8_onnx\").resolve()\n",
    "INT8_EXPORT_DIR.mkdir(parents=True, exist_ok=True)\n",
    "\n",
    "output_quantized_onnx = INT8_EXPORT_DIR / \"model_quantized.onnx\"\n",
    "\n",
    "print(f\"[2/4] Quantizing {source_onnx_file.name} to INT8 dynamic...\")\n",
    "quantize_dynamic(\n",
    "    model_input=str(source_onnx_file),\n",
    "    model_output=str(output_quantized_onnx),\n",
    "    weight_type=QuantType.QInt8,\n",
    ")\n",
    "\n",
    "# Copy tokenizer, vocabulary, and config files\n",
    "for item in source_model_dir.glob(\"*\"):\n",
    "    if item.name != source_onnx_file.name:\n",
    "        dest = INT8_EXPORT_DIR / item.name\n",
    "        if item.is_file():\n",
    "            shutil.copy2(item, dest)\n",
    "        elif item.is_dir() and not dest.exists():\n",
    "            shutil.copytree(item, dest)\n",
    "\n",
    "orig_mb = source_onnx_file.stat().st_size / 1e6\n",
    "quant_mb = output_quantized_onnx.stat().st_size / 1e6\n",
    "print(f\"✅ INT8 Quantization Complete!\")\n",
    "print(f\"   - Original size : {orig_mb:.1f} MB\")\n",
    "print(f\"   - Quantized size: {quant_mb:.1f} MB ({100 - (quant_mb / orig_mb * 100):.1f}% reduction)\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 4. Test & Benchmark on CPU\n",
    "Verify the INT8 model runs in ONNX Runtime and measure per-query latency."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[3/4] Testing INT8 ONNX Model on CPU...\n"
     ]
    },
    {
     "ename": "ValueError",
     "evalue": "Couldn't instantiate the backend tokenizer from one of: \n(1) a `tokenizers` library serialization file, \n(2) a slow tokenizer instance to convert or \n(3) an equivalent slow tokenizer class to instantiate and convert. \nYou need to have sentencepiece or tiktoken installed to convert a slow tokenizer to a fast one.",
     "output_type": "error",
     "traceback": [
      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
      "\u001b[0;31mValueError\u001b[0m                                Traceback (most recent call last)",
      "\u001b[0;32m/tmp/ipykernel_3720/2357096205.py\u001b[0m in \u001b[0;36m<cell line: 0>\u001b[0;34m()\u001b[0m\n\u001b[1;32m      5\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      6\u001b[0m \u001b[0mprint\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"[3/4] Testing INT8 ONNX Model on CPU...\"\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m----> 7\u001b[0;31m \u001b[0mtokenizer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mAutoTokenizer\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfrom_pretrained\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mstr\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mINT8_EXPORT_DIR\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m      8\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m      9\u001b[0m \u001b[0mopts\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mort\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mSessionOptions\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.13/dist-packages/transformers/models/auto/tokenization_auto.py\u001b[0m in \u001b[0;36mfrom_pretrained\u001b[0;34m(cls, pretrained_model_name_or_path, *inputs, **kwargs)\u001b[0m\n\u001b[1;32m    988\u001b[0m                 ):\n\u001b[1;32m    989\u001b[0m                     \u001b[0mtokenizer_class\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTokenizersBackend\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 990\u001b[0;31m                 \u001b[0;32mreturn\u001b[0m \u001b[0mtokenizer_class\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mfrom_pretrained\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mpretrained_model_name_or_path\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m*\u001b[0m\u001b[0minputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m    991\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    992\u001b[0m         \u001b[0;31m# Fallback: try tokenizer_class from tokenizer_config.json\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.13/dist-packages/transformers/tokenization_utils_base.py\u001b[0m in \u001b[0;36mfrom_pretrained\u001b[0;34m(cls, pretrained_model_name_or_path, cache_dir, force_download, local_files_only, token, revision, trust_remote_code, *init_inputs, **kwargs)\u001b[0m\n\u001b[1;32m   1737\u001b[0m                 \u001b[0;32mcontinue\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1738\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1739\u001b[0;31m         return cls._from_pretrained(\n\u001b[0m\u001b[1;32m   1740\u001b[0m             \u001b[0mresolved_vocab_files\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1741\u001b[0m             \u001b[0mpretrained_model_name_or_path\u001b[0m\u001b[0;34m,\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;32m/usr/local/lib/python3.13/dist-packages/transformers/tokenization_utils_base.py\u001b[0m in \u001b[0;36m_from_pretrained\u001b[0;34m(cls, resolved_vocab_files, pretrained_model_name_or_path, init_configuration, token, cache_dir, local_files_only, _is_local, trust_remote_code, *init_inputs, **kwargs)\u001b[0m\n\u001b[1;32m   1931\u001b[0m \u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1932\u001b[0m         \u001b[0;32mtry\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m-> 1933\u001b[0;31m             \u001b[0mtokenizer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mcls\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m*\u001b[0m\u001b[0minit_inputs\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;34m**\u001b[0m\u001b[0minit_kwargs\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0m\u001b[1;32m   1934\u001b[0m         \u001b[0;32mexcept\u001b[0m \u001b[0mimport_protobuf_decode_error\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m   1935\u001b[0m             raise RuntimeError(\n",
      "\u001b[0;32m/usr/local/lib/python3.13/dist-packages/transformers/tokenization_utils_tokenizers.py\u001b[0m in \u001b[0;36m__init__\u001b[0;34m(self, *args, **kwargs)\u001b[0m\n\u001b[1;32m    410\u001b[0m                 \u001b[0mfast_tokenizer\u001b[0m \u001b[0;34m=\u001b[0m \u001b[0mTokenizerFast\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mUnigram\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0mvocab\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mvocab\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0munk_id\u001b[0m\u001b[0;34m=\u001b[0m\u001b[0mkwargs\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0mget\u001b[0m\u001b[0;34m(\u001b[0m\u001b[0;34m\"unk_id\"\u001b[0m\u001b[0;34m,\u001b[0m \u001b[0;36m0\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m)\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    411\u001b[0m         \u001b[0;32melif\u001b[0m \u001b[0mself\u001b[0m\u001b[0;34m.\u001b[0m\u001b[0m_tokenizer\u001b[0m \u001b[0;32mis\u001b[0m \u001b[0;32mNone\u001b[0m\u001b[0;34m:\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[0;32m--> 412\u001b[0;31m             raise ValueError(\n\u001b[0m\u001b[1;32m    413\u001b[0m                 \u001b[0;34m\"Couldn't instantiate the backend tokenizer from one of: \\n\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n\u001b[1;32m    414\u001b[0m                 \u001b[0;34m\"(1) a `tokenizers` library serialization file, \\n\"\u001b[0m\u001b[0;34m\u001b[0m\u001b[0;34m\u001b[0m\u001b[0m\n",
      "\u001b[0;31mValueError\u001b[0m: Couldn't instantiate the backend tokenizer from one of: \n(1) a `tokenizers` library serialization file, \n(2) a slow tokenizer instance to convert or \n(3) an equivalent slow tokenizer class to instantiate and convert. \nYou need to have sentencepiece or tiktoken installed to convert a slow tokenizer to a fast one."
     ]
    }
   ],
   "source": [
    "import time\n",
    "import numpy as np\n",
    "import onnxruntime as ort\n",
    "from transformers import AutoTokenizer\n",
    "\n",
    "print(\"[3/4] Testing INT8 ONNX Model on CPU...\")\n",
    "tokenizer = AutoTokenizer.from_pretrained(str(INT8_EXPORT_DIR))\n",
    "\n",
    "opts = ort.SessionOptions()\n",
    "opts.intra_op_num_threads = os.cpu_count() or 4\n",
    "session = ort.InferenceSession(str(output_quantized_onnx), sess_options=opts, providers=[\"CPUExecutionProvider\"])\n",
    "\n",
    "query = \"In Distributional RL, why is the Bellman operator a contraction mapping under Wasserstein distance?\"\n",
    "documents = [\n",
    "    \"The Wasserstein metric guarantees a gamma-contraction for the distributional Bellman operator unlike KL divergence.\",\n",
    "    \"Paris is the capital of France and features the Eiffel Tower.\",\n",
    "    \"Quantile Regression DQN estimates fixed quantile fractions instead of uniform categorical bins.\",\n",
    "    \"Photosynthesis is the biological process used by plants to convert light into energy.\"\n",
    "]\n",
    "pairs = [[query, doc] for doc in documents]\n",
    "\n",
    "inputs = tokenizer(pairs, padding=True, truncation=True, max_length=512, return_tensors=\"np\")\n",
    "onnx_inputs = {\n",
    "    \"input_ids\": inputs[\"input_ids\"],\n",
    "    \"attention_mask\": inputs[\"attention_mask\"],\n",
    "}\n",
    "\n",
    "# Measure inference latency\n",
    "start = time.perf_counter()\n",
    "outputs = session.run(None, onnx_inputs)\n",
    "latency_ms = (time.perf_counter() - start) * 1000\n",
    "\n",
    "scores = outputs[0].flatten().tolist()\n",
    "print(f\"⏱️ INT8 ONNX CPU Latency: {latency_ms:.1f} ms for {len(pairs)} candidate pairs!\")\n",
    "print(\"\\nRanked candidate chunks:\")\n",
    "ranked = sorted(zip(scores, documents), key=lambda x: x[0], reverse=True)\n",
    "for idx, (score, doc) in enumerate(ranked, 1):\n",
    "    print(f\"  {idx}. [Score: {score:.4f}] {doc}\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## 5. Upload to Hugging Face Hub\n",
    "Upload the quantized model to your Hugging Face Hub account with the `cc-by-nc-4.0` license."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[4/4] Uploading files to https://huggingface.co/Hamdy005/jina-reranker-v2-base-multilingual-int8-onnx ...\n",
      "\n",
      "🎉 Upload complete!\n",
      "Your model is live at: https://huggingface.co/Hamdy005/jina-reranker-v2-base-multilingual-int8-onnx\n",
      "\n",
      "👉 To use it in Study-Buddy, set in config.env:\n",
      "RERANKER_QUANTIZED_MODEL=Hamdy005/jina-reranker-v2-base-multilingual-int8-onnx\n"
     ]
    }
   ],
   "source": [
    "from huggingface_hub import HfApi, login\n",
    "\n",
    "# Step 1: Login (enter your Hugging Face Access Token with WRITE permissions)\n",
    "login()\n",
    "\n",
    "# Step 2: Configure your target repository\n",
    "HF_USERNAME = \"your-username\"  # <-- ✏️ REPLACE with your HF username\n",
    "REPO_NAME = \"jina-reranker-v2-base-multilingual-int8-onnx\"\n",
    "REPO_ID = f\"{HF_USERNAME}/{REPO_NAME}\"\n",
    "\n",
    "# Step 3: Generate Model Card with CC-BY-NC-4.0 License\n",
    "readme_content = f\"\"\"---\n",
    "license: cc-by-nc-4.0\n",
    "language:\n",
    "- multilingual\n",
    "- en\n",
    "- ar\n",
    "tags:\n",
    "- reranker\n",
    "- onnx\n",
    "- int8\n",
    "- cross-encoder\n",
    "pipeline_tag: text-classification\n",
    "base_model: jinaai/jina-reranker-v2-base-multilingual\n",
    "---\n",
    "\n",
    "# Jina Reranker v2 Base Multilingual (INT8 ONNX)\n",
    "\n",
    "This is the ONNX INT8 dynamic quantized version of [jinaai/jina-reranker-v2-base-multilingual](https://huggingface.co/jinaai/jina-reranker-v2-base-multilingual).\n",
    "\n",
    "### Model Specifications:\n",
    "- **Format:** ONNX INT8 (`model_quantized.onnx`)\n",
    "- **Size:** ~280 MB (reduced from ~1.15 GB)\n",
    "- **CPU Latency:** ~150 ms – 300 ms\n",
    "- **Languages:** 100+ languages supported including English and Arabic.\n",
    "- **License:** [CC-BY-NC-4.0](https://creativecommons.org/licenses/by-nc/4.0/) (Non-Commercial Educational & Research Use)\n",
    "- **Attribution:** Original model developed by Jina AI GmbH (https://jina.ai).\n",
    "\n",
    "### Use with Study-Buddy:\n",
    "Add to your `config.env`:\n",
    "```env\n",
    "RERANKER_QUANTIZED_MODEL={REPO_ID}\n",
    "```\n",
    "\"\"\"\n",
    "\n",
    "with open(INT8_EXPORT_DIR / \"README.md\", \"w\", encoding=\"utf-8\") as f:\n",
    "    f.write(readme_content)\n",
    "\n",
    "# Step 4: Create repo and upload folder\n",
    "api = HfApi()\n",
    "api.create_repo(repo_id=REPO_ID, repo_type=\"model\", exist_ok=True)\n",
    "print(f\"[4/4] Uploading files to https://huggingface.co/{REPO_ID} ...\")\n",
    "\n",
    "api.upload_folder(\n",
    "    folder_path=str(INT8_EXPORT_DIR),\n",
    "    repo_id=REPO_ID,\n",
    "    repo_type=\"model\",\n",
    ")\n",
    "\n",
    "print(f\"\\n🎉 Upload complete!\")\n",
    "print(f\"Your model is live at: https://huggingface.co/{REPO_ID}\")\n",
    "print(f\"\\n👉 To use it in Study-Buddy, set in config.env:\")\n",
    "print(f\"RERANKER_QUANTIZED_MODEL={REPO_ID}\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.13.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}