Publish OpenVINO INT4 model

OpenVINO 2026.4 INT4 release of the Llama-3.2-1B Computer Engineering model. Includes CPU, Intel iGPU and Intel AI Boost NPU benchmarks collected on IULinux.

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+tokenizer.json filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,342 @@
+---
+library_name: openvino
+pipeline_tag: text-generation
+base_model:
+- Irfanuruchi/Llama-3.2-1B-Computer-Engineering-LLM
+base_model_relation: quantized
+language:
+- en
+license: llama3.2
+tags:
+- openvino
+- openvino-genai
+- int4
+- computer-engineering
+- llama-3.2
+- intel
+- npu
+- iulinux
+---
+
+# Llama-3.2-1B Computer Engineering - OpenVINO INT4
+
+OpenVINO INT4 conversion of `Irfanuruchi/Llama-3.2-1B-Computer-Engineering-LLM`, a 1B-parameter model specialized for Computer Engineering.
+
+The model was converted and tested on **IULinux** with OpenVINO 2026.4 across three Intel execution backends:
+
+- Intel Core Ultra 9 275HX CPU
+- Intel Graphics integrated GPU
+- Intel AI Boost NPU
+
+The resulting OpenVINO model directory is approximately **758 MB**.
+
+## Prompt format
+
+This model works best as a completion-style Q/A model rather than as a general-purpose chat assistant.
+
+Recommended format:
+
+```text
+Q: <computer engineering question>
+A:
+```
+
+Example:
+
+```text
+Q: Explain the purpose of CPU cache memory.
+A:
+```
+
+Recommended generation settings:
+
+```python
+max_new_tokens = 200
+temperature = 0.7
+top_p = 0.9
+do_sample = True
+repetition_penalty = 1.1
+```
+
+Short, direct technical questions generally work better than role-based conversational prompts.
+
+## Usage
+
+Install OpenVINO GenAI:
+
+```bash
+python -m pip install "openvino==2026.4" "openvino-genai==2026.4"
+```
+
+Example:
+
+```python
+import openvino_genai as ov_genai
+
+model_path = "./Llama-3.2-1B-Computer-Engineering-OpenVINO-INT4"
+
+# CPU, GPU, or NPU depending on available OpenVINO devices.
+device = "CPU"
+
+pipe = ov_genai.LLMPipeline(
+    model_path,
+    device,
+)
+
+config = ov_genai.GenerationConfig()
+
+config.max_new_tokens = 200
+config.do_sample = True
+config.temperature = 0.7
+config.top_p = 0.9
+config.repetition_penalty = 1.1
+config.apply_chat_template = False
+
+question = "Explain the purpose of CPU cache memory."
+
+prompt = f"""Q: {question}
+A:"""
+
+result = pipe.generate(
+    prompt,
+    config,
+)
+
+print(result)
+```
+
+Available devices can be checked with:
+
+```python
+import openvino as ov
+
+core = ov.Core()
+print(core.available_devices)
+```
+
+On the IULinux test system used for this release:
+
+```text
+['CPU', 'GPU', 'NPU']
+```
+
+## Conversion
+
+The parent model is a genuine BitsAndBytes INT8 checkpoint rather than a floating-point checkpoint carrying only quantization metadata.
+
+Inspection of the parent model showed INT8 linear weights, SCB scaling tensors, and 112 `Linear8bitLt` modules.
+
+The conversion path was:
+
+```text
+meta-llama/Llama-3.2-1B
+        ↓
+Computer Engineering fine-tuning
+        ↓
+BitsAndBytes INT8 parent checkpoint
+        ↓
+Dequantized dense FP16 representation
+        ↓
+OpenVINO INT4 weight compression
+```
+
+After dequantization, the intermediate checkpoint contained:
+
+```text
+146 FP16 tensors
+0 BitsAndBytes linear modules
+0 INT8 weight tensors
+0 SCB tensors
+```
+
+The OpenVINO conversion used:
+
+```bash
+optimum-cli export openvino \
+  --model ./source-dequant-fp16 \
+  --task text-generation-with-past \
+  --weight-format int4 \
+  --sym \
+  --ratio 1.0 \
+  --group-size 128 \
+  ./Llama-3.2-1B-Computer-Engineering-OpenVINO-INT4
+```
+
+NNCF reported that all **112 ratio-defining layers** were compressed using:
+
+```text
+INT4 symmetric
+group size 128
+ratio 1.0
+```
+
+A non-ratio-defining component used INT8 asymmetric backup precision.
+
+Because the parent model was already quantized, this model should not be described as a direct conversion from the original pre-quantization FP16 weights. The conversion includes a lossy INT8 → dense FP16 reconstruction before INT4 compression.
+
+## Size
+
+Observed local sizes during conversion:
+
+| Variant | Approximate size |
+|---|---:|
+| Dense FP16 | 2.4 GB |
+| OpenVINO FP16 | 2.4 GB |
+| OpenVINO INT8 | 1.2 GB |
+| OpenVINO INT4 | 758 MB |
+
+## Benchmarks
+
+Benchmarks were run on IULinux using the same model, prompt, and generation workload on all three devices.
+
+### System
+
+| Component | Configuration |
+|---|---|
+| OS | IULinux 0.1 Development |
+| Kernel | 7.0.0-31-generic |
+| CPU | Intel Core Ultra 9 275HX |
+| GPU | Intel Graphics (iGPU) |
+| NPU | Intel AI Boost |
+| OpenVINO | 2026.4 |
+| OpenVINO GenAI | 2026.4 |
+| Power profile | Performance |
+| Power source | AC |
+
+OpenVINO exposed:
+
+```text
+CPU: Intel(R) Core(TM) Ultra 9 275HX
+GPU: Intel(R) Graphics (iGPU)
+NPU: Intel(R) AI Boost
+```
+
+### Method
+
+Each backend used:
+
+```text
+Prompt tokens: 26
+Generated tokens: 128
+Warm-up runs: 2
+Measured runs: 5
+do_sample: false
+ignore_eos: true
+```
+
+Benchmark prompt:
+
+```text
+Q: Explain how CPU cache memory improves processor performance, including its relationship with RAM, latency, and memory locality.
+A:
+```
+
+`ignore_eos=True` was used so every backend performed the same 128-token generation workload.
+
+Greedy decoding was used only for deterministic performance measurement. The recommended sampling configuration above should be used for normal model interaction.
+
+### Results
+
+| Device | Pipeline load | TTFT | TPOT | Throughput |
+|---|---:|---:|---:|---:|
+| Intel Core Ultra 9 275HX | 382.59 ms | 51.57 ± 9.91 ms | 15.90 ± 0.74 ms/token | **62.98 ± 2.79 tok/s** |
+| Intel Graphics iGPU | 1218.85 ms | 54.24 ± 1.77 ms | 15.98 ± 0.29 ms/token | **62.59 ± 1.14 tok/s** |
+| Intel AI Boost NPU | 986.20 ms | 1147.08 ± 4.16 ms | 29.51 ± 0.10 ms/token | **33.89 ± 0.11 tok/s** |
+
+On this specific 1B INT4 workload, the CPU and integrated GPU produced very similar generation throughput.
+
+The NPU produced lower throughput and substantially higher TTFT in this configuration, but its measured runs were highly consistent.
+
+No device-level power measurements were collected, so these results compare latency and throughput only and should not be interpreted as a power-efficiency comparison.
+
+Results will vary with hardware, firmware, runtime versions, thermal conditions, power configuration, prompt length, and generation settings.
+
+## IULinux
+
+The complete conversion and validation workflow was performed on **IULinux**.
+
+The same IULinux installation exposed all three Intel OpenVINO backends:
+
+```text
+CPU
+GPU
+NPU
+```
+
+The Intel AI Boost NPU used the `intel_vpu` kernel driver together with the Intel NPU Level Zero userspace stack.
+
+The integrated GPU used the existing `i915` kernel driver together with the Intel OpenCL compute runtime.
+
+No NVIDIA driver modifications were required for this OpenVINO deployment.
+
+The benchmark files and environment capture included in this repository record the runtime configuration used for the published results.
+
+## Quality checks
+
+The parent model was initially tested using generic greedy prompts and showed significant repetition.
+
+Testing against the original parent checkpoint showed that the model behaves substantially better using its intended Q/A completion format:
+
+```text
+Q: <question>
+A:
+```
+
+The same prompt format was then tested on the OpenVINO variants.
+
+Observed average repeated 4-gram ratios:
+
+| Variant | Average repeated 4-gram ratio |
+|---|---:|
+| OpenVINO FP16 | 0.002 |
+| OpenVINO INT8 | 0.000 |
+| OpenVINO INT4 | 0.003 |
+
+These repetition measurements are diagnostic checks rather than formal model-quality scores.
+
+## Files
+
+The repository contains the OpenVINO model, tokenizer and detokenizer IR files, configuration files, benchmark data, environment information, and SHA-256 checksums.
+
+Benchmark files are stored under:
+
+```text
+benchmarks/
+```
+
+The final benchmark contains both raw console output and machine-readable JSON.
+
+## Limitations
+
+This is a specialized Computer Engineering completion model and is not intended to behave like a general-purpose conversational assistant.
+
+Technical answers can contain factual errors and should be independently verified for critical engineering work.
+
+The model may continue generating additional Q/A pairs after completing an answer. Reducing `max_new_tokens` can help control output length.
+
+The INT4 model was derived from a previously quantized BitsAndBytes INT8 checkpoint. Quantization and dequantization are lossy operations, and the original pre-quantization precision cannot be reconstructed exactly.
+
+The benchmark results apply only to the tested hardware and software configuration.
+
+## License
+
+This model is governed by the Llama 3.2 Community License.
+
+Use, modification, and redistribution must comply with the Llama 3.2 Community License and its associated requirements.
+
+Llama 3.2 is licensed under the Llama 3.2 Community License, Copyright © Meta Platforms, Inc. All Rights Reserved.
+
+## Parent model
+
+`Irfanuruchi/Llama-3.2-1B-Computer-Engineering-LLM`
+
+## Citation
+
+```bibtex
+@misc{uruchi2026llama32ceopenvino,
+  title     = {Llama-3.2-1B-Computer-Engineering-OpenVINO-INT4},
+  author    = {Irfanuruchi},
+  year      = {2026},
+  publisher = {Hugging Face}
+}
+```

SHA256SUMS.txt

@@ -0,0 +1,16 @@
+6d76894816f64065359464c6a0cf3e00d3c63b38847a41ee3ae97f9a0d765ed7  ./benchmarks/benchmark_release.py
+2605216c8a29165dac1bd7e97158e72ca3981fdf70d3f30067ed424cefa388f6  ./benchmarks/final-release-benchmark.json
+dbde37a822cb6e3a630a85b7f2d23ece5913c62f90071369cfc7e27d53b5700c  ./benchmarks/final-release-benchmark.txt
+d22e3353ee9116a01b3826ea3acb7236b1a50c08323d5c4e276fcc7ebf7d1f08  ./benchmarks/release-environment.txt
+3df7cf4aa6d95dd16f408e3e98ae49d049b13c9f9f89c3fad82bcaafb195400e  ./config.json
+47f788591879f0622d5ab7299abc2e0a98d2e6954ab16054c7c3010c0263be0f  ./generation_config.json
+89c5c47561f08b757faca9fec6500fd19b465baa0fd6a0a0ad381fbc491688f5  ./openvino_config.json
+0643181dd67c7f88a9fc3b6309ce4fe96433348ae83709c72fa100f6afca5a53  ./openvino_detokenizer.bin
+713125bebf179dc4d823b2c11d5416a28d2b25f75077ac79e09f3312387a79dc  ./openvino_detokenizer.xml
+c9feb7d66e0b698c79519b54a6281a787e59c87f08d46f4eb9c9202a13bf7779  ./openvino_model.bin
+9aecc65bb8adcb1a8ed5faef656ad42465cbe26bf40a34acc842144f4413fbc7  ./openvino_model.xml
+e85dcd84b4ad36ae6104c68e063edb024f2ec32c3b74536324b5536df1f55cc8  ./openvino_tokenizer.bin
+b6d59f076e45a4b179b0242ce518d2ea218e31b9949216a88d525a542ab56098  ./openvino_tokenizer.xml
+69e6c0c19c6b113c0bf230e987db6b14fc73877c18e7234b735deeeb9b470ee2  ./README.md
+fcb449400c8ede301786f3f653d88a59208c716d857b3f78e2ebbb0425d9b0ce  ./tokenizer_config.json
+6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b  ./tokenizer.json

benchmarks/benchmark_release.py

@@ -0,0 +1,376 @@
+import argparse
+import gc
+import json
+import platform
+import statistics
+import subprocess
+import time
+from datetime import datetime
+from pathlib import Path
+
+import openvino as ov
+import openvino_genai as ov_genai
+
+
+MODEL = Path(
+    "./Llama-3.2-1B-Computer-Engineering-OpenVINO-INT4"
+)
+
+PROMPT_TEXT = """Q: Explain how CPU cache memory improves processor performance, including its relationship with RAM, latency, and memory locality.
+A:"""
+
+# OpenVINO GenAI returns DecodedResults with perf_metrics
+# when generation input is a list.
+PROMPT_BATCH = [PROMPT_TEXT]
+
+
+def command_output(command):
+    try:
+        return subprocess.check_output(
+            command,
+            text=True,
+            stderr=subprocess.DEVNULL,
+        ).strip()
+    except Exception:
+        return "unknown"
+
+
+def benchmark_device(device, warmups, iterations, tokens):
+    core = ov.Core()
+
+    full_name = core.get_property(
+        device,
+        "FULL_DEVICE_NAME",
+    )
+
+    print()
+    print("=" * 78)
+    print(f" DEVICE: {device}")
+    print(f" NAME:   {full_name}")
+    print("=" * 78)
+
+    started = time.perf_counter()
+
+    pipe = ov_genai.LLMPipeline(
+        str(MODEL),
+        device,
+    )
+
+    load_ms = (
+        time.perf_counter() - started
+    ) * 1000
+
+    config = ov_genai.GenerationConfig()
+    config.max_new_tokens = tokens
+
+    # Force identical work on every device.
+    config.ignore_eos = True
+
+    # Deterministic benchmark path.
+    config.do_sample = False
+    config.apply_chat_template = False
+
+    encoded = pipe.get_tokenizer().encode(PROMPT_BATCH)
+    prompt_tokens = encoded.input_ids.get_shape()[1]
+
+    print(f"Pipeline load: {load_ms:.2f} ms")
+    print(f"Prompt tokens: {prompt_tokens}")
+    print(f"Forced output tokens: {tokens}")
+
+    print()
+    print("=== WARMUP ===")
+
+    for i in range(warmups):
+        pipe.generate(
+            PROMPT_BATCH,
+            config,
+        )
+
+        print(
+            f"Warmup {i + 1}/{warmups}: complete"
+        )
+
+    print()
+    print("=== MEASURED RUNS ===")
+
+    runs = []
+
+    for i in range(iterations):
+        result = pipe.generate(
+            PROMPT_BATCH,
+            config,
+        )
+
+        metrics = result.perf_metrics
+
+        run = {
+            "ttft_ms":
+                float(metrics.get_ttft().mean),
+
+            "tpot_ms_per_token":
+                float(metrics.get_tpot().mean),
+
+            "throughput_tokens_per_second":
+                float(metrics.get_throughput().mean),
+
+            "generated_tokens":
+                int(metrics.get_num_generated_tokens()),
+
+            "input_tokens":
+                int(metrics.get_num_input_tokens()),
+        }
+
+        runs.append(run)
+
+        print(
+            f"Run {i + 1}: "
+            f"TTFT={run['ttft_ms']:.2f} ms | "
+            f"TPOT={run['tpot_ms_per_token']:.2f} ms/token | "
+            f"Throughput="
+            f"{run['throughput_tokens_per_second']:.2f} tok/s"
+        )
+
+    def stats(key):
+        values = [run[key] for run in runs]
+
+        mean = statistics.mean(values)
+
+        std = (
+            statistics.stdev(values)
+            if len(values) > 1
+            else 0.0
+        )
+
+        return {
+            "mean": mean,
+            "std": std,
+            "min": min(values),
+            "max": max(values),
+        }
+
+    summary = {
+        "device": device,
+        "full_device_name": full_name,
+        "pipeline_load_ms": load_ms,
+        "prompt_tokens": prompt_tokens,
+        "forced_output_tokens": tokens,
+        "warmups": warmups,
+        "iterations": iterations,
+        "ttft_ms": stats("ttft_ms"),
+        "tpot_ms_per_token": stats(
+            "tpot_ms_per_token"
+        ),
+        "throughput_tokens_per_second": stats(
+            "throughput_tokens_per_second"
+        ),
+        "runs": runs,
+    }
+
+    print()
+    print("=== FINAL ===")
+
+    print(
+        f"TTFT:       "
+        f"{summary['ttft_ms']['mean']:.2f} ± "
+        f"{summary['ttft_ms']['std']:.2f} ms"
+    )
+
+    print(
+        f"TPOT:       "
+        f"{summary['tpot_ms_per_token']['mean']:.2f} ± "
+        f"{summary['tpot_ms_per_token']['std']:.2f} ms/token"
+    )
+
+    print(
+        f"Throughput: "
+        f"{summary['throughput_tokens_per_second']['mean']:.2f} ± "
+        f"{summary['throughput_tokens_per_second']['std']:.2f} tok/s"
+    )
+
+    del pipe
+    gc.collect()
+
+    return summary
+
+
+def main():
+    parser = argparse.ArgumentParser()
+
+    parser.add_argument(
+        "--device",
+        choices=["CPU", "GPU", "NPU", "ALL"],
+        default="ALL",
+    )
+
+    parser.add_argument(
+        "--warmup",
+        type=int,
+        default=2,
+    )
+
+    parser.add_argument(
+        "--iterations",
+        type=int,
+        default=5,
+    )
+
+    parser.add_argument(
+        "--tokens",
+        type=int,
+        default=128,
+    )
+
+    args = parser.parse_args()
+
+    core = ov.Core()
+    available = core.available_devices
+
+    requested = (
+        ["CPU", "NPU", "GPU"]
+        if args.device == "ALL"
+        else [args.device]
+    )
+
+    selected = [
+        device
+        for device in requested
+        if device in available
+    ]
+
+    print("=" * 78)
+    print(" OPENVINO INT4 RELEASE BENCHMARK")
+    print("=" * 78)
+
+    print("Date:", datetime.now().astimezone().isoformat())
+    print("Model:", MODEL)
+    print("OpenVINO:", ov.__version__)
+
+    print(
+        "OpenVINO GenAI:",
+        getattr(
+            ov_genai,
+            "__version__",
+            "unknown",
+        ),
+    )
+
+    print("Available devices:", available)
+    print("Benchmark devices:", selected)
+
+    print(
+        "Power profile:",
+        command_output(
+            ["powerprofilesctl", "get"]
+        ),
+    )
+
+    print("Kernel:", platform.release())
+
+    print()
+    print("NOTE:")
+    print(
+        "Q:/A: prompt format is used because this model "
+        "is a completion-style Computer Engineering model."
+    )
+
+    print(
+        "ignore_eos=True forces every backend to generate "
+        f"exactly {args.tokens} tokens."
+    )
+
+    results = []
+
+    for device in selected:
+        results.append(
+            benchmark_device(
+                device,
+                args.warmup,
+                args.iterations,
+                args.tokens,
+            )
+        )
+
+    report = {
+        "timestamp":
+            datetime.now().astimezone().isoformat(),
+
+        "model":
+            str(MODEL),
+
+        "openvino_version":
+            ov.__version__,
+
+        "openvino_genai_version":
+            getattr(
+                ov_genai,
+                "__version__",
+                "unknown",
+            ),
+
+        "power_profile":
+            command_output(
+                ["powerprofilesctl", "get"]
+            ),
+
+        "kernel":
+            platform.release(),
+
+        "prompt":
+            PROMPT_TEXT,
+
+        "benchmark_method": {
+            "warmups": args.warmup,
+            "iterations": args.iterations,
+            "max_new_tokens": args.tokens,
+            "ignore_eos": True,
+            "do_sample": False,
+        },
+
+        "devices": results,
+    }
+
+    Path("results").mkdir(
+        parents=True,
+        exist_ok=True,
+    )
+
+    Path(
+        "results/final-release-benchmark.json"
+    ).write_text(
+        json.dumps(
+            report,
+            indent=2,
+        ) + "\n",
+        encoding="utf-8",
+    )
+
+    print()
+    print("=" * 78)
+    print(" FINAL COMPARISON")
+    print("=" * 78)
+
+    print(
+        f"{'DEVICE':<8}"
+        f"{'TTFT ms':>14}"
+        f"{'TPOT ms/tok':>16}"
+        f"{'TOKENS/s':>14}"
+    )
+
+    print("-" * 52)
+
+    for item in results:
+        print(
+            f"{item['device']:<8}"
+            f"{item['ttft_ms']['mean']:>14.2f}"
+            f"{item['tpot_ms_per_token']['mean']:>16.2f}"
+            f"{item['throughput_tokens_per_second']['mean']:>14.2f}"
+        )
+
+    print()
+    print(
+        "Saved: results/final-release-benchmark.json"
+    )
+
+
+if __name__ == "__main__":
+    main()

benchmarks/final-release-benchmark.json

@@ -0,0 +1,210 @@
+{
+  "timestamp": "2026-09-28T16:10:37.573753+02:00",
+  "model": "Llama-3.2-1B-Computer-Engineering-OpenVINO-INT4",
+  "openvino_version": "2026.4.0-22959-99c81491cc3-releases/2026/4",
+  "openvino_genai_version": "2026.4.0.0-3407-7ea2546852a",
+  "power_profile": "performance",
+  "kernel": "7.0.0-31-generic",
+  "prompt": "Q: Explain how CPU cache memory improves processor performance, including its relationship with RAM, latency, and memory locality.\nA:",
+  "benchmark_method": {
+    "warmups": 2,
+    "iterations": 5,
+    "max_new_tokens": 128,
+    "ignore_eos": true,
+    "do_sample": false
+  },
+  "devices": [
+    {
+      "device": "CPU",
+      "full_device_name": "Intel(R) Core(TM) Ultra 9 275HX",
+      "pipeline_load_ms": 382.5941410004816,
+      "prompt_tokens": 26,
+      "forced_output_tokens": 128,
+      "warmups": 2,
+      "iterations": 5,
+      "ttft_ms": {
+        "mean": 51.57345275878906,
+        "std": 9.913688884312469,
+        "min": 40.536006927490234,
+        "max": 62.45323181152344
+      },
+      "tpot_ms_per_token": {
+        "mean": 15.903347778320313,
+        "std": 0.7387210062345533,
+        "min": 15.300374984741211,
+        "max": 17.175537109375
+      },
+      "throughput_tokens_per_second": {
+        "mean": 62.983341979980466,
+        "std": 2.7861940137138532,
+        "min": 58.22234344482422,
+        "max": 65.35787200927734
+      },
+      "runs": [
+        {
+          "ttft_ms": 43.170963287353516,
+          "tpot_ms_per_token": 15.536717414855957,
+          "throughput_tokens_per_second": 64.36366271972656,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        },
+        {
+          "ttft_ms": 51.07387924194336,
+          "tpot_ms_per_token": 15.300374984741211,
+          "throughput_tokens_per_second": 65.35787200927734,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        },
+        {
+          "ttft_ms": 62.45323181152344,
+          "tpot_ms_per_token": 15.853533744812012,
+          "throughput_tokens_per_second": 63.07741928100586,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        },
+        {
+          "ttft_ms": 40.536006927490234,
+          "tpot_ms_per_token": 17.175537109375,
+          "throughput_tokens_per_second": 58.22234344482422,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        },
+        {
+          "ttft_ms": 60.633182525634766,
+          "tpot_ms_per_token": 15.650575637817383,
+          "throughput_tokens_per_second": 63.89541244506836,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        }
+      ]
+    },
+    {
+      "device": "NPU",
+      "full_device_name": "Intel(R) AI Boost",
+      "pipeline_load_ms": 986.1992260002808,
+      "prompt_tokens": 26,
+      "forced_output_tokens": 128,
+      "warmups": 2,
+      "iterations": 5,
+      "ttft_ms": {
+        "mean": 1147.07705078125,
+        "std": 4.160363485341516,
+        "min": 1143.1307373046875,
+        "max": 1152.9591064453125
+      },
+      "tpot_ms_per_token": {
+        "mean": 29.50558738708496,
+        "std": 0.09963660070982523,
+        "min": 29.38804054260254,
+        "max": 29.640350341796875
+      },
+      "throughput_tokens_per_second": {
+        "mean": 33.89219512939453,
+        "std": 0.11440889767595792,
+        "min": 33.73779296875,
+        "max": 34.02744674682617
+      },
+      "runs": [
+        {
+          "ttft_ms": 1152.9591064453125,
+          "tpot_ms_per_token": 29.53006935119629,
+          "throughput_tokens_per_second": 33.86378860473633,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        },
+        {
+          "ttft_ms": 1143.8863525390625,
+          "tpot_ms_per_token": 29.640350341796875,
+          "throughput_tokens_per_second": 33.73779296875,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        },
+        {
+          "ttft_ms": 1145.6927490234375,
+          "tpot_ms_per_token": 29.54069709777832,
+          "throughput_tokens_per_second": 33.85160446166992,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        },
+        {
+          "ttft_ms": 1149.71630859375,
+          "tpot_ms_per_token": 29.42877960205078,
+          "throughput_tokens_per_second": 33.980342864990234,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        },
+        {
+          "ttft_ms": 1143.1307373046875,
+          "tpot_ms_per_token": 29.38804054260254,
+          "throughput_tokens_per_second": 34.02744674682617,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        }
+      ]
+    },
+    {
+      "device": "GPU",
+      "full_device_name": "Intel(R) Graphics (iGPU)",
+      "pipeline_load_ms": 1218.8496329999907,
+      "prompt_tokens": 26,
+      "forced_output_tokens": 128,
+      "warmups": 2,
+      "iterations": 5,
+      "ttft_ms": {
+        "mean": 54.238851928710936,
+        "std": 1.7680915292515056,
+        "min": 52.691978454589844,
+        "max": 57.2708854675293
+      },
+      "tpot_ms_per_token": {
+        "mean": 15.981847381591797,
+        "std": 0.29188068731692896,
+        "min": 15.636512756347656,
+        "max": 16.417612075805664
+      },
+      "throughput_tokens_per_second": {
+        "mean": 62.58757781982422,
+        "std": 1.1355620861494284,
+        "min": 60.91019821166992,
+        "max": 63.952877044677734
+      },
+      "runs": [
+        {
+          "ttft_ms": 53.909603118896484,
+          "tpot_ms_per_token": 16.062355041503906,
+          "throughput_tokens_per_second": 62.25736999511719,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        },
+        {
+          "ttft_ms": 53.927425384521484,
+          "tpot_ms_per_token": 15.96897029876709,
+          "throughput_tokens_per_second": 62.62144470214844,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        },
+        {
+          "ttft_ms": 53.39436721801758,
+          "tpot_ms_per_token": 15.823786735534668,
+          "throughput_tokens_per_second": 63.19599914550781,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        },
+        {
+          "ttft_ms": 52.691978454589844,
+          "tpot_ms_per_token": 16.417612075805664,
+          "throughput_tokens_per_second": 60.91019821166992,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        },
+        {
+          "ttft_ms": 57.2708854675293,
+          "tpot_ms_per_token": 15.636512756347656,
+          "throughput_tokens_per_second": 63.952877044677734,
+          "generated_tokens": 128,
+          "input_tokens": 26
+        }
+      ]
+    }
+  ]
+}

benchmarks/final-release-benchmark.txt

@@ -0,0 +1,98 @@
+==============================================================================
+ OPENVINO INT4 RELEASE BENCHMARK
+==============================================================================
+Date: 2026-09-28T16:09:30.299304+02:00
+Model: Llama-3.2-1B-Computer-Engineering-OpenVINO-INT4
+OpenVINO: 2026.4.0-22959-99c81491cc3-releases/2026/4
+OpenVINO GenAI: 2026.4.0.0-3407-7ea2546852a
+Available devices: ['CPU', 'GPU', 'NPU']
+Benchmark devices: ['CPU', 'NPU', 'GPU']
+Power profile: performance
+Kernel: 7.0.0-31-generic
+
+NOTE:
+Q:/A: prompt format is used because this model is a completion-style Computer Engineering model.
+ignore_eos=True forces every backend to generate exactly 128 tokens.
+
+==============================================================================
+ DEVICE: CPU
+ NAME:   Intel(R) Core(TM) Ultra 9 275HX
+==============================================================================
+Pipeline load: 382.59 ms
+Prompt tokens: 26
+Forced output tokens: 128
+
+=== WARMUP ===
+Warmup 1/2: complete
+Warmup 2/2: complete
+
+=== MEASURED RUNS ===
+Run 1: TTFT=43.17 ms | TPOT=15.54 ms/token | Throughput=64.36 tok/s
+Run 2: TTFT=51.07 ms | TPOT=15.30 ms/token | Throughput=65.36 tok/s
+Run 3: TTFT=62.45 ms | TPOT=15.85 ms/token | Throughput=63.08 tok/s
+Run 4: TTFT=40.54 ms | TPOT=17.18 ms/token | Throughput=58.22 tok/s
+Run 5: TTFT=60.63 ms | TPOT=15.65 ms/token | Throughput=63.90 tok/s
+
+=== FINAL ===
+TTFT:       51.57 ± 9.91 ms
+TPOT:       15.90 ± 0.74 ms/token
+Throughput: 62.98 ± 2.79 tok/s
+
+==============================================================================
+ DEVICE: NPU
+ NAME:   Intel(R) AI Boost
+==============================================================================
+Pipeline load: 986.20 ms
+Prompt tokens: 26
+Forced output tokens: 128
+
+=== WARMUP ===
+Warmup 1/2: complete
+Warmup 2/2: complete
+
+=== MEASURED RUNS ===
+Run 1: TTFT=1152.96 ms | TPOT=29.53 ms/token | Throughput=33.86 tok/s
+Run 2: TTFT=1143.89 ms | TPOT=29.64 ms/token | Throughput=33.74 tok/s
+Run 3: TTFT=1145.69 ms | TPOT=29.54 ms/token | Throughput=33.85 tok/s
+Run 4: TTFT=1149.72 ms | TPOT=29.43 ms/token | Throughput=33.98 tok/s
+Run 5: TTFT=1143.13 ms | TPOT=29.39 ms/token | Throughput=34.03 tok/s
+
+=== FINAL ===
+TTFT:       1147.08 ± 4.16 ms
+TPOT:       29.51 ± 0.10 ms/token
+Throughput: 33.89 ± 0.11 tok/s
+
+==============================================================================
+ DEVICE: GPU
+ NAME:   Intel(R) Graphics (iGPU)
+==============================================================================
+Pipeline load: 1218.85 ms
+Prompt tokens: 26
+Forced output tokens: 128
+
+=== WARMUP ===
+Warmup 1/2: complete
+Warmup 2/2: complete
+
+=== MEASURED RUNS ===
+Run 1: TTFT=53.91 ms | TPOT=16.06 ms/token | Throughput=62.26 tok/s
+Run 2: TTFT=53.93 ms | TPOT=15.97 ms/token | Throughput=62.62 tok/s
+Run 3: TTFT=53.39 ms | TPOT=15.82 ms/token | Throughput=63.20 tok/s
+Run 4: TTFT=52.69 ms | TPOT=16.42 ms/token | Throughput=60.91 tok/s
+Run 5: TTFT=57.27 ms | TPOT=15.64 ms/token | Throughput=63.95 tok/s
+
+=== FINAL ===
+TTFT:       54.24 ± 1.77 ms
+TPOT:       15.98 ± 0.29 ms/token
+Throughput: 62.59 ± 1.14 tok/s
+
+==============================================================================
+ FINAL COMPARISON
+==============================================================================
+DEVICE         TTFT ms     TPOT ms/tok      TOKENS/s
+----------------------------------------------------
+CPU              51.57           15.90         62.98
+NPU            1147.08           29.51         33.89
+GPU              54.24           15.98         62.59
+
+Saved: results/final-release-benchmark.json

benchmarks/release-environment.txt

@@ -0,0 +1,53 @@
+=== DATE ===
+2026-09-28T16:11:02+02:00
+
+=== OS ===
+NAME="IULinux"
+VERSION="0.1 Development"
+
+=== KERNEL ===
+7.0.0-31-generic
+
+=== POWER PROFILE ===
+performance
+
+=== OPENVINO DEVICES ===
+OpenVINO: 2026.4.0-22959-99c81491cc3-releases/2026/4
+OpenVINO: 2026.4.0-22959-99c81491cc3-releases/2026/4
+Devices: ['CPU', 'GPU', 'NPU']
+
+DEVICE: CPU
+NAME: Intel(R) Core(TM) Ultra 9 275HX
+
+DEVICE: GPU
+NAME: Intel(R) Graphics (iGPU)
+
+DEVICE: NPU
+NAME: Intel(R) AI Boost
+Devices: ['CPU', 'GPU', 'NPU']
+
+DEVICE: CPU
+NAME: Intel(R) Core(TM) Ultra 9 275HX
+
+DEVICE: GPU
+NAME: Intel(R) Graphics (iGPU)
+
+DEVICE: NPU
+NAME: Intel(R) AI Boost
+
+=== OPENCL ===
+Platform #0: Intel(R) OpenCL Graphics
+ `-- Device #0: Intel(R) Graphics
+
+=== NPU STACK ===
+ii  intel-driver-compiler-npu                           1.38.0.20260910-34487311128~ubuntu26.04    amd64        Intel(R) Driver Compiler for NPU hardware
+ii  intel-fw-npu                                        1.38.0.20260910-34487311128~ubuntu26.04    amd64        Intel(R) Firmware package for NPU hardware
+ii  intel-level-zero-npu                                1.38.0.20260910-34487311128~ubuntu26.04    amd64        Intel(R) Level Zero Driver for NPU hardware
+ii  libze1:amd64                                        1.32.0-1~26.04~ppa1                        amd64        oneAPI Level Zero -- share libraries
+
+=== GPU STACK ===
+ii  intel-opencl-icd                                    26.05.37020.3-1                            amd64        Intel graphics compute runtime for OpenCL
+ii  intel-opencl-icd-legacy                             24.35.30872.45-1                           amd64        Intel graphics compute runtime for OpenCL -- legacy platforms
+ii  libigc1:amd64                                       1.0.17791.18+1-3                           amd64        Intel graphics compiler for OpenCL -- core libs
+ii  libigc2:amd64                                       2.28.4-4                                   amd64        Intel graphics compiler for OpenCL -- core libs
+ii  libigdgmm12:amd64                                   22.9.0+ds1-1                               amd64        Intel Graphics Memory Management Library -- shared library

config.json

@@ -0,0 +1,36 @@
+{
+  "architectures": [
+    "LlamaForCausalLM"
+  ],
+  "attention_bias": false,
+  "attention_dropout": 0.0,
+  "bos_token_id": 128000,
+  "dtype": "float16",
+  "eos_token_id": 128001,
+  "head_dim": 64,
+  "hidden_act": "silu",
+  "hidden_size": 2048,
+  "initializer_range": 0.02,
+  "intermediate_size": 8192,
+  "max_position_embeddings": 131072,
+  "mlp_bias": false,
+  "model_type": "llama",
+  "num_attention_heads": 32,
+  "num_hidden_layers": 16,
+  "num_key_value_heads": 8,
+  "pad_token_id": null,
+  "pretraining_tp": 1,
+  "rms_norm_eps": 1e-05,
+  "rope_parameters": {
+    "factor": 32.0,
+    "high_freq_factor": 4.0,
+    "low_freq_factor": 1.0,
+    "original_max_position_embeddings": 8192,
+    "rope_theta": 500000.0,
+    "rope_type": "llama3"
+  },
+  "tie_word_embeddings": true,
+  "transformers_version": "5.5.4",
+  "use_cache": true,
+  "vocab_size": 128256
+}

generation_config.json

@@ -0,0 +1,9 @@
+{
+  "_from_model_config": true,
+  "bos_token_id": 128000,
+  "do_sample": true,
+  "eos_token_id": 128001,
+  "temperature": 0.6,
+  "top_p": 0.9,
+  "transformers_version": "5.5.4"
+}

openvino_config.json

@@ -0,0 +1,20 @@
+{
+  "dtype": "",
+  "input_info": null,
+  "optimum_version": "2.3.0",
+  "output_attentions": false,
+  "quantization_config": {
+    "_dataset_kwargs": {},
+    "dataset": null,
+    "default_config": {
+      "quant_method": "default"
+    },
+    "ignored_scope": null,
+    "num_samples": null,
+    "processor": null,
+    "quantization_configs": {},
+    "tokenizer": "./source-dequant-fp16"
+  },
+  "save_onnx_model": false,
+  "transformers_version": "5.5.4"
+}

openvino_detokenizer.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:0643181dd67c7f88a9fc3b6309ce4fe96433348ae83709c72fa100f6afca5a53
+size 1865893

openvino_detokenizer.xml

@@ -0,0 +1,273 @@
+<?xml version="1.0"?>
+<net name="detokenizer" version="11">
+	<layers>
+		<layer id="0" name="Parameter_58393" type="Parameter" version="opset1">
+			<data shape="?,?" element_type="i64" />
+			<output>
+				<port id="0" precision="I64" names="Parameter_58393">
+					<dim>-1</dim>
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="1" name="Convert_58433" type="Convert" version="opset1">
+			<data destination_type="i32" />
+			<input>
+				<port id="0" precision="I64">
+					<dim>-1</dim>
+					<dim>-1</dim>
+				</port>
+			</input>
+			<output>
+				<port id="1" precision="I32">
+					<dim>-1</dim>
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="2" name="Constant_58395" type="Const" version="opset1">
+			<data element_type="i32" shape="128256" offset="0" size="513024" />
+			<output>
+				<port id="0" precision="I32">
+					<dim>128256</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="3" name="Constant_58397" type="Const" version="opset1">
+			<data element_type="i32" shape="128256" offset="513024" size="513024" />
+			<output>
+				<port id="0" precision="I32">
+					<dim>128256</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="4" name="Constant_58399" type="Const" version="opset1">
+			<data element_type="u8" shape="838768" offset="1026048" size="838768" />
+			<output>
+				<port id="0" precision="U8">
+					<dim>838768</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="5" name="Slice_58404" type="Const" version="opset1">
+			<data element_type="i32" shape="256" offset="1864816" size="1024" />
+			<output>
+				<port id="0" precision="I32">
+					<dim>256</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="6" name="VocabDecoder_58406" type="VocabDecoder" version="extension">
+			<data skip_tokens="" />
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+					<dim>-1</dim>
+				</port>
+				<port id="1" precision="I32">
+					<dim>128256</dim>
+				</port>
+				<port id="2" precision="I32">
+					<dim>128256</dim>
+				</port>
+				<port id="3" precision="U8">
+					<dim>838768</dim>
+				</port>
+				<port id="4" precision="I32">
+					<dim>256</dim>
+				</port>
+			</input>
+			<output>
+				<port id="5" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="6" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="7" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="8" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="9" precision="U8">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="7" name="FuzeRagged_58407" type="FuzeRagged" version="extension">
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="1" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="2" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="3" precision="I32">
+					<dim>-1</dim>
+				</port>
+			</input>
+			<output>
+				<port id="4" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="5" precision="I32">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="8" name="UTF8Validate_58408" type="UTF8Validate" version="extension">
+			<data replace_mode="true" />
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="1" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="2" precision="U8">
+					<dim>-1</dim>
+				</port>
+			</input>
+			<output>
+				<port id="3" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="4" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="5" precision="U8">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="9" name="Constant_58410" type="Const" version="opset1">
+			<data element_type="u8" shape="51" offset="1865840" size="51" />
+			<output>
+				<port id="0" precision="U8">
+					<dim>51</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="10" name="Constant_58412" type="Const" version="opset1">
+			<data element_type="u8" shape="2" offset="1865891" size="2" />
+			<output>
+				<port id="0" precision="U8">
+					<dim>2</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="11" name="RegexNormalization_58413" type="RegexNormalization" version="extension">
+			<data global_replace="true" />
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="1" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="2" precision="U8">
+					<dim>-1</dim>
+				</port>
+				<port id="3" precision="U8">
+					<dim>51</dim>
+				</port>
+				<port id="4" precision="U8">
+					<dim>2</dim>
+				</port>
+			</input>
+			<output>
+				<port id="5" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="6" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="7" precision="U8">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="12" name="StringTensorPack_58414" type="StringTensorPack" version="opset15">
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="1" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="2" precision="U8">
+					<dim>-1</dim>
+				</port>
+			</input>
+			<output>
+				<port id="3" precision="STRING" names="Result_58415,string_output">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="13" name="Result_58415" type="Result" version="opset1" output_names="Result_58415,string_output">
+			<input>
+				<port id="0" precision="STRING">
+					<dim>-1</dim>
+				</port>
+			</input>
+		</layer>
+	</layers>
+	<edges>
+		<edge from-layer="0" from-port="0" to-layer="1" to-port="0" />
+		<edge from-layer="1" from-port="1" to-layer="6" to-port="0" />
+		<edge from-layer="2" from-port="0" to-layer="6" to-port="1" />
+		<edge from-layer="3" from-port="0" to-layer="6" to-port="2" />
+		<edge from-layer="4" from-port="0" to-layer="6" to-port="3" />
+		<edge from-layer="5" from-port="0" to-layer="6" to-port="4" />
+		<edge from-layer="6" from-port="5" to-layer="7" to-port="0" />
+		<edge from-layer="6" from-port="6" to-layer="7" to-port="1" />
+		<edge from-layer="6" from-port="7" to-layer="7" to-port="2" />
+		<edge from-layer="6" from-port="8" to-layer="7" to-port="3" />
+		<edge from-layer="6" from-port="9" to-layer="8" to-port="2" />
+		<edge from-layer="7" from-port="5" to-layer="8" to-port="1" />
+		<edge from-layer="7" from-port="4" to-layer="8" to-port="0" />
+		<edge from-layer="8" from-port="3" to-layer="11" to-port="0" />
+		<edge from-layer="8" from-port="4" to-layer="11" to-port="1" />
+		<edge from-layer="8" from-port="5" to-layer="11" to-port="2" />
+		<edge from-layer="9" from-port="0" to-layer="11" to-port="3" />
+		<edge from-layer="10" from-port="0" to-layer="11" to-port="4" />
+		<edge from-layer="11" from-port="5" to-layer="12" to-port="0" />
+		<edge from-layer="11" from-port="6" to-layer="12" to-port="1" />
+		<edge from-layer="11" from-port="7" to-layer="12" to-port="2" />
+		<edge from-layer="12" from-port="3" to-layer="13" to-port="0" />
+	</edges>
+	<rt_info>
+		<info name="OpenVINO Runtime" value="2026.4.0-22959-99c81491cc3-releases/2026/4" />
+		<add_attention_mask value="True" />
+		<add_prefix_space />
+		<add_special_tokens value="True" />
+		<bos_token_id value="128000" />
+		<clean_up_tokenization_spaces />
+		<detokenizer_input_type value="i64" />
+		<eos_token_id value="128001" />
+		<handle_special_tokens_with_re />
+		<max_length />
+		<number_of_inputs value="1" />
+		<openvino_tokenizers_version value="2026.4.0.0-736-a04accf6282" />
+		<openvino_version value="2026.4.0-22959-99c81491cc3-releases/2026/4" />
+		<original_post_processor_template value="{&quot;type&quot;: &quot;Sequence&quot;, &quot;processors&quot;: [{&quot;type&quot;: &quot;ByteLevel&quot;, &quot;add_prefix_space&quot;: true, &quot;trim_offsets&quot;: false, &quot;use_regex&quot;: true}, {&quot;type&quot;: &quot;TemplateProcessing&quot;, &quot;single&quot;: [{&quot;SpecialToken&quot;: {&quot;id&quot;: &quot;&lt;|begin_of_text|>&quot;, &quot;type_id&quot;: 0}}, {&quot;Sequence&quot;: {&quot;id&quot;: &quot;A&quot;, &quot;type_id&quot;: 0}}], &quot;pair&quot;: [{&quot;SpecialToken&quot;: {&quot;id&quot;: &quot;&lt;|begin_of_text|>&quot;, &quot;type_id&quot;: 0}}, {&quot;Sequence&quot;: {&quot;id&quot;: &quot;A&quot;, &quot;type_id&quot;: 0}}, {&quot;SpecialToken&quot;: {&quot;id&quot;: &quot;&lt;|begin_of_text|>&quot;, &quot;type_id&quot;: 1}}, {&quot;Sequence&quot;: {&quot;id&quot;: &quot;B&quot;, &quot;type_id&quot;: 1}}], &quot;special_tokens&quot;: {&quot;&lt;|begin_of_text|>&quot;: {&quot;id&quot;: &quot;&lt;|begin_of_text|>&quot;, &quot;ids&quot;: [128000], &quot;tokens&quot;: [&quot;&lt;|begin_of_text|>&quot;]}}}]}" />
+		<original_tokenizer_class value="&lt;class 'transformers.tokenization_utils_tokenizers.TokenizersBackend'>" />
+		<pad_token_id value="128001" />
+		<processed_post_processor_template value="{&quot;single&quot;: {&quot;ids&quot;: [128000, -1], &quot;type_ids&quot;: [0, 0]}, &quot;pair&quot;: {&quot;ids&quot;: [128000, -1, 128000, -2], &quot;type_ids&quot;: [0, 0, 1, 1]}}" />
+		<skip_special_tokens value="True" />
+		<streaming_detokenizer value="False" />
+		<tokenizer_output_type value="i64" />
+		<tokenizers_version value="0.22.2" />
+		<transformers_version value="5.5.4" />
+		<truncation value="False" />
+		<use_max_padding value="False" />
+		<use_sentencepiece_backend value="False" />
+		<utf8_replace_mode value="replace" />
+		<with_detokenizer value="True" />
+	</rt_info>
+</net>

openvino_model.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:c9feb7d66e0b698c79519b54a6281a787e59c87f08d46f4eb9c9202a13bf7779
+size 765067477

openvino_tokenizer.bin

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:e85dcd84b4ad36ae6104c68e063edb024f2ec32c3b74536324b5536df1f55cc8
+size 8396841

openvino_tokenizer.xml

@@ -0,0 +1,811 @@
+<?xml version="1.0"?>
+<net name="tokenizer" version="11">
+	<layers>
+		<layer id="0" name="Parameter_58258" type="Parameter" version="opset1">
+			<data shape="?" element_type="string" />
+			<output>
+				<port id="0" precision="STRING" names="Parameter_58258">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="1" name="Constant_58366" type="Const" version="opset1">
+			<data element_type="boolean" shape="" offset="0" size="1" />
+			<output>
+				<port id="0" precision="BOOL" />
+			</output>
+		</layer>
+		<layer id="2" name="ReadValue_58367" type="ReadValue" version="opset6">
+			<data variable_id="truncation" variable_type="dynamic" variable_shape="..." />
+			<input>
+				<port id="0" precision="BOOL" />
+			</input>
+			<output>
+				<port id="1" precision="BOOL" />
+			</output>
+		</layer>
+		<layer id="3" name="Constant_58377" type="Const" version="opset1">
+			<data element_type="i32" shape="" offset="1" size="4" />
+			<output>
+				<port id="0" precision="I32" />
+			</output>
+		</layer>
+		<layer id="4" name="Constant_58378" type="Const" version="opset1">
+			<data element_type="i32" shape="" offset="5" size="4" />
+			<output>
+				<port id="0" precision="I32" />
+			</output>
+		</layer>
+		<layer id="5" name="Constant_58379" type="Const" version="opset1">
+			<data element_type="i32" shape="1" offset="9" size="4" />
+			<output>
+				<port id="0" precision="I32">
+					<dim>1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="6" name="Constant_58264" type="Const" version="opset1">
+			<data element_type="i64" shape="" offset="13" size="8" />
+			<output>
+				<port id="0" precision="I64" />
+			</output>
+		</layer>
+		<layer id="7" name="StringTensorUnpack_58259" type="StringTensorUnpack" version="opset15">
+			<input>
+				<port id="0" precision="STRING">
+					<dim>-1</dim>
+				</port>
+			</input>
+			<output>
+				<port id="1" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="2" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="3" precision="U8">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="8" name="ShapeOf_58260" type="ShapeOf" version="opset3">
+			<data output_type="i64" />
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+				</port>
+			</input>
+			<output>
+				<port id="1" precision="I64">
+					<dim>1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="9" name="Constant_58261" type="Const" version="opset1">
+			<data element_type="i64" shape="" offset="13" size="8" />
+			<output>
+				<port id="0" precision="I64" />
+			</output>
+		</layer>
+		<layer id="10" name="Constant_58262" type="Const" version="opset1">
+			<data element_type="i64" shape="" offset="13" size="8" />
+			<output>
+				<port id="0" precision="I64" />
+			</output>
+		</layer>
+		<layer id="11" name="Gather_58263" type="Gather" version="opset8">
+			<data batch_dims="0" />
+			<input>
+				<port id="0" precision="I64">
+					<dim>1</dim>
+				</port>
+				<port id="1" precision="I64" />
+				<port id="2" precision="I64" />
+			</input>
+			<output>
+				<port id="3" precision="I64" />
+			</output>
+		</layer>
+		<layer id="12" name="Constant_58265" type="Const" version="opset1">
+			<data element_type="i64" shape="" offset="21" size="8" />
+			<output>
+				<port id="0" precision="I64" />
+			</output>
+		</layer>
+		<layer id="13" name="Range_58266" type="Range" version="opset4">
+			<data output_type="i32" />
+			<input>
+				<port id="0" precision="I64" />
+				<port id="1" precision="I64" />
+				<port id="2" precision="I64" />
+			</input>
+			<output>
+				<port id="3" precision="I32">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="14" name="Constant_58267" type="Const" version="opset1">
+			<data element_type="i64" shape="" offset="21" size="8" />
+			<output>
+				<port id="0" precision="I64" />
+			</output>
+		</layer>
+		<layer id="15" name="Constant_58268" type="Const" version="opset1">
+			<data element_type="i64" shape="" offset="21" size="8" />
+			<output>
+				<port id="0" precision="I64" />
+			</output>
+		</layer>
+		<layer id="16" name="Add_58269" type="Add" version="opset1">
+			<data auto_broadcast="numpy" />
+			<input>
+				<port id="0" precision="I64" />
+				<port id="1" precision="I64" />
+			</input>
+			<output>
+				<port id="2" precision="I64" />
+			</output>
+		</layer>
+		<layer id="17" name="Constant_58270" type="Const" version="opset1">
+			<data element_type="i64" shape="" offset="21" size="8" />
+			<output>
+				<port id="0" precision="I64" />
+			</output>
+		</layer>
+		<layer id="18" name="Range_58271" type="Range" version="opset4">
+			<data output_type="i32" />
+			<input>
+				<port id="0" precision="I64" />
+				<port id="1" precision="I64" />
+				<port id="2" precision="I64" />
+			</input>
+			<output>
+				<port id="3" precision="I32">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="19" name="Constant_58335" type="Const" version="opset1">
+			<data element_type="u8" shape="8738" offset="29" size="8738" />
+			<output>
+				<port id="0" precision="U8">
+					<dim>8738</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="20" name="SpecialTokensSplit_58336" type="SpecialTokensSplit" version="extension">
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="1" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="2" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="3" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="4" precision="U8">
+					<dim>-1</dim>
+				</port>
+				<port id="5" precision="U8">
+					<dim>8738</dim>
+				</port>
+			</input>
+			<output>
+				<port id="6" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="7" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="8" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="9" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="10" precision="U8">
+					<dim>-1</dim>
+				</port>
+				<port id="11" precision="BOOL">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="21" name="Constant_58338" type="Const" version="opset1">
+			<data element_type="u8" shape="115" offset="8767" size="115" />
+			<output>
+				<port id="0" precision="U8">
+					<dim>115</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="22" name="RegexSplit_58339" type="RegexSplit" version="extension">
+			<data behaviour="isolate" invert="false" max_splits="-1" />
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="1" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="2" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="3" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="4" precision="U8">
+					<dim>-1</dim>
+				</port>
+				<port id="5" precision="BOOL">
+					<dim>-1</dim>
+				</port>
+				<port id="6" precision="U8">
+					<dim>115</dim>
+				</port>
+			</input>
+			<output>
+				<port id="7" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="8" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="9" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="10" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="11" precision="U8">
+					<dim>-1</dim>
+				</port>
+				<port id="12" precision="BOOL">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="23" name="Constant_58341" type="Const" version="opset1">
+			<data element_type="i32" shape="128256" offset="8882" size="513024" />
+			<output>
+				<port id="0" precision="I32">
+					<dim>128256</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="24" name="Constant_58343" type="Const" version="opset1">
+			<data element_type="i32" shape="128256" offset="521906" size="513024" />
+			<output>
+				<port id="0" precision="I32">
+					<dim>128256</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="25" name="Constant_58345" type="Const" version="opset1">
+			<data element_type="u8" shape="838768" offset="1034930" size="838768" />
+			<output>
+				<port id="0" precision="U8">
+					<dim>838768</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="26" name="Constant_58353" type="Const" version="opset1">
+			<data element_type="i32" shape="280147" offset="1873698" size="1120588" />
+			<output>
+				<port id="0" precision="I32">
+					<dim>280147</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="27" name="Constant_58355" type="Const" version="opset1">
+			<data element_type="i32" shape="280147" offset="2994286" size="1120588" />
+			<output>
+				<port id="0" precision="I32">
+					<dim>280147</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="28" name="Constant_58357" type="Const" version="opset1">
+			<data element_type="u8" shape="993515" offset="4114874" size="993515" />
+			<output>
+				<port id="0" precision="U8">
+					<dim>993515</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="29" name="Constant_58359" type="Const" version="opset1">
+			<data element_type="i32" shape="280147" offset="5108389" size="1120588" />
+			<output>
+				<port id="0" precision="I32">
+					<dim>280147</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="30" name="Constant_58361" type="Const" version="opset1">
+			<data element_type="i32" shape="280147" offset="6228977" size="1120588" />
+			<output>
+				<port id="0" precision="I32">
+					<dim>280147</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="31" name="Constant_58363" type="Const" version="opset1">
+			<data element_type="u8" shape="1036718" offset="7349565" size="1036718" />
+			<output>
+				<port id="0" precision="U8">
+					<dim>1036718</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="32" name="Constant_58347" type="Const" version="opset1">
+			<data element_type="i32" shape="256" offset="8386283" size="1024" />
+			<output>
+				<port id="0" precision="I32">
+					<dim>256</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="33" name="Constant_58349" type="Const" version="opset1">
+			<data element_type="i32" shape="256" offset="8387307" size="1024" />
+			<output>
+				<port id="0" precision="I32">
+					<dim>256</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="34" name="Constant_58351" type="Const" version="opset1">
+			<data element_type="u8" shape="7457" offset="8388331" size="7457" />
+			<output>
+				<port id="0" precision="U8">
+					<dim>7457</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="35" name="Constant_58364" type="Const" version="opset1">
+			<data element_type="i32" shape="256" offset="8395788" size="1024" />
+			<output>
+				<port id="0" precision="I32">
+					<dim>256</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="36" name="BPETokenizer_58365" type="BPETokenizer" version="extension">
+			<data unk_token="" fuse_unk="false" suffix_indicator="" end_suffix="" byte_fallback="false" cache_capacity="25600" />
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="1" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="2" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="3" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="4" precision="U8">
+					<dim>-1</dim>
+				</port>
+				<port id="5" precision="I32">
+					<dim>128256</dim>
+				</port>
+				<port id="6" precision="I32">
+					<dim>128256</dim>
+				</port>
+				<port id="7" precision="U8">
+					<dim>838768</dim>
+				</port>
+				<port id="8" precision="I32">
+					<dim>280147</dim>
+				</port>
+				<port id="9" precision="I32">
+					<dim>280147</dim>
+				</port>
+				<port id="10" precision="U8">
+					<dim>993515</dim>
+				</port>
+				<port id="11" precision="I32">
+					<dim>280147</dim>
+				</port>
+				<port id="12" precision="I32">
+					<dim>280147</dim>
+				</port>
+				<port id="13" precision="U8">
+					<dim>1036718</dim>
+				</port>
+				<port id="14" precision="I32">
+					<dim>256</dim>
+				</port>
+				<port id="15" precision="I32">
+					<dim>256</dim>
+				</port>
+				<port id="16" precision="U8">
+					<dim>7457</dim>
+				</port>
+				<port id="17" precision="I32">
+					<dim>256</dim>
+				</port>
+			</input>
+			<output>
+				<port id="18" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="19" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="20" precision="I32">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="37" name="Constant_58369" type="Const" version="opset1">
+			<data element_type="i32" shape="" offset="8396812" size="4" />
+			<output>
+				<port id="0" precision="I32" />
+			</output>
+		</layer>
+		<layer id="38" name="Constant_58370" type="Const" version="opset1">
+			<data element_type="i32" shape="" offset="8396816" size="4" />
+			<output>
+				<port id="0" precision="I32" />
+			</output>
+		</layer>
+		<layer id="39" name="Select_58371" type="Select" version="opset1">
+			<data auto_broadcast="numpy" />
+			<input>
+				<port id="0" precision="BOOL" />
+				<port id="1" precision="I32" />
+				<port id="2" precision="I32" />
+			</input>
+			<output>
+				<port id="3" precision="I32" />
+			</output>
+		</layer>
+		<layer id="40" name="Constant_58373" type="Const" version="opset1">
+			<data element_type="u8" shape="4" offset="8396820" size="4" />
+			<output>
+				<port id="0" precision="U8">
+					<dim>4</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="41" name="Constant_58375" type="Const" version="opset1">
+			<data element_type="u8" shape="13" offset="8396824" size="13" />
+			<output>
+				<port id="0" precision="U8">
+					<dim>13</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="42" name="Truncate_58376" type="Truncate" version="extension">
+			<data m_num_inputs="1" />
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="1" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="2" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="3" precision="I32" />
+				<port id="4" precision="U8">
+					<dim>4</dim>
+				</port>
+				<port id="5" precision="U8">
+					<dim>13</dim>
+				</port>
+			</input>
+			<output>
+				<port id="6" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="7" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="8" precision="I32">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="43" name="Constant_58380" type="Const" version="opset1">
+			<data element_type="i32" shape="2" offset="13" size="8" />
+			<output>
+				<port id="0" precision="I32">
+					<dim>2</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="44" name="CombineSegments_58381" type="CombineSegments" version="extension">
+			<input>
+				<port id="0" precision="I32" />
+				<port id="1" precision="I32" />
+				<port id="2" precision="I32">
+					<dim>1</dim>
+				</port>
+				<port id="3" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="4" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="5" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="6" precision="I32">
+					<dim>2</dim>
+				</port>
+			</input>
+			<output>
+				<port id="7" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="8" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="9" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="10" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="11" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="12" precision="I32">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="45" name="Subtract_58382" type="Subtract" version="opset1">
+			<data auto_broadcast="numpy" />
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="1" precision="I32">
+					<dim>-1</dim>
+				</port>
+			</input>
+			<output>
+				<port id="2" precision="I32">
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="46" name="Constant_58383" type="Const" version="opset1">
+			<data element_type="i32" shape="" offset="1" size="4" />
+			<output>
+				<port id="0" precision="I32" />
+			</output>
+		</layer>
+		<layer id="47" name="ReduceMax_58384" type="ReduceMax" version="opset1">
+			<data keep_dims="false" />
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="1" precision="I32" />
+			</input>
+			<output>
+				<port id="2" precision="I32" />
+			</output>
+		</layer>
+		<layer id="48" name="Constant_58385" type="Const" version="opset1">
+			<data element_type="i32" shape="" offset="8396837" size="4" />
+			<output>
+				<port id="0" precision="I32" />
+			</output>
+		</layer>
+		<layer id="49" name="RaggedToDense_58386" type="RaggedToDense" version="extension">
+			<data pad_right="false" m_pad_max_length="false" />
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="1" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="2" precision="I32">
+					<dim>-1</dim>
+				</port>
+				<port id="3" precision="I32" />
+				<port id="4" precision="I32" />
+			</input>
+			<output>
+				<port id="5" precision="I32">
+					<dim>-1</dim>
+					<dim>-1</dim>
+				</port>
+				<port id="6" precision="BOOL">
+					<dim>-1</dim>
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="50" name="Convert_58387" type="Convert" version="opset1">
+			<data destination_type="i32" />
+			<input>
+				<port id="0" precision="BOOL">
+					<dim>-1</dim>
+					<dim>-1</dim>
+				</port>
+			</input>
+			<output>
+				<port id="1" precision="I32">
+					<dim>-1</dim>
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="51" name="Convert_58387.0" type="Convert" version="opset1">
+			<data destination_type="i64" />
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+					<dim>-1</dim>
+				</port>
+			</input>
+			<output>
+				<port id="1" precision="I64" names="attention_mask">
+					<dim>-1</dim>
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="52" name="RaggedToDense_58386.0" type="Convert" version="opset1">
+			<data destination_type="i64" />
+			<input>
+				<port id="0" precision="I32">
+					<dim>-1</dim>
+					<dim>-1</dim>
+				</port>
+			</input>
+			<output>
+				<port id="1" precision="I64" names="input_ids">
+					<dim>-1</dim>
+					<dim>-1</dim>
+				</port>
+			</output>
+		</layer>
+		<layer id="53" name="Assign_58368" type="Assign" version="opset6">
+			<data variable_id="truncation" />
+			<input>
+				<port id="0" precision="BOOL" />
+			</input>
+			<output>
+				<port id="1" precision="BOOL" />
+			</output>
+		</layer>
+		<layer id="54" name="Result_58390" type="Result" version="opset1" output_names="input_ids">
+			<input>
+				<port id="0" precision="I64">
+					<dim>-1</dim>
+					<dim>-1</dim>
+				</port>
+			</input>
+		</layer>
+		<layer id="55" name="Result_58392" type="Result" version="opset1" output_names="attention_mask">
+			<input>
+				<port id="0" precision="I64">
+					<dim>-1</dim>
+					<dim>-1</dim>
+				</port>
+			</input>
+		</layer>
+	</layers>
+	<edges>
+		<edge from-layer="0" from-port="0" to-layer="7" to-port="0" />
+		<edge from-layer="1" from-port="0" to-layer="2" to-port="0" />
+		<edge from-layer="2" from-port="1" to-layer="39" to-port="0" />
+		<edge from-layer="2" from-port="1" to-layer="53" to-port="0" />
+		<edge from-layer="3" from-port="0" to-layer="44" to-port="0" />
+		<edge from-layer="4" from-port="0" to-layer="44" to-port="1" />
+		<edge from-layer="5" from-port="0" to-layer="44" to-port="2" />
+		<edge from-layer="6" from-port="0" to-layer="13" to-port="0" />
+		<edge from-layer="7" from-port="1" to-layer="8" to-port="0" />
+		<edge from-layer="7" from-port="3" to-layer="20" to-port="4" />
+		<edge from-layer="7" from-port="2" to-layer="20" to-port="3" />
+		<edge from-layer="7" from-port="1" to-layer="20" to-port="2" />
+		<edge from-layer="8" from-port="1" to-layer="11" to-port="0" />
+		<edge from-layer="9" from-port="0" to-layer="11" to-port="1" />
+		<edge from-layer="10" from-port="0" to-layer="11" to-port="2" />
+		<edge from-layer="11" from-port="3" to-layer="16" to-port="0" />
+		<edge from-layer="11" from-port="3" to-layer="13" to-port="1" />
+		<edge from-layer="12" from-port="0" to-layer="13" to-port="2" />
+		<edge from-layer="13" from-port="3" to-layer="20" to-port="0" />
+		<edge from-layer="14" from-port="0" to-layer="18" to-port="0" />
+		<edge from-layer="15" from-port="0" to-layer="16" to-port="1" />
+		<edge from-layer="16" from-port="2" to-layer="18" to-port="1" />
+		<edge from-layer="17" from-port="0" to-layer="18" to-port="2" />
+		<edge from-layer="18" from-port="3" to-layer="20" to-port="1" />
+		<edge from-layer="19" from-port="0" to-layer="20" to-port="5" />
+		<edge from-layer="20" from-port="10" to-layer="22" to-port="4" />
+		<edge from-layer="20" from-port="6" to-layer="22" to-port="0" />
+		<edge from-layer="20" from-port="7" to-layer="22" to-port="1" />
+		<edge from-layer="20" from-port="11" to-layer="22" to-port="5" />
+		<edge from-layer="20" from-port="9" to-layer="22" to-port="3" />
+		<edge from-layer="20" from-port="8" to-layer="22" to-port="2" />
+		<edge from-layer="21" from-port="0" to-layer="22" to-port="6" />
+		<edge from-layer="22" from-port="7" to-layer="36" to-port="0" />
+		<edge from-layer="22" from-port="8" to-layer="36" to-port="1" />
+		<edge from-layer="22" from-port="9" to-layer="36" to-port="2" />
+		<edge from-layer="22" from-port="10" to-layer="36" to-port="3" />
+		<edge from-layer="22" from-port="11" to-layer="36" to-port="4" />
+		<edge from-layer="23" from-port="0" to-layer="36" to-port="5" />
+		<edge from-layer="24" from-port="0" to-layer="36" to-port="6" />
+		<edge from-layer="25" from-port="0" to-layer="36" to-port="7" />
+		<edge from-layer="26" from-port="0" to-layer="36" to-port="8" />
+		<edge from-layer="27" from-port="0" to-layer="36" to-port="9" />
+		<edge from-layer="28" from-port="0" to-layer="36" to-port="10" />
+		<edge from-layer="29" from-port="0" to-layer="36" to-port="11" />
+		<edge from-layer="30" from-port="0" to-layer="36" to-port="12" />
+		<edge from-layer="31" from-port="0" to-layer="36" to-port="13" />
+		<edge from-layer="32" from-port="0" to-layer="36" to-port="14" />
+		<edge from-layer="33" from-port="0" to-layer="36" to-port="15" />
+		<edge from-layer="34" from-port="0" to-layer="36" to-port="16" />
+		<edge from-layer="35" from-port="0" to-layer="36" to-port="17" />
+		<edge from-layer="36" from-port="19" to-layer="42" to-port="1" />
+		<edge from-layer="36" from-port="20" to-layer="42" to-port="2" />
+		<edge from-layer="36" from-port="18" to-layer="42" to-port="0" />
+		<edge from-layer="37" from-port="0" to-layer="39" to-port="1" />
+		<edge from-layer="38" from-port="0" to-layer="39" to-port="2" />
+		<edge from-layer="39" from-port="3" to-layer="42" to-port="3" />
+		<edge from-layer="40" from-port="0" to-layer="42" to-port="4" />
+		<edge from-layer="41" from-port="0" to-layer="42" to-port="5" />
+		<edge from-layer="42" from-port="6" to-layer="44" to-port="3" />
+		<edge from-layer="42" from-port="7" to-layer="44" to-port="4" />
+		<edge from-layer="42" from-port="8" to-layer="44" to-port="5" />
+		<edge from-layer="43" from-port="0" to-layer="44" to-port="6" />
+		<edge from-layer="44" from-port="7" to-layer="45" to-port="1" />
+		<edge from-layer="44" from-port="7" to-layer="49" to-port="0" />
+		<edge from-layer="44" from-port="8" to-layer="49" to-port="1" />
+		<edge from-layer="44" from-port="9" to-layer="49" to-port="2" />
+		<edge from-layer="44" from-port="8" to-layer="45" to-port="0" />
+		<edge from-layer="45" from-port="2" to-layer="47" to-port="0" />
+		<edge from-layer="46" from-port="0" to-layer="47" to-port="1" />
+		<edge from-layer="47" from-port="2" to-layer="49" to-port="3" />
+		<edge from-layer="48" from-port="0" to-layer="49" to-port="4" />
+		<edge from-layer="49" from-port="6" to-layer="50" to-port="0" />
+		<edge from-layer="49" from-port="5" to-layer="52" to-port="0" />
+		<edge from-layer="50" from-port="1" to-layer="51" to-port="0" />
+		<edge from-layer="51" from-port="1" to-layer="55" to-port="0" />
+		<edge from-layer="52" from-port="1" to-layer="54" to-port="0" />
+	</edges>
+	<rt_info>
+		<info name="OpenVINO Runtime" value="2026.4.0-22959-99c81491cc3-releases/2026/4" />
+		<add_attention_mask value="True" />
+		<add_prefix_space />
+		<add_special_tokens value="True" />
+		<bos_token_id value="128000" />
+		<clean_up_tokenization_spaces />
+		<detokenizer_input_type value="i64" />
+		<eos_token_id value="128001" />
+		<handle_special_tokens_with_re />
+		<max_length />
+		<number_of_inputs value="1" />
+		<openvino_tokenizers_version value="2026.4.0.0-736-a04accf6282" />
+		<openvino_version value="2026.4.0-22959-99c81491cc3-releases/2026/4" />
+		<original_post_processor_template value="{&quot;type&quot;: &quot;Sequence&quot;, &quot;processors&quot;: [{&quot;type&quot;: &quot;ByteLevel&quot;, &quot;add_prefix_space&quot;: true, &quot;trim_offsets&quot;: false, &quot;use_regex&quot;: true}, {&quot;type&quot;: &quot;TemplateProcessing&quot;, &quot;single&quot;: [{&quot;SpecialToken&quot;: {&quot;id&quot;: &quot;&lt;|begin_of_text|>&quot;, &quot;type_id&quot;: 0}}, {&quot;Sequence&quot;: {&quot;id&quot;: &quot;A&quot;, &quot;type_id&quot;: 0}}], &quot;pair&quot;: [{&quot;SpecialToken&quot;: {&quot;id&quot;: &quot;&lt;|begin_of_text|>&quot;, &quot;type_id&quot;: 0}}, {&quot;Sequence&quot;: {&quot;id&quot;: &quot;A&quot;, &quot;type_id&quot;: 0}}, {&quot;SpecialToken&quot;: {&quot;id&quot;: &quot;&lt;|begin_of_text|>&quot;, &quot;type_id&quot;: 1}}, {&quot;Sequence&quot;: {&quot;id&quot;: &quot;B&quot;, &quot;type_id&quot;: 1}}], &quot;special_tokens&quot;: {&quot;&lt;|begin_of_text|>&quot;: {&quot;id&quot;: &quot;&lt;|begin_of_text|>&quot;, &quot;ids&quot;: [128000], &quot;tokens&quot;: [&quot;&lt;|begin_of_text|>&quot;]}}}]}" />
+		<original_tokenizer_class value="&lt;class 'transformers.tokenization_utils_tokenizers.TokenizersBackend'>" />
+		<pad_token_id value="128001" />
+		<processed_post_processor_template value="{&quot;single&quot;: {&quot;ids&quot;: [128000, -1], &quot;type_ids&quot;: [0, 0]}, &quot;pair&quot;: {&quot;ids&quot;: [128000, -1, 128000, -2], &quot;type_ids&quot;: [0, 0, 1, 1]}}" />
+		<skip_special_tokens value="True" />
+		<streaming_detokenizer value="False" />
+		<tokenizer_output_type value="i64" />
+		<tokenizers_version value="0.22.2" />
+		<transformers_version value="5.5.4" />
+		<truncation value="False" />
+		<use_max_padding value="False" />
+		<use_sentencepiece_backend value="False" />
+		<utf8_replace_mode value="replace" />
+		<with_detokenizer value="True" />
+	</rt_info>
+</net>

tokenizer.json

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:6b9e4e7fb171f92fd137b777cc2714bf87d11576700a1dcd7a399e7bbe39537b
+size 17209920

tokenizer_config.json

@@ -0,0 +1,14 @@
+{
+  "backend": "tokenizers",
+  "bos_token": "<|begin_of_text|>",
+  "clean_up_tokenization_spaces": true,
+  "eos_token": "<|end_of_text|>",
+  "is_local": true,
+  "model_input_names": [
+    "input_ids",
+    "attention_mask"
+  ],
+  "model_max_length": 131072,
+  "pad_token": "<|end_of_text|>",
+  "tokenizer_class": "TokenizersBackend"
+}