| library_name: onnx | |
| tags: | |
| - text-generation | |
| - phi-4 | |
| - onnx | |
| - int4 | |
| - cpu | |
| - onnxruntime-genai | |
| - inference4j | |
| - onnx | |
| - inference4j | |
| license: mit | |
| pipeline_tag: text-generation | |
| # Phi-4-mini-instruct — ONNX (INT4) | |
| INT4-quantized ONNX export of [Phi-4-mini-instruct](https://huggingface.co/microsoft/Phi-4-mini-instruct), a 3.8B-parameter lightweight language model from Microsoft. Optimized for CPU inference with INT4 RTN block-32 quantization in onnxruntime-genai format. | |
| Mirrored for use with [inference4j](https://github.com/inference4j/inference4j), an inference-only AI library for Java. | |
| ## Original Source | |
| - **Repository:** [Microsoft](https://huggingface.co/microsoft/Phi-4-mini-instruct-onnx) | |
| - **License:** mit | |
| ## Usage with inference4j | |
| ```java | |
| try (TextGenerator gen = TextGenerator.builder() | |
| .model(ModelSources.phi4Mini()) | |
| .build()) { | |
| GenerationResult result = gen.generate("What is Java in one sentence?"); | |
| System.out.println(result.text()); | |
| } | |
| ``` | |
| ## Model Details | |
| | Property | Value | | |
| |----------|-------| | |
| | Architecture | Phi-4-mini (3.8B parameters, 32 layers, 3072 hidden) | | |
| | Task | Text generation / chat | | |
| | Context length | 128k tokens | | |
| | Quantization | INT4 RTN block-32 acc-level-4 | | |
| | Original framework | PyTorch (transformers) | | |
| ## License | |
| This model is licensed under the [MIT License](https://opensource.org/licenses/MIT). Original model and ONNX conversion by [Microsoft](https://huggingface.co/microsoft). | |