README.md

---
library_name: onnx
tags:
  - text-generation
  - phi-4
  - onnx
  - int4
  - cpu
  - onnxruntime-genai
  - inference4j
  - onnx
  - inference4j
license: mit
pipeline_tag: text-generation
---

# Phi-4-mini-instruct — ONNX (INT4)

INT4-quantized ONNX export of [Phi-4-mini-instruct](https://huggingface.co/microsoft/Phi-4-mini-instruct), a 3.8B-parameter lightweight language model from Microsoft. Optimized for CPU inference with INT4 RTN block-32 quantization in onnxruntime-genai format.

Mirrored for use with [inference4j](https://github.com/inference4j/inference4j), an inference-only AI library for Java.

## Original Source

- **Repository:** [Microsoft](https://huggingface.co/microsoft/Phi-4-mini-instruct-onnx)
- **License:** mit

## Usage with inference4j

```java
try (TextGenerator gen = TextGenerator.builder()
        .model(ModelSources.phi4Mini())
        .build()) {
    GenerationResult result = gen.generate("What is Java in one sentence?");
    System.out.println(result.text());
}
```

## Model Details

| Property | Value |
|----------|-------|
| Architecture | Phi-4-mini (3.8B parameters, 32 layers, 3072 hidden) |
| Task | Text generation / chat |
| Context length | 128k tokens |
| Quantization | INT4 RTN block-32 acc-level-4 |
| Original framework | PyTorch (transformers) |

## License

This model is licensed under the [MIT License](https://opensource.org/licenses/MIT). Original model and ONNX conversion by [Microsoft](https://huggingface.co/microsoft).