add gpu optimized wi4b32_wi8_afp32 model

README.md

@@ -24,23 +24,24 @@ This repository hosts the [**LiteRT-LM**](https://ai.google.dev/edge/litert-lm)
 
 ## Available Models
 
+* **`minicpm_dynamic_wi8_afp32_gpu_opt.litertlm`**: This model features dynamic weight-only INT8 quantization (wi8) with FP32 activations (afp32), heavily optimized for GPU execution.
 * **`minicpm_wi4b32_wi8_afp32.litertlm`**: This model features mixed INT4-block32(linear)/INT8(embed and lmhead) quantization (wi4b32_wi8) with FP32 activations (afp32).
-* **`MiniCPM5-1B_dynamic_wi8_afp32.litertlm`**: This model features dynamic weight-only INT8 quantization (wi8) with FP32 activations (afp32). It is configured with static prefill memory allocation.
+* **`minicpm_wi4b32_wi8_afp32_gpu_opt.litertlm`**: This model uses same quantization recipe with minicpm_wi4b32_wi8_afp32 but optimized for GPU execution.
 
 ## Quantization Benchmark
 
-The table below compares the accuracy of the FP baseline and the LiteRT-LM quantized model on the same evaluation benchmark. Scores are reported as percentages.
+The table below compares the accuracy of the LiteRT-LM quantized models on the same evaluation benchmark. Scores are reported as percentages.
 
-| Category | FP (bf16) | W4 |
+| Category | W8 (`minicpm_dynamic_wi8_afp32_gpu_opt.litertlm`) | W4 (`minicpm_wi4b32_wi8_afp32.litertlm`) |
 | --- | --- | --- |
-| Overall | 66.16 | 60.91 |
-| STEM | 58.07 | 53.36 |
-| Social Science | 77.91 | 70.87 |
-| Humanities | 67.65 | 64.35 |
-| Other | 68.71 | 62.14 |
-| Hard | 47.08 | 44.23 |
+| Overall | 63.78 | 60.91 |
+| STEM | 56.35 | 53.36 |
+| Social Science | 73.65 | 70.87 |
+| Humanities | 66.68 | 64.35 |
+| Other | 65.40 | 62.14 |
+| Hard | 43.79 | 44.23 |
 
-FP (bf16) is the full-precision baseline. W4 uses mixed INT4-block32 (linear) / INT8 (embed and lmhead) quantization (`wi4b32_wi8`) with FP32 activations (`afp32`). Thinking mode has been turned off to reduce context usage. All evaluations use a temperature of 0.6, top-p of 0.95, and a random seed of 0.
+W8 uses dynamic weight-only INT8 quantization (`wi8`) with FP32 activations (`afp32`). W4 uses mixed INT4-block32 (linear) / INT8 (embed and lmhead) quantization (`wi4b32_wi8`) with FP32 activations (`afp32`). Thinking mode has been turned off to reduce context usage.
 
 ## What is MiniCPM?
 
@@ -74,7 +75,7 @@ FP (bf16) is the full-precision baseline. W4 uses mixed INT4-block32 (linear) /
 1. **Get the App**: Install the [app](https://play.google.com/store/apps/details?id=com.google.ai.edge.gallery&pli=1) from Google Play or download the latest APK from the [GitHub releases page](https://github.com/google-ai-edge/gallery/releases).
 2. **Importing the Model**: Navigate to the **Model manager** within the app and click the **"+" (plus)** icon in the bottom-right corner. Two options will appear:
    * **Import from HF (Recommended)**: Select this option, and a dialog box will appear showing an example Hugging Face model URL. Enter the HF link for the desired `.litertlm` model and click submit. The model will then appear in your list, and you can proceed to download it (a Hugging Face account login is required).
-   * **From local model file**: First, download the `.litertlm` model directly to your Android device, OR download it to your computer and push it via ADB (e.g., `adb push MiniCPM5-1B_dynamic_wi8_afp32.litertlm /sdcard/Download/`). Then, select this option, choose the downloaded file from your storage, configure your preferred parameters, and tap **"Import"**.
+   * **From local model file**: First, download the `.litertlm` model directly to your Android device, OR download it to your computer and push it via ADB (e.g., `adb push minicpm_dynamic_wi8_afp32_gpu_opt.litertlm /sdcard/Download/`). Then, select this option, choose the downloaded file from your storage, configure your preferred parameters, and tap **"Import"**.
 
 For full details on importing models and other features, see the [Edge Gallery App Wiki](https://github.com/google-ai-edge/gallery/wiki).
 
@@ -86,7 +87,7 @@ Install `uv` and run the model directly from the LiteRT-LM command line:
 
 ```bash
 uv tool install litert-lm
-uvx litert-lm run --from-huggingface-repo=litert-community/MiniCPM5-1B MiniCPM5-1B_dynamic_wi8_afp32.litertlm --prompt="What is the capital of France?"
+uvx litert-lm run --from-huggingface-repo=litert-community/MiniCPM5-1B minicpm_dynamic_wi8_afp32_gpu_opt.litertlm --prompt="What is the capital of France?"
 ```
 
 ## Links

minicpm_wi4b32_wi8_afp32_gpu_opt.litertlm

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:4e15a7cc735ae36e9888d341d44bd53c0579153d7a3379aaf958620eea4816e3
+size 793034752