Sync model repo (text/metadata)

README.md

@@ -1,10 +1,11 @@
 ---
 library_name: executorch
-display_name: Gemma-4-E2B 8da4w — ExecuTorch
+display_name: Gemma-4-E2B-IT 8da4w+emb4 — ExecuTorch
 license: apache-2.0
 tags:
   - text-generation
   - 8da4w
+  - emb4
   - quantized
   - xnnpack
   - kleidiai
@@ -12,13 +13,13 @@ tags:
   - executorch
   - edge-ai
 pipeline_tag: text-generation
-base_model: google/gemma-4-E2B
+base_model: google/gemma-4-E2B-it
 base_model_relation: quantized
 ---
 
-# Gemma-4-E2B 8da4w (ExecuTorch + XNNPACK)
-This is an **8da4w-quantized** (8-bit dynamic per-token activations + 4-bit per-channel grouped weights) version of [google/gemma-4-E2B](https://huggingface.co/google/gemma-4-E2B), optimized for edge deployment on ARM devices using [ExecuTorch](https://github.com/pytorch/executorch) with the **XNNPACK + KleidiAI** backend. 
-The model was quantized by ExecuTorch’s Gemma 4 export script without SDPA, with a custom static KV-cache, and exported to the `.pte` format for efficient on-device inference on ARM Cortex-A processors (modern Android phones, AWS Graviton, embedded ARM).
+# Gemma-4-E2B-IT 8da4w+emb4 (ExecuTorch + XNNPACK)
+This is an **8da4w+emb4-quantized** (8-bit dynamic per-token activations + 4-bit per-channel grouped linear weights + packed 4-bit embedding weights) version of [google/gemma-4-E2B-it](https://huggingface.co/google/gemma-4-E2B-it), optimized for edge deployment on ARM devices using [ExecuTorch](https://github.com/pytorch/executorch) with the **XNNPACK + KleidiAI** backend. 
+The model was quantized by ExecuTorch’s Gemma 4 export script with custom fused SDPA and a static KV cache, then exported to the `.pte` format for efficient on-device inference on ARM Cortex-A processors (modern Android phones, AWS Graviton, embedded ARM).
 
 ## Key Highlights
 
@@ -30,8 +31,8 @@ Compared to the FP32 baseline:
 
 ### Model Description
 
-Gemma 4 E2B-IT is an instruction-tuned multimodal generative model from Google DeepMind. This text-only artifact is the same model after 8da4w post-training quantization and ExecuTorch/XNNPACK export, no fine-tuning was performed.
-Linear weights use groupwise INT4 quantization, activations are dynamically quantized to INT8, and embeddings use INT8.
+Gemma 4 E2B-IT is an instruction-tuned multimodal generative model from Google DeepMind. This text-only artifact is the same model after 8da4w+emb4 post-training quantization and ExecuTorch/XNNPACK export; no fine-tuning was performed.
+Linear weights use groupwise INT4 quantization, activations are dynamically quantized to INT8, and embeddings use packed INT4 weights.
 
 - **Developed by:** Google DeepMind (base model); Arm Model Optimization pipeline (quantization and export)
 - **Model type:** Text generation (autoregressive language model)
@@ -40,7 +41,7 @@ Linear weights use groupwise INT4 quantization, activations are dynamically quan
 
 ### Model Sources
 
-- **Repository:** https://huggingface.co/google/gemma-4-E2B
+- **Repository:** https://huggingface.co/google/gemma-4-E2B-it
 - **Paper:** [Gemma 4 Technical Report](https://arxiv.org/abs/2607.02770)
 
 ## How to Get Started with the Model
@@ -61,35 +62,34 @@ Linear weights use groupwise INT4 quantization, activations are dynamically quan
 
 ### Objective
 
-Open-ended text generation and multiple-choice scoring with the Gemma 4 E2B architecture, deployed on ARM CPUs using INT4 linear weights and INT8 activations.
+Open-ended text generation and multiple-choice scoring with the instruction-tuned Gemma 4 E2B architecture, deployed on ARM CPUs using INT4 linear weights and INT8 activations.
 
 ### Quantization
 
 - **Activations**: 8-bit dynamic, per-token quantization
-- **Linear weights**: 4-bit, per-channel grouped quantization with group size 32
-- **Token embedding**: 8-bit, per-channel weight quantization
+- **Linear weights**: 4-bit grouped quantization with group size 128
+- **Token embedding**: packed 4-bit weight quantization
 - **Backend**: XNNPACK with KleidiAI INT4 micro-kernels
-- **Higher-precision layers**: None. Every linear layer and the token embedding are quantized. Attention internals remain in higher precision within the custom fused SDPA path.
-- **Calibration**: 200 prompts randomly sampled from the MMLU test split
+- **Higher-precision operations**: Custom fused SDPA keeps attention internals in floating point. Linear layers with shapes incompatible with grouped INT4 quantization, if any, are retained at their original precision.
+- **Calibration**: None. Dynamic activation quantization does not use a calibration dataset or observer-calibration pass.
 
 ### Export Pipeline
 
 1. Load the pretrained Gemma 4 E2B-IT weights from Hugging Face safetensors and convert them into ExecuTorch’s custom Gemma 4 text-decoder architecture.
 2. Configure a static KV cache with a maximum sequence length of 1,024 tokens and enable dynamically sized input sequences.
-3. Replace standard attention with the fused llama::custom_sdpa tiled-attention operator.
+3. Replace standard attention with the fused `llama::custom_sdpa` tiled-attention operator.
 4. Apply TorchAO 8da4w post-training quantization:
     - dynamic INT8 activations
     - grouped INT4 linear weights
     - group size 128
     - incompatible layers retained at higher precision
-5. Quantize embedding tables to INT8. No calibration dataset or observer-calibration pass is used.
+5. Quantize embedding tables to packed INT4 weights. No calibration dataset or observer-calibration pass is used.
 6. Capture the quantized decoder with `torch.export` and decompose it into an ExecuTorch-compatible graph.
 7. Lower supported quantized and floating-point operations to the XNNPACK delegate and apply static memory planning.
 8. Serialize the graph and weights as an ExecuTorch `.pte` artifact compatible with the generic LLM runner.
 
 ## Known Limitations
 
-- This is a text-only export. The original Gemma 4 E2B model’s image and audio encoders are not included.
-- 8da4w post-training quantization can reduce accuracy relative to the original model. Linear weights are INT4, dynamic activations are INT8, and embeddings are INT8.
+- This is a text-only export. The original Gemma 4 E2B-IT model’s image and audio encoders are not included.
+- 8da4w+emb4 post-training quantization can reduce accuracy relative to the original model. Linear and embedding weights are INT4, and dynamic activations are INT8.
 - Initial inference may incur one-time weight preparation or repacking overhead. Subsequent inference can be faster while the model remains loaded.
-

metadata.yaml

@@ -1,20 +0,0 @@
-schema_version: 1.0.0
-report_type: llm-generative
-task_type: text-generation
-title: Gemma-4-E2B 8da4w — ExecuTorch
-id: Arm/gemma-4-e2b-text-executorch
-base_model_id: google/gemma-4-E2B
-vendor: Google
-base_model_url: https://huggingface.co/google/gemma-4-E2B
-profile: Arm-Optimized
-weight_dtype: int4
-model_size_mb: 2563.959
-format: executorch
-filename: gemma4_e2b_text_8da4w_emb4.pte
-quantization:
-  method: PTQ-dynamic
-  variant: 8da4w
-  weight_bits: 4
-  activation_bits: 8
-  mode: dynamic
-  weight_granularity: per-group