Preserve DA3 Small source behavior in FP16 model

README.md

@@ -8,112 +8,51 @@ tags:
   - depth-estimation
   - monocular-depth
   - on-device
-  - gpu
   - depth-anything
 base_model: depth-anything/DA3-SMALL
 base_model_relation: quantized
 ---
 
-# Depth Anything 3 (Small) — LiteRT GPU, monocular depth
+# Depth Anything 3 Small — LiteRT
 
-On-device **LiteRT / TFLite** conversion of [**Depth Anything 3 — Small**](https://huggingface.co/depth-anything/DA3-SMALL)
-(ByteDance-Seed, Apache-2.0) for **monocular depth**, running fully on the mobile **GPU** via the LiteRT
-`CompiledModel` API (ML Drift delegate). No CPU fallback ops — the whole graph is GPU-compatible.
+[Depth Anything 3 Small](https://huggingface.co/depth-anything/DA3-SMALL)
+for single-image monocular depth estimation in LiteRT/TFLite format. The model
+preserves the source depth head's interpolation and UV positional refinement.
 
-| | |
+| Property | Value |
 |---|---|
-| Task | Monocular depth (single RGB → depth) |
-| Backbone | DINOv2 ViT-S + RoPE, DPT/DualDPT depth head |
-| Input | `[1, 3, 896, 504]` NCHW float32, ImageNet-normalized, **native portrait aspect** |
-| Output | `[1, 1, 896, 504]` depth |
-| Precision / size | FP16, **55 MB** |
-| Device | Pixel 8a, LiteRT GPU (`Accelerator.GPU`), **~0.9 s / image** (FP16, CompiledModel.Run) |
-| Fidelity | **corr 0.99948** vs official PyTorch; on-device **GPU-vs-CPU cos 0.99993** (re-verified, see below) |
-
-![Depth Anything 3 Small — input photo and on-device depth map (Pixel 8a, LiteRT GPU)](sample-depth.png)
-
-## Why a fixed 896×504 (native aspect, not square)
-
-DA3 processes images at their **native aspect ratio** (`upper_bound_resize`, longer side → 896, multiple of 14).
-Forcing a square `896×896` and letterbox-padding drops the match to corr **0.977** (the black padding leaks into
-the content through global attention). Converting at the native rectangle restores **corr 0.9994** and is also
-faster (fewer tokens). This checkpoint is built for **portrait ~9:16**. For another aspect, re-convert at that
-shape (or your camera's fixed aspect) with the script below.
-
-## Preprocessing (must match)
-
-```
-resize to 504×896 (W×H)  →  x/255  →  (x - mean) / std
-mean = [0.485, 0.456, 0.406], std = [0.229, 0.224, 0.225]   # ImageNet, RGB, NCHW
-```
-
-## GPU-clean conversion (what was patched)
-
-Converted with `litert-torch`. DA3 is not GPU-clean out of the box; the following exact, GPU-clean rewrites
-were applied (all numerically faithful unless noted):
-
-1. checkpoint `model.` key-prefix strip (load fix)
-2. RoPE `max_position = int(positions.max())+1` → constant (torch.export data-dependent)
-3. fused-QKV attention → 3 separate Linears + 4D attention (avoids 5D RESHAPE; exact, 1e-6)
-4. **LayerScale** `gamma` folded into `attn.proj` / `mlp.fc2` (the LayerScale MUL otherwise mis-lays-out the
-   token dim on the GPU delegate: `fully_connected {1,1,N,C} vs {N,1,1,C}`)
-5. `pos_embed` bicubic interpolation **baked** to a constant (the interpolate of a constant emits `GATHER_ND`
-   on desktop and `RESIZE_BILINEAR` with 0 runtime inputs on device)
-6. **ConvTranspose2d(k=s,stride=s)** → zero-stuff (nearest-upsample × top-left mask) + `Conv2d` (flipped
-   weight) — exact equivalent (~1e-7), because the Pixel-8a GPU rejects `TRANSPOSE_CONV` and the conv+
-   depth-to-space alternative needs >4D
-7. DPT-head `custom_interpolate` `align_corners=True → False` (GPU bans `align_corners=True` resize) — **the
-   only non-exact rewrite**; source of the residual ~0.05 % vs the official model
-8. head UV pos-embed-again disabled (its `make_sincos` broadcast emits `BROADCAST_TO`; ratio-0.1 refinement)
-9. camera-token insertion `x[:, :, 0] = cam_token` → `torch.cat` (in-place index-assign → `SELECT_V2`)
-
-Net result: `GATHER_ND = 0`, no `>4D` tensors, no `TRANSPOSE_CONV` / `BROADCAST_TO` / banned ops.
-
-## Fidelity note (honest)
-
-corr **0.99948** vs the official FP32 PyTorch pipeline. FP16 is **not** a factor (FP32≡FP16, corr 1.0). The
-residual ~0.05 % is the `align_corners=True→False` change in (7), which the mobile GPU forces — an irreducible
-hardware constraint, not a conversion error. Structure and edge sharpness are visually identical.
-
-## On-device GPU verification (re-confirmed)
-
-Re-verified on a Pixel 8a with the official LiteRT C++ runtime + ML Drift accelerator: the model compiles
-to **`Replacing 1460 out of 1460 node(s) with delegate (LITERT_CL)`** (full residency, single partition,
-no XNNPACK CPU fallback), and the **on-device GPU output matches the CPU/XNNPACK reference at cos 0.99993 /
-Pearson 0.99975** for the same input — i.e. the GPU result is numerically faithful, not merely "resident"
-(GPU full residency does not by itself guarantee a correct result).
-
-## Minimal usage
-
-**Android (Kotlin, CompiledModel GPU)**
-
-```kotlin
-val model = CompiledModel.create(context.assets, "da3_small_gpu_fp16.tflite",
-    CompiledModel.Options(Accelerator.GPU), null)
-val inputs = model.createInputBuffers()
-val outputs = model.createOutputBuffers()
-inputs[0].writeFloat(chw)              // [1,3,896,504] ImageNet-normalized, NCHW (portrait)
-model.run(inputs, outputs)
-val depth = outputs[0].readFloat()     // [1,1,896,504]
-```
-
-**Python (desktop verification)**
+| File | `da3_small_gpu_fp16.tflite` |
+| Size | 54.77 MB |
+| Weights | FP16 storage for supported weights; other constants can remain FP32 |
+| Input | RGB, float32 NCHW `[1, 3, 896, 504]` |
+| Output | Float32 depth `[1, 1, 896, 504]` |
+| Signature | `serving_default(args_0) -> output_0` |
+| Input tensor | `serving_default_args_0` |
+| Output tensor | `serving_default_output_0_output` |
+
+Resize the RGB image directly to **504 × 896 pixels (width × height)** with
+Pillow bilinear interpolation. Divide pixels by 255, then normalize using
+ImageNet mean `[0.485, 0.456, 0.406]` and standard deviation
+`[0.229, 0.224, 0.225]`. The fixed portrait shape uses a direct resize, without
+cropping or letterboxing.
+
+Python dependencies: `ai-edge-litert`, `numpy`, and `Pillow`.
 
 ```python
-MEAN = np.array([0.485, 0.456, 0.406], np.float32)
-STD  = np.array([0.229, 0.224, 0.225], np.float32)
 import numpy as np
 from PIL import Image
 from ai_edge_litert.interpreter import Interpreter
 
-img = Image.open("portrait.jpg").convert("RGB").resize((504, 896))   # W=504, H=896
-x = ((np.asarray(img, np.float32) / 255 - MEAN) / STD).transpose(2, 0, 1)[None]
+image = Image.open("image.jpg").convert("RGB")
+image = image.resize((504, 896), Image.Resampling.BILINEAR)
+pixels = np.asarray(image, dtype=np.float32) / 255.0
+normalized = (pixels - [0.485, 0.456, 0.406]) / [0.229, 0.224, 0.225]
+x = np.ascontiguousarray(normalized.transpose(2, 0, 1)[None], dtype=np.float32)
 
-it = Interpreter(model_path="da3_small_gpu_fp16.tflite"); it.allocate_tensors()
-it.set_tensor(it.get_input_details()[0]["index"], x); it.invoke()
-d = it.get_tensor(it.get_output_details()[0]["index"])[0, 0]         # [896,504]
-d = (d - d.min()) / (d.max() - d.min())
-Image.fromarray((d * 255).astype(np.uint8)).save("depth.png")
+interpreter = Interpreter(model_path="da3_small_gpu_fp16.tflite", num_threads=4)
+interpreter.allocate_tensors()
+run = interpreter.get_signature_runner("serving_default")
+depth = run(args_0=x)["output_0"][0, 0]  # float32, shape [896, 504]
 ```
 
 ## Training data & PII
@@ -129,6 +68,15 @@ release and [paper](https://arxiv.org/abs/2511.10647) for full dataset details.
 
 ## Performance
 
+These benchmark results are retained from the [previously published model card
+at revision `5cd25d9`](https://huggingface.co/litert-community/Depth-Anything-3-Small/blob/5cd25d936e3fde2edef68f53b4123401454ac9c8/README.md).
+The Pixel 8a, Snapdragon NPU, and Raspberry Pi measurements below refer to that
+previous model file. Measurements of the proposed replacement are in [PR #1](https://huggingface.co/litert-community/Depth-Anything-3-Small/discussions/1).
+
+The original card also reports approximately **0.9 seconds per image** on a
+**Pixel 8a** through LiteRT `CompiledModel` GPU (`LITERT_CL`), with
+**1,460 / 1,460 operators delegated**.
+
 Measured on a **Pixel 8a** (Tensor G3, Android 16) with the standard TFLite [`benchmark_model`](https://ai.google.dev/edge/litert/models/measurement) tool — 10 warm-up runs then 50 timed runs, reported as the tool's mean.
 
 | Runtime | Backend | Graph on GPU | Latency |
@@ -163,7 +111,7 @@ Measured on a Raspberry Pi 5 Model B Rev 1.1 (8 GB, Raspberry Pi OS 64-bit) with
 |---|---:|---:|---:|---:|
 | `da3_small_gpu_fp16.tflite` | 4,201.3 ms | 4,055.0–4,251.9 ms | 109 | 445 MB |
 
-## License
-
-Apache-2.0, inherited from the upstream [Depth Anything 3](https://github.com/ByteDance-Seed/depth-anything-3).
-This is a format conversion; all credit to the original authors (ByteDance-Seed).
+Source checkpoint:
+[`depth-anything/DA3-SMALL` at `e08cab65ca0ec38e7826075418411ab90cab4da3`](https://huggingface.co/depth-anything/DA3-SMALL/tree/e08cab65ca0ec38e7826075418411ab90cab4da3).
+See the [Depth Anything 3 project](https://github.com/ByteDance-Seed/Depth-Anything-3)
+for the original model. License: Apache 2.0.

da3_small_gpu_fp16.tflite

@@ -1,3 +1,3 @@
 version https://git-lfs.github.com/spec/v1
-oid sha256:e170369a72ba1bba7486a4d2de555639fccd0595a9bb5b5349f7733ed4aebd1f
-size 55035456
+oid sha256:ae153854383ad4599651eb8ac0a6606c9101a29b41609ffad8f43703cf78d7df
+size 54766600