| # LapSRN x4 |
|
|
| Single-image super-resolution with a Laplacian Pyramid Super-Resolution Network. One forward |
| pass produces both a 2x and a 4x image, because the pyramid reconstructs each scale from the |
| previous one and both intermediate results are exposed as outputs. |
|
|
| The model was originally distributed as a frozen TensorFlow graph (`LapSRN_x4.pb`, the model |
| used by opencv_contrib's `dnn_superres` module) and converted to ONNX for use with OpenCV's |
| DNN module. |
|
|
| ## Model Details |
| - **Architecture**: LapSRN — Laplacian pyramid, progressive upsampling with `DepthToSpace` |
| and recursive residual blocks (24 `Conv`, 4 `DepthToSpace`) |
| - **Input**: `IteratorGetNext:0`, **NHWC** `[1, H, W, 1]` — a single **luma (Y)** channel, |
| float32 normalized to `[0, 1]`. `H`/`W` are dynamic; the network is fully convolutional. |
| - **Outputs** (both produced in one pass, NCHW): |
| - `NCHW_output_2x` — `[1, 1, H*2, W*2]` |
| - `NCHW_output_4x` — `[1, 1, H*4, W*4]` |
| - **Framework**: ONNX (converted from the TensorFlow frozen graph via tf2onnx, opset 13) |
| - **Original weights**: https://github.com/fannymonori/TF-LapSRN/tree/master/export |
| - **Paper**: Lai, Huang, Ahuja, Yang, *Deep Laplacian Pyramid Networks for Fast and Accurate |
| Super-Resolution*, CVPR 2017 — https://arxiv.org/abs/1704.03915 |
|
|
| > **Note on layout.** The input is NHWC while the outputs are NCHW (hence their names). This |
| > is not a mistake in the conversion — it is what `dnn_superres` feeds and reads, so the |
| > conversion preserves it. Do not use `blobFromImage` here; it would produce NCHW input. |
| |
| ### Colour handling |
| |
| The network only sees the Y channel. To get a colour image, the demos follow exactly what |
| `dnn_superres` does (`preprocess_YCrCb` / `reconstruct_YCrCb`): |
|
|
| 1. Convert BGR to YCrCb, scale to float `[0, 1]`. |
| 2. Feed **only Y**, reshaped to `[1, H, W, 1]`. |
| 3. Bicubically upscale Cr and Cb by the same factor. |
| 4. Merge `(Y_hr, Cr_hr, Cb_hr)`, scale back to 8-bit, convert YCrCb to BGR. |
|
|
| ## Usage |
|
|
| ### Python |
| ```bash |
| python demo.py --model lapsrn_x4_2026sep.onnx --image example_outputs/input_image.png --output-dir example_outputs |
| ``` |
| Writes `output_image_2x.png` and `output_image_4x.png`. Pass `--scale 2` or `--scale 4` to |
| run just one head. |
|
|
| ### C++ |
| The C++ demo runs inference with OpenCV's DNN module (default engine — no ONNX Runtime |
| needed). Adjust the OpenCV paths to your setup: |
| ```bash |
| OCV=/path/to/opencv # OpenCV source tree |
| OCVBUILD=/path/to/opencv/build # OpenCV build directory (generated headers + libs) |
| g++ -std=c++17 demo.cpp -o demo \ |
| -I$OCV/include \ |
| -I$OCV/modules/core/include \ |
| -I$OCV/modules/dnn/include \ |
| -I$OCV/modules/imgproc/include \ |
| -I$OCV/modules/imgcodecs/include \ |
| -I$OCVBUILD \ |
| -L$OCVBUILD/lib -Wl,-rpath,$OCVBUILD/lib -lopencv_dnn -lopencv_imgcodecs -lopencv_imgproc -lopencv_core |
| ./demo --model lapsrn_x4_2026sep.onnx --image example_outputs/input_image.png --output-dir example_outputs |
| ``` |
|
|
| The Python and C++ demos agree to within 1–2 intensity levels on under 1% of pixels — NumPy |
| and OpenCV break rounding ties differently, which the YCrCb to BGR conversion mildly amplifies. |
|
|
| ### With the dnn_superres module |
| |
| This is the same file the module's own test loads, so it also works through the module API: |
| |
| ```cpp |
| DnnSuperResImpl sr; |
| sr.readModel("lapsrn_x4_2026sep.onnx"); |
| sr.setModel("lapsrn", 4); |
| |
| std::vector<Mat> outputs; |
| sr.upsampleMultioutput(img, outputs, {2, 4}, {"NCHW_output_2x", "NCHW_output_4x"}); |
| ``` |
| |
| ## Conversion |
| |
| Exported from the frozen TensorFlow graph with tf2onnx (opset 13) via |
| [convert_to_onnx.py](./convert_to_onnx.py). Requires `tensorflow`, `tf2onnx` and `onnx`. |
| |
| ```bash |
| python convert_to_onnx.py --pb LapSRN_x4.pb |
| ``` |
| |
| Three details are specific to this model and are handled by the script: |
| |
| - **The weights are stored quantized and have to be folded back to float.** The frozen graph |
| went through TensorFlow's `quantize_weights` transform, so each of the 20 weight tensors is |
| a `quint8` const plus a min/max pair, reconstituted at runtime by a `Dequantize` node in |
| `MIN_FIRST` mode. tf2onnx has no mapping for `Dequantize`, so the script evaluates those |
| nodes in a TF session and rewrites each one as a plain float32 const. This is why the ONNX |
| (2.7 MB) is about 4x the size of the frozen graph (703 KB). |
| - **The graph is pruned to the two heads that are used.** The frozen graph carries a third |
| head, `NCHW_output`, that nothing reads. `extract_sub_graph` on `NCHW_output_2x` / |
| `NCHW_output_4x` drops it — that head and its `perm` const are the only two nodes removed, |
| so no compute changes (329 to 327 nodes). |
| - **The output names are stripped of their `:0` suffix.** tf2onnx names ONNX outputs after TF |
| tensors, so the heads come out as `NCHW_output_2x:0`, and |
| `net.forward("NCHW_output_2x")` cannot resolve that. The script renames them after |
| conversion. |
| |
| Note that `shape_override` uses `None`, not `-1`, for the dynamic H/W dims: the frozen graph's |
| placeholder has unknown rank, and tf2onnx's shape inference passes the override straight to |
| `set_shape`, which rejects `-1`. |
| |
| Re-running the script on `LapSRN_x4.pb` reproduces the shipped model: same opset, node count |
| and initializers, and **bit-identical inference output**. The only differences are tf2onnx's |
| auto-generated constant names (`const_fold_opt__611` and friends), whose numbering depends on |
| graph traversal order. |
| |
| The input is deliberately left NHWC — `--inputs-as-nchw` is *not* used here, unlike the other |
| `dnn_superres` conversions. |
| |
| ## Example |
| |
| | | | |
| |---|---| |
| | `example_outputs/input_image.png` | 256x256 (`butterfly.png`, the `dnn_superres` test image) | |
| | `example_outputs/output_image_2x.png` | 512x512, from `NCHW_output_2x` | |
| | `example_outputs/output_image_4x.png` | 1024x1024, from `NCHW_output_4x` | |
| |
| ## License |
| See [LICENSE](./LICENSE) — the model is released by the TF-LapSRN author under the |
| Apache License 2.0. |
| |
