--- title: Editable Image emoji: ✏️ colorFrom: blue colorTo: indigo sdk: gradio app_file: app.py python_version: "3.11" pinned: false --- # Editable Image Editable Image is a local Python script that removes printed text from bitmap images and reconstructs it as a portable, editable SVG text layer. It uses compact PP-OCRv6 models through RapidOCR. ## Install Python 3.11 or newer and `uv` are recommended. ```bash uv sync --extra cpu --extra test uv run python make_editable_image.py models install ocr ``` For NVIDIA OCR acceleration, use `--extra cuda` instead of `--extra cpu`. Add `--extra lama` to install optional neural inpainting support. LaMa's approximately 200 MB model is only downloaded by an explicit command: ```bash uv run python make_editable_image.py models install lama ``` ## Use ```bash uv run python make_editable_image.py image.png --output output uv run python make_editable_image.py images/ --recursive --output output ``` Each input produces `//-background.png` and `-overlay.svg`. The SVG links the sibling background PNG, so keep both files together when moving the output. Run `uv run python make_editable_image.py --help` for all options. The installed `editable-image convert ...` command is equivalent. ### Gradio / Hugging Face Spaces The repository includes a Gradio interface in `app.py` with controls for every conversion option. To run it locally: ```bash uv run --with-requirements requirements.txt python app.py ``` For Hugging Face Spaces, create a Gradio Space and upload the repository. The README metadata selects `app.py`, and `requirements.txt` installs the package with CPU ONNX Runtime. The default OpenCV cleanup and bitmap extraction work without additional model downloads. LaMa and SAM 2 remain available in the UI when their optional dependencies and model files are added to the Space. Add `--vectorize-shapes` to promote confident diagram primitives—rounded rectangles, circles, ellipses, simple polygons, and connector-like paths—into editable SVG elements: ```bash uv run python make_editable_image.py image.png --output output --vectorize-shapes ``` Detailed illustrations are preserved as transparent PNG crops under `-assets/` and linked from the SVG. Move the complete per-image output directory together. Shape recognition is conservative: uncertain elements remain in the background bitmap. Bitmap masks use OpenCV by default. SAM 2 can optionally refine difficult masks: ```bash uv sync --extra cpu --extra sam2 uv run python make_editable_image.py models install sam2 uv run python make_editable_image.py image.png --output output \ --vectorize-shapes --bitmap-extractor sam2 ``` The SAM 2 checkpoint is downloaded only by the explicit install command. Detected text orientation snaps to `0`, `45`, `90`, `-45`, or `-90` degrees when it is within 6 degrees of a target. Override this with `--snap-angles 0,30,60,90,-30,-60,-90` and `--snap-tolerance 4`. Add `--snap-font-sizes` to estimate the font-size distribution across each image and snap nearby estimates to its supported peaks. Isolated sizes, such as a single large heading, remain unchanged. SVGs reference font family names by default to keep files small. Add `--embed-fonts` to embed the used WOFF2 font faces for portable rendering. ## Example output The repository includes a shape-vectorization result for [`examples/scheme1.png`](examples/scheme1.png) under [`examples/generated-output/scheme1/`](examples/generated-output/scheme1/). The directory contains the cleared background, editable SVG overlay, linked transparent bitmap assets, and a rendered PNG preview. ## Notes - The default OpenCV cleanup is fast and works best on flat or slowly varying backgrounds. - LaMa can improve larger removals over textured backgrounds but adds PyTorch and a much larger model. - Font recovery is approximate. The app bundles OFL-licensed Lato and Liberation faces; embedding them is optional. - The script accepts PNG, JPEG, and WebP images up to 50 MB and 40 megapixels. - Shape vectorization targets clean diagram geometry, not photographs, logos, or detailed illustrations. # Method The conversion is a hybrid OCR and image-processing pipeline. It does not use a large generative model for the normal workflow. 1. **Decode the bitmap.** Pillow reads PNG, JPEG, or WebP input, applies EXIF orientation, preserves the ICC profile, and converts the pixels to an RGBA NumPy array. 2. **Detect and recognize text.** RapidOCR runs the compact PP-OCRv6-small detection and recognition models through ONNX Runtime. ONNX Runtime provides either CPU execution or optional NVIDIA CUDA acceleration. The OCR result supplies the text, confidence, and four-corner polygon for each line. 3. **Estimate text appearance.** OpenCV measures the line angle and foreground color from pixels inside each OCR polygon. Pillow renders candidates from the bundled Lato faces, and OpenCV compares their masks with the source to estimate regular or bold weight, italic style, and font size. With `--snap-font-sizes`, NumPy evaluates a Gaussian kernel density over all estimated sizes, keeps peaks supported by multiple lines, and snaps nearby sizes to those peaks. This is an approximation rather than recovery of the original font file. 4. **Normalize orientation.** The measured line angle is snapped to a configured target when it falls within the snap tolerance. The default targets are `0`, `45`, `90`, `-45`, and `-90` degrees. 5. **Protect recognized text.** The OCR polygons are converted into a protection mask before graphical analysis. This prevents letters, bullets, and text strokes from being promoted into shapes and later provides transparent text holes in extracted bitmap layers. 6. **Recognize optional diagram geometry.** With `--vectorize-shapes`, OpenCV analyzes the untouched source at two scales. Median-filtered color clustering proposes large, low-frequency panels; LAB color clusters, connected components, contours, and edge rectangles propose smaller geometry. A dedicated arrow pass groups neutral and overlapping HSV color bands, then scores concave silhouettes for a sharp extremal tip, a supported head, a tail, and approximate symmetry. Repeated-arrow context inside panels suppresses arrow-like illustration fragments, while connectors between panels are retained. The remaining fitter tests rounded rectangles, ellipses, simple polygons, and low-complexity paths. Nested candidates and textured regions are rejected conservatively so detailed illustrations remain raster. 7. **Fit and score SVG primitives.** Every proposal is rendered back into a mask and scored using geometric overlap, visible-edge support, color support, size, and containment. Accepted shapes retain both their editable geometry and original source contour. Their appearance is fitted as a solid fill or, when it materially lowers reconstruction error, a two-stop linear gradient with optional stroke styling. 8. **Extract bitmap foreground layers.** For an accepted container such as a colored panel, the fitted fill is compared with the source in LAB space. OpenCV uses the residual as foreground/background seeds for GrabCut, filters small components, and derives a feathered alpha channel. `--bitmap-extractor sam2` optionally refines that mask with locally installed SAM 2 masks. Original RGB pixels are never traced: each retained illustration is written as a transparent PNG crop. 9. **Build the cleanup mask.** Text masks and accepted source contours are combined with any manual mask strokes. Large vectorized containers are cleared by estimating the underlying low-frequency canvas color from their outer perimeter; smaller text and shape remnants use OpenCV Telea inpainting or optional LaMa cleanup. Filled arrows are exported as editable closed SVG paths with `data-role="arrow"`; centerline arrows use SVG stroke markers so their arrowheads rotate and scale with the path. 10. **Write ordered hybrid output.** Pillow saves the cleared bitmap and transparent bitmap assets as PNG. The SVG links the cleared background, emits editable primitives and bitmap crops in layer order, and places editable `` in the final foreground group. With `--embed-fonts`, FontTools converts the used bundled faces to WOFF2 and stores them inside the SVG; otherwise the SVG names the font families.