README.md

metadata

title: Editable Image
emoji: ✏️
colorFrom: blue
colorTo: indigo
sdk: gradio
app_file: app.py
python_version: '3.11'
pinned: false

Editable Image

Editable Image is a local Python script that removes printed text from bitmap images and reconstructs it as a portable, editable SVG text layer. It uses compact PP-OCRv6 models through RapidOCR.

Install

Python 3.11 or newer and uv are recommended.

uv sync --extra cpu --extra test
uv run python make_editable_image.py models install ocr

For NVIDIA OCR acceleration, use --extra cuda instead of --extra cpu. Add --extra lama to install optional neural inpainting support. LaMa's approximately 200 MB model is only downloaded by an explicit command:

uv run python make_editable_image.py models install lama

Use

uv run python make_editable_image.py image.png --output output
uv run python make_editable_image.py images/ --recursive --output output

Each input produces <output>/<image-name>/<image-name>-background.png and <image-name>-overlay.svg. The SVG links the sibling background PNG, so keep both files together when moving the output. Run uv run python make_editable_image.py --help for all options. The installed editable-image convert ... command is equivalent.

Gradio / Hugging Face Spaces

The repository includes a Gradio interface in app.py with controls for every conversion option. To run it locally:

uv run --with-requirements requirements.txt python app.py

For Hugging Face Spaces, create a Gradio Space and upload the repository. The README metadata selects app.py, and requirements.txt installs the package with CPU ONNX Runtime. The default OpenCV cleanup and bitmap extraction work without additional model downloads. LaMa and SAM 2 remain available in the UI when their optional dependencies and model files are added to the Space.

Add --vectorize-shapes to promote confident diagram primitives—rounded rectangles, circles, ellipses, simple polygons, and connector-like paths—into editable SVG elements:

uv run python make_editable_image.py image.png --output output --vectorize-shapes

Detailed illustrations are preserved as transparent PNG crops under <image-name>-assets/ and linked from the SVG. Move the complete per-image output directory together. Shape recognition is conservative: uncertain elements remain in the background bitmap.

Bitmap masks use OpenCV by default. SAM 2 can optionally refine difficult masks:

uv sync --extra cpu --extra sam2
uv run python make_editable_image.py models install sam2
uv run python make_editable_image.py image.png --output output \
  --vectorize-shapes --bitmap-extractor sam2

The SAM 2 checkpoint is downloaded only by the explicit install command.

Detected text orientation snaps to 0, 45, 90, -45, or -90 degrees when it is within 6 degrees of a target. Override this with --snap-angles 0,30,60,90,-30,-60,-90 and --snap-tolerance 4.

Add --snap-font-sizes to estimate the font-size distribution across each image and snap nearby estimates to its supported peaks. Isolated sizes, such as a single large heading, remain unchanged.

SVGs reference font family names by default to keep files small. Add --embed-fonts to embed the used WOFF2 font faces for portable rendering.

Example output

The repository includes a shape-vectorization result for examples/scheme1.png under examples/generated-output/scheme1/. The directory contains the cleared background, editable SVG overlay, linked transparent bitmap assets, and a rendered PNG preview.

Notes

• The default OpenCV cleanup is fast and works best on flat or slowly varying backgrounds.
• LaMa can improve larger removals over textured backgrounds but adds PyTorch and a much larger model.
• Font recovery is approximate. The app bundles OFL-licensed Lato and Liberation faces; embedding them is optional.
• The script accepts PNG, JPEG, and WebP images up to 50 MB and 40 megapixels.
• Shape vectorization targets clean diagram geometry, not photographs, logos, or detailed illustrations.

Method

The conversion is a hybrid OCR and image-processing pipeline. It does not use a large generative model for the normal workflow.

1. Decode the bitmap. Pillow reads PNG, JPEG, or WebP input, applies EXIF orientation, preserves the ICC profile, and converts the pixels to an RGBA NumPy array.
2. Detect and recognize text. RapidOCR runs the compact PP-OCRv6-small detection and recognition models through ONNX Runtime. ONNX Runtime provides either CPU execution or optional NVIDIA CUDA acceleration. The OCR result supplies the text, confidence, and four-corner polygon for each line.
3. Estimate text appearance. OpenCV measures the line angle and foreground color from pixels inside each OCR polygon. Pillow renders candidates from the bundled Lato faces, and OpenCV compares their masks with the source to estimate regular or bold weight, italic style, and font size. With --snap-font-sizes, NumPy evaluates a Gaussian kernel density over all estimated sizes, keeps peaks supported by multiple lines, and snaps nearby sizes to those peaks. This is an approximation rather than recovery of the original font file.
4. Normalize orientation. The measured line angle is snapped to a configured target when it falls within the snap tolerance. The default targets are 0, 45, 90, -45, and -90 degrees.
5. Protect recognized text. The OCR polygons are converted into a protection mask before graphical analysis. This prevents letters, bullets, and text strokes from being promoted into shapes and later provides transparent text holes in extracted bitmap layers.
6. Recognize optional diagram geometry. With --vectorize-shapes, OpenCV analyzes the untouched source at two scales. Median-filtered color clustering proposes large, low-frequency panels; LAB color clusters, connected components, contours, and edge rectangles propose smaller geometry. A dedicated arrow pass groups neutral and overlapping HSV color bands, then scores concave silhouettes for a sharp extremal tip, a supported head, a tail, and approximate symmetry. Repeated-arrow context inside panels suppresses arrow-like illustration fragments, while connectors between panels are retained. The remaining fitter tests rounded rectangles, ellipses, simple polygons, and low-complexity paths. Nested candidates and textured regions are rejected conservatively so detailed illustrations remain raster.
7. Fit and score SVG primitives. Every proposal is rendered back into a mask and scored using geometric overlap, visible-edge support, color support, size, and containment. Accepted shapes retain both their editable geometry and original source contour. Their appearance is fitted as a solid fill or, when it materially lowers reconstruction error, a two-stop linear gradient with optional stroke styling.
8. Extract bitmap foreground layers. For an accepted container such as a colored panel, the fitted fill is compared with the source in LAB space. OpenCV uses the residual as foreground/background seeds for GrabCut, filters small components, and derives a feathered alpha channel. --bitmap-extractor sam2 optionally refines that mask with locally installed SAM 2 masks. Original RGB pixels are never traced: each retained illustration is written as a transparent PNG crop.
9. Build the cleanup mask. Text masks and accepted source contours are combined with any manual mask strokes. Large vectorized containers are cleared by estimating the underlying low-frequency canvas color from their outer perimeter; smaller text and shape remnants use OpenCV Telea inpainting or optional LaMa cleanup. Filled arrows are exported as editable closed SVG paths with data-role="arrow"; centerline arrows use SVG stroke markers so their arrowheads rotate and scale with the path.
10. Write ordered hybrid output. Pillow saves the cleared bitmap and transparent bitmap assets as PNG. The SVG links the cleared background, emits editable primitives and bitmap crops in layer order, and places editable <text> in the final foreground group. With --embed-fonts, FontTools converts the used bundled faces to WOFF2 and stores them inside the SVG; otherwise the SVG names the font families.