SayedShaun/resemble-enhance-litert

Resemble Enhance LiteRT

Experimental Resemble Enhance conversion for local speech denoising and enhancement. Choose the model weight format with dtype:

dtype	Approximate model size	Description
"fp32"	1.42 GB	Float32 weights.
"fp16"	712 MB	Float16-stored weights; default.
"int8"	360 MB	Experimental INT8 weights; the vocoder uses weight-only quantization.

All versions have float32 inputs and outputs. INT8 is a mixed quantization configuration, not a fully integer model; parts of the pipeline use floating-point computation.

Original model author: Resemble AI
License: MIT, copyright © 2023 Resemble AI

Python inference

Tested on Linux x86-64 with Python 3.12. Install the dependencies and download the inference class:

pip install 'huggingface_hub>=0.23,<3' ai-edge-litert==2.1.2 'numpy>=1.26,<3' 'scipy>=1.11,<2' 'soundfile>=0.12,<1'
hf download SayedShaun/resemble-enhance-litert inference.py --local-dir .

Create the class with the repository ID and enhance a file:

import soundfile as sf
from inference import ResembleEnhancer

model = ResembleEnhancer("SayedShaun/resemble-enhance-litert", dtype="int8")
audio, sample_rate = model.enhance("input.wav")
sf.write("enhanced.wav", audio, sample_rate)

The constructor downloads only the selected model files from the Hub and uses the Hugging Face cache. Set dtype="fp32", "fp16" (the default), or "int8"; from_pretrained(repo_id, dtype=...) accepts the same option. Inference runs locally. You can also pass a mono or (frames, channels) NumPy float array with model.enhance(samples, sample_rate=16000). The returned audio is mono float32 at 44.1 kHz.

inference.py runs the neural networks using ai-edge-litert; NumPy and SciPy provide audio processing and the flow solver. PyTorch is not needed. It processes overlapping one-second windows.

Android Kotlin

Copy inference.kt into your Android project, and change its package line to match your app. Add these dependencies to the app module's build.gradle.kts:

implementation("com.google.ai.edge.litert:litert:2.3.0")
implementation("com.github.wendykierp:JTransforms:3.1")

Add internet access to AndroidManifest.xml, directly inside <manifest>:

<uses-permission android:name="android.permission.INTERNET" />

The tested project uses Kotlin 2.4.0, compile SDK 35, and minimum SDK 24. The constructor downloads the selected weights from the public Hub repository the first time and caches them in app-private storage. Downloading and inference must run on a worker thread. For example:

import android.content.Context
import android.os.Handler
import android.os.Looper
import java.util.concurrent.Executors

// samples must contain decoded mono PCM at 44,100 Hz, with values near -1..1.
// onComplete runs on the main thread.
fun enhanceSpeech(context: Context, samples: FloatArray, onComplete: (Result<FloatArray>) -> Unit) {
    val appContext = context.applicationContext
    val main = Handler(Looper.getMainLooper())
    val worker = Executors.newSingleThreadExecutor()
    worker.execute {
        val result = runCatching {
            ResembleEnhancer(appContext, "SayedShaun/resemble-enhance-litert", dtype = "int8").use { model ->
                model.enhance(samples)
            }
        }
        main.post { onComplete(result) }
        worker.shutdown()
    }
}

Use dtype = "fp32", "fp16" (default), or "int8". Use result.onSuccess { enhanced -> /* use the mono 44.1 kHz samples */ }.onFailure { error -> /* handle the error */ } in the callback. The caller handles file decoding, resampling, playback and output encoding. You can instead load previously downloaded files with ResembleEnhancer(modelDirectory, dtype = "int8").

The source and example compile, and its DSP and solver unit tests pass. The downloader and cache were tested with INT8 files on an x86-64 Android emulator. One INT8 inference completed on the unaccelerated emulator, producing finite audio for a half-second input. This is a smoke test, not a phone performance or audio-quality validation. FP32 and FP16 Android inference have not been tested end to end.

Limitations

This is an experimental conversion, not an official Resemble AI release. It has not been verified on a physical Android device. One-second chunking can introduce artifacts. For one half-second INT8 test using identical input and random values, the Android Kotlin WAV differed from desktop Python LiteRT (RMS difference 0.0142 on a reference RMS of 0.0606); low-magnitude FFT phase differences contributed to the mismatch. Wider parity and audio-quality testing are needed. In an earlier matched-noise, one-second FP16 comparison with PyTorch, 49 of 44,100 samples exceeded an atol=0.002, rtol=0.002 numerical threshold; maximum absolute difference was 0.00378.

Original model and license

The original Resemble Enhance model and code were created by Resemble AI.

• Original model: ResembleAI/resemble-enhance
• Original code: resemble-ai/resemble-enhance
• License: MIT; original copyright © 2023 Resemble AI.

This repository is an independent experimental LiteRT conversion, not an official Resemble AI release.