compressionkit E2A2 codec, redistributed from danjacobellis/E2A2 with the author's permission
4e60ba0 | license: other | |
| license_name: redistributed-with-permission | |
| license_link: https://huggingface.co/danjacobellis/E2A2/tree/aafe5975f62631d55b418d997a63d81abae11494/audio | |
| # E2A2 `audio` codec for compressionkit | |
| E2A2 (*Encoding Efficient Asymmetric Autoencoders*) `audio` is an audio codec for 44.1 kHz stereo music, the two channels coded together. This repository holds its weights in the layout that [compressionkit](https://github.com/AmbiqAI/compressionkit) v0.3 reads, with an encoder half and a decoder half that load separately. | |
| ## Source and permission | |
| The weights are redistributed from their original public release, [`danjacobellis/E2A2`, subfolder `audio/`, revision `aafe5975`](https://huggingface.co/danjacobellis/E2A2/tree/aafe5975f62631d55b418d997a63d81abae11494/audio), with the permission of the author, Dan Jacobellis. Ambiq did not train this codec. Its description, development and evaluation are documented at the original source and on the codec's page, <https://danjacobellis.net/compressors/E2A2_audio>. The entropy coder is GGDLPC (<https://danjacobellis.net/GGDLPC/>). | |
| Every tensor here equals the tensor of the same name at that revision; this repository divides them into the encoder and decoder halves listed below and adds the files compressionkit uses. | |
| ## Files | |
| - `encoder/config.json`, `encoder/weights.safetensors` (the `merged.{op}.encoders.*` tensors) and `encoder/entropy.json` (the GGDLPC tables): everything needed to encode. | |
| - `decoder/config.json`, `decoder/weights.safetensors` (the `merged.{op}.decoder.*` tensors): everything needed to decode. Each `config.json` is the full source configuration, so each half constructs alone. | |
| - `rd.json`: the rate–distortion curve of the original release, byte-identical to it. Each validation item was zero-meaned and peak-normalized over its whole length before coding, a scaling a real-time encoder cannot compute within its block latency, so the curve is not a measurement of a configuration deployable in real time. | |
| - `calibrations/<dataset>.json` (`musdb`): fixed input maps, published by Ambiq for `compressionkit.calibrate.load`, that turn int16 codes of a dataset into the codec's input with two constants (an offset and a scale). | |
| - `web/`: the decoder of each operating point that the compressionkit browser demo offers, as LiteRT files (section below). The compressionkit loaders do not download `web/`. | |
| | file | bytes | SHA-256 | | |
| |---|---|---| | |
| | `calibrations/musdb.json` | 6,541 | `6da7d18d21657312254bfa572a7486f8b306494f340d0ef8ae56ecb3130ec901` | | |
| | `decoder/config.json` | 2,280 | `296eecf80011ff77a0efda76d208586ed70a197eb98b2cabbc566127555373f8` | | |
| | `decoder/weights.safetensors` | 4,855,248,808 | `3ecdc061c498bb2fe6159f20a50c134a790d2f793ceb8572ad6254f11c857a73` | | |
| | `encoder/config.json` | 2,280 | `296eecf80011ff77a0efda76d208586ed70a197eb98b2cabbc566127555373f8` | | |
| | `encoder/entropy.json` | 13,601 | `d0a03b4886c89f37e53984b5efdd3717343525ab3577de25e45a99d0695f0f0f` | | |
| | `encoder/weights.safetensors` | 183,932 | `a0196b471984981c5734f84924672ca1cf507487fc7b46d945d2afd5cca8c4a2` | | |
| | `rd.json` | 1,151 | `1b8ca06684a13c573c1e6e34f2e1b4c05a87c8093fe4ad4d2656be9ab59843d6` | | |
| ## Input, output and operating points | |
| Input: float32 `(2, L)` samples in [−1, 1] at 44.1 kHz. Output: a reconstruction in the same form. Operating points (cumulative latent channels): `[4, 9, 15, 21, 33, 42, 51]`. The stream of a lower operating point is a literal bit-prefix of the stream of the highest, and the rates are entropy-coded bits. compressionkit converts nothing: each codec states its input type and range. | |
| ## Using it with compressionkit | |
| compressionkit v0.3 is in pre-release; these loaders are part of it. | |
| ```python | |
| from compressionkit.e2a2 import hub | |
| # both halves | |
| config, weights, coder, ops = hub.load_split_from_hub('audio') | |
| model = hub.load_codec(config=config, weights=weights, n_ch=ops[-1]) | |
| # one half (encode-only shown; half='decoder' for decode-only) | |
| cfg, w, coder, ops = hub.load_split_from_hub('audio', half='encoder') | |
| encoder = hub.build_partial(cfg, w, ops[-1]) | |
| ``` | |
| ```python | |
| from compressionkit import calibrate | |
| cal = calibrate.load('audio', 'musdb') # the published map for that dataset | |
| ``` | |
| ## Status and limitations | |
| - A reference codec for evaluation and development with compressionkit, redistributed as released; not a production model, and not qualified for any product or regulated use. | |
| - Its quality on data unlike the data of its original release is not characterized here; measure it on your own data (`compressionkit.calibrate` fits an input map on your training split). | |
| - Maintainer for compressionkit: Dan Jacobellis (Ambiq). Questions about the codec itself: the original source. | |
| <!-- web/: hub/e2a2_web.py --> | |
| ## Browser decoders (`web/`) | |
| For each operating point the compressionkit browser demo offers, its decoder as a LiteRT `.tflite` window decoder, which runs in a web browser on LiteRT.js (CPU), with a `.json` beside it that gives its shapes and windowing. A window decodes `chunk` output samples (pixels along each side for the image codec) with `margin` of context on each side; the latent grid is cut into overlapping windows and the outputs joined. The PSNR column is the largest difference between the PSNR of the joined output, decoded in LiteRT.js in Chromium, and that of the full-size PyTorch decode, over the example recordings of the demo. The compressionkit loaders do not download `web/`. | |
| | file | operating point | chunk | margin | bytes | PSNR difference (dB, rows) | decoder weights (SHA-256) | | |
| |---|---|---|---|---|---|---| | |
| | `web/audio_op4_chunk16384_m4096.tflite` | 4 | 16384 samples | 4096 | 404,739,432 | 7.0e-09 (1) | `3ecdc061c498` | | |
| | `web/audio_op9_chunk16384_m4096.tflite` | 9 | 16384 samples | 4096 | 404,800,872 | 1.4e-07 (1) | `3ecdc061c498` | | |
| | `web/audio_op15_chunk16384_m4096.tflite` | 15 | 16384 samples | 4096 | 808,238,040 | 4.1e-07 (1) | `3ecdc061c498` | | |
| | `web/audio_op21_chunk16384_m4096.tflite` | 21 | 16384 samples | 4096 | 808,385,496 | 3.6e-07 (1) | `3ecdc061c498` | | |
| | `web/audio_op33_chunk16384_m4096.tflite` | 33 | 16384 samples | 4096 | 808,975,320 | 2.9e-07 (1) | `3ecdc061c498` | | |
| | `web/audio_op42_chunk16384_m4096.tflite` | 42 | 16384 samples | 4096 | 809,860,056 | 2.2e-07 (1) | `3ecdc061c498` | | |
| | `web/audio_op51_chunk16384_m4096.tflite` | 51 | 16384 samples | 4096 | 811,629,528 | 1.1e-06 (1) | `3ecdc061c498` | | |
| Weights fp32, converted from `decoder/weights.safetensors` with litert-torch 0.10.0. The last column gives the start of the SHA-256 of the `decoder/weights.safetensors` it was converted from, which is this repository's. | |
| <!-- /web/ --> | |