Document split checkpoint API and private development code

README.md

@@ -8,60 +8,88 @@ tags:
 
 # HeartCodec OSS Encoder
 
-Encoder-only weights for HeartCodec audio encoding. This package contains the
-audio feature extractors and query encoder, with original `encoder.*` tensor
-names preserved. It does **not** include decoder or RVQ quantizer weights.
+Encoder-only weights for HeartCodec audio tokenization. Pair this checkpoint
+with [HeartCodec-oss-20260123](https://huggingface.co/HeartMuLa/HeartCodec-oss-20260123),
+which supplies the shared RVQ quantizer and audio decoder. Together they support
+audio → eight token streams at 12.5 Hz → 48 kHz stereo reconstruction.
 
-## Contents
+This repository is a private development preview. The matching code is in
+[HeartMuLa/heartlib-codec-dev](https://github.com/HeartMuLa/heartlib-codec-dev);
+access to both repositories is required. The encoder loading API has not yet
+been merged into the public `HeartMuLa/heartlib` repository.
 
-- `encoder.safetensors`: 1,006 encoder tensors (approximately 2.15 GB).
-- `encoder_config.json`: configuration for the encoder module.
-- `SHA256SUMS`: SHA-256 checksum of the weight file.
-- `LICENSE`: license supplied with the source checkpoint.
+## Installation and reconstruction
 
-## Dependencies and loading
+Use Python 3.10, a CUDA-compatible PyTorch installation, and `ffmpeg` for MP3
+output. Install this development copy of `heartlib` in a separate environment;
+do not install both the public and development copies into one environment.
 
-This is a component checkpoint, not a standalone Transformers model. Do not
-load this repository directly with `HeartCodec.from_pretrained()`.
+```bash
+git clone https://github.com/HeartMuLa/heartlib-codec-dev.git
+cd heartlib-codec-dev
+pip install -e .
+hf auth login
+hf download HeartMuLa/HeartCodec-oss-encoder --local-dir ./ckpt/HeartCodec-oss-encoder
+hf download HeartMuLa/HeartCodec-oss-20260123 --local-dir ./ckpt/HeartCodec-oss-20260123
+python examples/run_music_reconstruction.py \
+  --encoder_path ./ckpt/HeartCodec-oss-encoder \
+  --decoder_path ./ckpt/HeartCodec-oss-20260123 \
+  --input_path /path/to/your/audio.mp3 \
+  --save_path ./reconstructed.mp3
+```
+
+Use authorized input audio. MP3 is encoded directly from float32 output at
+320 kbps, without an intermediate PCM WAV. WAV output is available by selecting
+a `.wav` output path. Output is trimmed to the input duration.
 
-The encoder implementation is available in
-[`mula-labs/heartlib_codec`](https://github.com/mula-labs/heartlib_codec),
-at revision `306b163fe2ab777fb16ba973fe2c25f39cb428f5` (access may be required).
-The main `HeartMuLa/heartlib` implementation may not contain these encoder
-classes. With the encoder-enabled implementation installed, load the component:
+## Python API
 
 ```python
-import json
-from pathlib import Path
-from huggingface_hub import snapshot_download
-from safetensors.torch import load_file
-from heartlib.heartcodec.models.encoder import HeartCodecEncoder
-
-root = Path(snapshot_download("HeartMuLa/HeartCodec-oss-encoder"))
-config = json.loads((root / "encoder_config.json").read_text())
-encoder = HeartCodecEncoder(config)
-weights = load_file(str(root / "encoder.safetensors"), device="cpu")
-encoder.load_state_dict(
-    {key.removeprefix("encoder."): value for key, value in weights.items()},
-    strict=True,
-)
-encoder.eval()
+import torch
+from heartlib.heartcodec.modeling_heartcodec import HeartCodec
+
+codec = HeartCodec.from_encoder_decoder_pretrained(
+    "HeartMuLa/HeartCodec-oss-20260123",
+    "HeartMuLa/HeartCodec-oss-encoder",
+    dtype=torch.float32,
+).to("cuda").eval()
+
+# waveform: float32 tensor [samples], [1, samples], or [2, samples]
+# sample_rate: positive integer sampling rate of the input waveform
+# tokens = codec.tokenize(waveform, sample_rate, batch_size=1)
+# reconstructed = codec.detokenize(tokens, num_steps=10, guidance_scale=1.25)
 ```
 
-Producing discrete tokens additionally requires the matching shared RVQ
-quantizer, passed to `encoder.tokenize(waveform, sample_rate, quantizer)`.
-Audio reconstruction also requires the decoder. The existing decoder release is
-[`HeartMuLa/HeartCodec-oss-20260123`](https://huggingface.co/HeartMuLa/HeartCodec-oss-20260123).
-No decoder or quantizer weights are duplicated here. End-to-end compatibility
-with that separate release has not yet been validated for this split package.
+Do not pass this encoder-only repository to `HeartCodec.from_pretrained()`.
+The existing decoder-only loading API remains supported. Encoder and decoder
+weights are strictly checked by the split-checkpoint loader.
+
+## Files
+
+- `encoder.safetensors`: 1,006 encoder tensors, approximately 2.15 GB;
+  original `encoder.*` names, shapes, dtypes and values are preserved.
+- `encoder_config.json`: feature extractor and query encoder configuration.
+- `SHA256SUMS`: weight-file SHA-256 checksum.
+- `LICENSE`: the source checkpoint's Apache-2.0 license.
+
+No decoder or RVQ weights are included here. The shared quantizer belongs to
+the paired decoder checkpoint; loading two independently trained quantizers
+is not supported.
+
+## Validation
 
-## Validation status
+On an NVIDIA B300, the split encoder and published decoder completed a
+239.293-second stereo audio round trip using FP32, batch size 1, seed 42,
+10 decoder steps and guidance scale 1.25. Tokens had shape `[8, 2992]`.
+Output was finite, non-silent, 48 kHz stereo and matched the input duration.
+PyTorch peak allocated memory was approximately 12.65 GiB; peak reserved
+memory was approximately 15.41 GiB. These are measurements for one input and
+configuration, not minimum-memory requirements or an audio-quality benchmark.
 
-The split preserves the source tensor names, dtypes, shapes, and raw tensor bytes.
-All 1,006 exported keys start with `encoder.`; no decoder or RVQ tensors are
-included. File structure and payload integrity are checked during extraction.
-This does not constitute an audio-inference or reconstruction-quality test.
+All exported encoder tensor bytes match the source checkpoint. The paired
+decoder's 818 tensors match the source decoder numerically; eight RVQ
+initialization flags differ only in stored shape (scalar versus `[1]`).
 
 ## License
 
-The source checkpoint declares Apache-2.0; its license is retained in `LICENSE`.
+Apache-2.0, as declared by the source checkpoint. See `LICENSE`.