CLAP larger_clap_general for Core AI: asset, tokenizer files, logit scales

.gitattributes

@@ -33,3 +33,4 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+crate-clap-general-float32.aimodel/main.mlirb filter=lfs diff=lfs merge=lfs -text

README.md

@@ -0,0 +1,56 @@
+---
+license: apache-2.0
+base_model: laion/larger_clap_general
+tags:
+  - audio
+  - clap
+  - audio-text-retrieval
+  - zero-shot-audio-classification
+  - core-ai
+  - apple-silicon
+  - macos
+library_name: swift-sample-search
+---
+
+# crate-clap-general — CLAP for Core AI (macOS 27)
+
+LAION's [`larger_clap_general`](https://huggingface.co/laion/larger_clap_general) (Wu et al.,
+"Large-scale Contrastive Language-Audio Pretraining", ICASSP 2023; Apache 2.0) exported to an
+Apple Core AI `.aimodel` for on-device sample-library search, similarity and zero-shot tagging.
+Why this checkpoint and not `larger_clap_music`: the music checkpoint's Hugging Face conversion
+collapses every clip to nearly one vector (audio–audio cosines 0.81–0.99, audio–text 0.01–0.04,
+logit scale 1.03, measured in `transformers` itself), so it cannot search or tag anything. The
+general checkpoint was trained on music too and behaves (a kick file scores 0.49 against
+"a kick drum").
+
+Read by [swift-sample-search](https://github.com/arraypress/swift-sample-search) and the
+[`crate`](https://github.com/arraypress/swift-crate-cli) command-line tool.
+
+## Files
+
+| Path | What |
+|---|---|
+| `crate-clap-general-float32.aimodel/` | Two entry points: `audio` (log-mel `[1,1,1001,64]` → `[1,512]`, HTSAT + projection) and `text` (RoBERTa ids `[1,77]` + mask `[1,77]` → `[1,512]`). Float32, 797 MB. |
+| `clap-support/vocab.json`, `merges.txt` | The checkpoint's own RoBERTa byte-level BPE files. |
+| `clap-support/scales.json` | `exp(logit_scale_a)` and `exp(logit_scale_t)` from the checkpoint (38.66 and 14.29), the token length, the source model id. |
+
+The host computes the log-mel exactly as `ClapFeatureExtractor` does (48 kHz, 64 Slaney mels
+50–14000 Hz, n_fft 1024, hop 480, `repeatpad` for clips under ten seconds), tokenises with the
+files above, and L2-normalises the outputs — `get_audio_features` / `get_text_features`.
+
+## Fidelity
+
+Measured against `transformers` 5.17 on the same inputs: tokenizer ids identical over 20
+phrases; log-mel 116–156 dB PSNR; text embeddings 142 dB; audio embeddings 144–147 dB
+(cosine 0.99999+); zero-shot probabilities within 1e-8. The export script and the fixtures are
+in the library repo (`Tools/export_clap.py`).
+
+## Use
+
+```sh
+hf download arraypress/crate-clap-general --local-dir models
+crate model install models
+crate index ~/Samples && crate search "punchy 808 kick"
+```
+
+Requires macOS 27 (Core AI) on Apple silicon. Converted with coreai-torch 0.4.2 / torch 2.13.

clap-support/scales.json

@@ -0,0 +1,6 @@
+{
+ "logit_scale_audio": 38.66471862792969,
+ "logit_scale_text": 14.285714149475098,
+ "max_tokens": 77,
+ "model": "laion/larger_clap_general"
+}

crate-clap-general-float32.aimodel/main.mlirb

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:7ff9511b7cd94860359eb9008cc50cb929e19a7c0fafe3aff6d434bb4ff59e0c
+size 797349985

crate-clap-general-float32.aimodel/metadata.json

@@ -0,0 +1,8 @@
+{
+  "author" : "LAION (Wu, Chen, Zhang, Hui, Berg-Kirkpatrick, Dubnov) — CLAP; Core AI export by SampleSearch",
+  "creationDate" : "20260924T112203Z",
+  "assetVersion" : "2.0",
+  "description" : "CLAP laion\/larger_clap_general: audio = log-mel [1,1,1001,64] → embedding [1,512] (HTSAT + projection); text = RoBERTa ids [1,77] + mask → embedding [1,512]. L2-normalise in the host. Tokenizer files beside the asset.",
+  "producer" : "coreai-core 1.0.0b2",
+  "license" : "Apache-2.0"
+}