README.md

metadata

license: apache-2.0
library_name: flaxspeaker
pipeline_tag: audio-classification
tags:
  - audio
  - speaker-verification
  - speaker-recognition
  - speaker-embeddings
  - jax
  - flax
  - safetensors
  - conformer
  - ge2e_softmax
  - cosine
datasets:
  - librispeech_asr
  - cnceleb
  - voxceleb
metrics:
  - eer
  - min_dcf
  - roc_auc

FlaxSpeaker: ls_conformer_tiny_ge2e (conformer - tiny)

Disclaimer & Open-Source Reproduction Notice

This is not an officially supported Google product.

This model checkpoint and the accompanying FlaxSpeaker library are an independent, clean-room open-source reproduction in JAX/Flax of speaker recognition algorithms published in:

• GE2E: Generalized End-to-End Loss for Speaker Verification (arXiv:1710.10467)
• Dr-Vectors: Decision Residual Networks and an Improved Loss for Speaker Recognition (arXiv:2104.01989)
• Conformer-ATP: Attentive Temporal Pooling for Conformer-based Streaming Language Identification in Long-form Speech (arXiv:2202.12163)
• PFAS: Parameter-Free Attentive Scoring for Speaker Verification (arXiv:2203.05642v3)

The experiments in those original papers were conducted using Google's proprietary internal infrastructure and internal datasets, not this open-source library or public dataset split.

Model Summary

Property	Value
Model ID	ls_conformer_tiny_ge2e
Hugging Face Repository	wq2012/ls_conformer_tiny_ge2e
Hugging Face Collection	FlaxSpeaker Collection
GitHub Repository	wq2012/FlaxSpeaker
PyPI Package	flaxspeaker
Backbone Architecture	conformer (tiny)
Temporal Pooling	atp
Training Loss	ge2e_softmax
Verification Scoring	cosine
Output Embedding Dim	128
Encoder Parameters	693,126
Training Domain	librispeech
Audio Frontend	80-dim log_mel @ 16000 Hz + Energy VAD + CMVN

Model Files Included

• model.safetensors: Primary model weights in SafeTensors format
• flax_model.msgpack: Native Flax Msgpack serialized parameters
• config.json: FlaxSpeakerConfig model & training architecture configuration
• preprocessor_config.json: WhisperFeatureExtractor-compatible frontend configuration
• eval_results.json: Verified multi-domain evaluation metrics (1-utt & 3-utt EER, minDCF, ROC-AUC, 95% bootstrap CIs)

Quickstart Usage

Python API (from Hugging Face Hub or Local Directory)

from flaxspeaker import load_pretrained

model = load_pretrained("wq2012/ls_conformer_tiny_ge2e")
emb = model.extract_embedding("path/to/utterance.flac")
score = model.verify("path/to/enroll.flac", "path/to/test.flac")
print(f"Embedding shape: {emb.shape}, Verification score: {score:.4f}")

Command-Line Inference

python scripts/inference.py --model_dir wq2012/ls_conformer_tiny_ge2e \
    --enroll_audio path/to/enroll.flac --test_audio path/to/test.flac

Multi-Domain Evaluation Results

1-Utterance Verification Across Evaluation Domains

Evaluation Split	Trials	EER (%)	95% Bootstrap CI (%)	minDCF@0.01	minDCF@0.10	ROC-AUC
LibriSpeech (English Read)	3,000	10.53%	[9.40, 11.53]	0.7720	0.4893	0.9608
CN-Celeb2 (Chinese Multi-Genre)	3,000	30.00%	[28.65, 31.87]	0.9993	0.9547	0.7787
VoxCeleb1 (In-the-Wild Celebrity)	2,000	25.70%	[23.90, 27.48]	0.9380	0.8950	0.8282
Universal Pooled (All 3 Domains)	8,000	21.91%	[21.05, 22.81]	0.9968	0.8525	0.8711
Universal Macro-Average (3 Domains)	8,000	22.08%	—	0.9031	0.7797	0.8559

3-Utterance Multi-Enrollment Verification Across Evaluation Domains

Evaluation Split	Trials	3-Utt EER (%)	95% Bootstrap CI (%)	minDCF@0.01	ROC-AUC
LibriSpeech (3-Utt)	1,000	6.20%	[4.80, 7.95]	0.4880	0.9845
CN-Celeb2 (3-Utt)	1,000	25.80%	[23.20, 28.80]	0.8960	0.8256
VoxCeleb1 (3-Utt)	1,000	21.60%	[19.00, 23.91]	0.9100	0.8708
Universal Pooled (3-Utt)	3,000	18.07%	[16.50, 19.33]	0.8647	0.9049
Universal Macro-Average (3-Utt)	3,000	17.87%	—	0.7647	0.8936

Citation

@inproceedings{wan2018generalized,
  title={Generalized End-to-End Loss for Speaker Verification},
  author={Wan, Li and Wang, Quan and Papir, Alan and Moreno, Ignacio Lopez},
  booktitle={IEEE International Conference on Acoustics, Speech and Signal Processing (ICASSP)},
  pages={4879--4883},
  year={2018}
}

@inproceedings{pelecanos2021dr,
  title={Dr-Vectors: Decision Residual Networks and an Improved Loss for Speaker Recognition},
  author={Pelecanos, Jason and Wang, Quan and Moreno, Ignacio Lopez},
  booktitle={Interspeech},
  pages={4603--4607},
  year={2021}
}

@inproceedings{wang2022attentive,
  title={Attentive Temporal Pooling for Conformer-based Streaming Language Identification in Long-form Speech},
  author={Wang, Quan and Yu, Yang and Pelecanos, Jason and Huang, Yiling and Moreno, Ignacio Lopez},
  booktitle={The Speaker and Language Recognition Workshop (Odyssey)},
  year={2022}
}

@inproceedings{pelecanos2022parameter,
  title={Parameter-Free Attentive Scoring for Speaker Verification},
  author={Pelecanos, Jason and Wang, Quan and Huang, Yiling and Moreno, Ignacio Lopez},
  booktitle={The Speaker and Language Recognition Workshop (Odyssey)},
  year={2022}
}