Upload folder using huggingface_hub

.gitattributes

@@ -33,3 +33,5 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+assistant.wav filter=lfs diff=lfs merge=lfs -text
+assets/banner.jpeg filter=lfs diff=lfs merge=lfs -text

README.md

@@ -1,3 +1,257 @@
 ---
 license: mit
+pipeline_tag: audio-classification
+library_name: transformers
+language:
+- multilingual
+
+tags:
+- audio-deepfake
+- deepfake-detection
+- deepfake
+- voice-cloning
+- anti-spoofing
+- asvspoof
+- wav2vec2
+- speech
+- audio
+- synthetic-voice
+- voice-conversion
+- tts-detection
+- trust-and-safety
+- security
+- SoTA
+- Modotte
+
+inference:
+  parameters:
+    function_to_apply: "softmax"
 ---
+
+## Modotte
+
+<p align="center">
+  <img 
+    src="https://huggingface.co/Modotte/AIRealNet-Audio/resolve/main/assets/banner.jpeg" 
+    alt="AIRealNet-Audio Banner" 
+    width="95%" 
+    style="border-radius:15px;"
+  />
+</p>
+
+## Overview
+
+> This is the future iteration of [AIRealNet](https://huggingface.co/Modotte/AIRealNet)
+
+In an era of rapidly advancing AI-generated speech, voice cloning, and audio deepfakes, the need for reliable detection tools has never been higher. **AIRealNet-Audio** is a binary audio classifier designed to distinguish AI-generated / spoofed audio from real human speech.
+
+It is built on a Wav2Vec-based audio encoder that processes audio in fixed **12-second chunks**. The model uses a default decision threshold of **50%**, which can be adjusted based on your use case.
+
+A key design choice addresses a common failure mode of public deepfake detectors: models quickly learn a shortcut based on embedding vector length (magnitude). To prevent this, all embeddings from the base model are **L1-normalized onto the unit hypersphere**, forcing the classifier to rely purely on angular (directional) information rather than magnitude.
+
+- **Class 0**: AI-generated / spoof audio  
+- **Class 1**: Real human audio  
+
+**Default decision threshold**: 0.5 (tune per use-case).
+
+---
+
+## Architecture
+
+This is a **Wav2Vec-based audio encoder** operating at **16 kHz**, paired with a projection layer that maps its base features of dimension **768** (pooled from shape `[T, 768]`) down to **256**. This 256-dimensional representation is the expected input shape for the classification head, which is a **single-layer MLP** producing 2 output logits (with softmax applied during inference).
+
+The core innovation of this architecture is the **dual L1 unit-norm** applied to the model features:
+
+1. The base features are L1-normalized **once** when passed to the projector.  
+2. The projected features are L1-normalized **again** when passed to the classification head.
+
+This dual-hypersphere design completely eliminates size (magnitude) bias.
+
+### Why Dual L1 Normalization?
+
+Most studies show that deepfake embeddings tend to cluster in a specific region of feature space **and** exhibit a characteristic vector magnitude. As a result, many detectors learn a shortcut: they simply detect the length of the embedding vector (or its presence in a particular cluster) instead of learning meaningful acoustic or spectral cues.
+
+To eliminate this shortcut, we force **every** feature vector onto the unit hypersphere via L1 normalization. Consequently:
+
+- The classification head has only one meaningful signal left to learn from — the **direction (θ)** of the vector.  
+- Because all vectors lie on the hypersphere, the opportunity for models to exploit magnitude-based or region-specific clustering is drastically reduced.
+
+This approach is similar in spirit to **GenD**, which applied a single L1 normalization onto the unit hypersphere. AIRealNet-Audio extends the idea by applying the normalization **twice** (dual hypersphere), once before the projector and once before the head.
+
+### Training Behavior
+
+When training **without** L1 normalization we observed a consistent pattern: the model performs well in the early stages, then begins to degrade after a fixed number of steps — even after extensive hyperparameter tuning and aggressive learning-rate reduction aimed at slower convergence.
+
+With the **dual-hypersphere** architecture the opposite occurs:
+
+- Steady upward trend in performance.  
+- The model continues to improve with each successive step instead of collapsing after a few thousand steps.
+
+**Loss per step**  
+
+<p align="center">
+  <img 
+    src="https://huggingface.co/Modotte/AIRealNet-Audio/resolve/main/assets/loss.png" 
+    alt="loss Banner" 
+    width="90%" 
+    style="border-radius:15px;"
+  />
+</p>
+
+**Accuracy per 1000 steps**  
+
+<p align="center">
+  <img 
+    src="https://huggingface.co/Modotte/AIRealNet-Audio/resolve/main/assets/accuracy.png" 
+    alt="accuracy Banner" 
+    width="90%" 
+    style="border-radius:15px;"
+  />
+</p>
+
+---
+
+## Training Data
+
+- AI-generated speech produced by more than **100 different TTS / voice-cloning systems**.
+- Large collection of real human speech drawn from varied recording conditions and sources.
+- On-the-fly augmentations including multiple compression codecs, bitrate changes, and additive noise.
+- Separate balanced evaluation and development sets used for monitoring.
+
+The combination of high TTS diversity and aggressive augmentation is intended to force the model to learn genuine synthesis artifacts rather than dataset-specific fingerprints.
+
+---
+
+## Limitations
+
+- Very short utterances (significantly under 12 s) must be padded or repeated; performance on extremely short clips may be lower.
+- Highly adversarial or “nano-edit” modifications of real audio remain challenging.
+- Completely unseen generators or extreme domain shifts (e.g., heavy telephony distortion not seen during training) can still reduce accuracy.
+- The model is a probabilistic detector; it should not be used as the sole evidence in high-stakes forensic or legal settings without human review.
+
+## Performance
+
+Across a range of standard and challenging evaluation sets the model demonstrates strong generalization. In most evaluation datasets AIRealNet-Audio maintains an **Equal error rate (EER) well below 6%**.
+
+> We eliminated audio files below 3sec as model is trained on 12-seconds.
+
+| Evaluation Dataset     | EER (%) |
+|------------------------|---------|
+| ASVspoof5              | 1.21    |
+| Voxness                | 1.48    |
+| Malaad                 | 2.11    |
+| ASVspoof2019 DF        | 3.14    |
+| In-the-Wild            | 3.64    |
+
+**Additional training-time observations**
+- After 2 epochs: ~**0.99 accuracy** on both the held-out evaluation set and the development set.
+- Subsequent checkpoints continue to show steady gains in accuracy (see accuracy curve above).
+- Training without the dual L1 normalization exhibits the classic early-peak-then-degradation pattern; the dual-hypersphere design removes this failure mode.
+
+*Note*: Extremely high numbers on controlled evaluation sets are expected. Real-world performance depends on the distribution of generators and acoustic conditions encountered at deployment time. Always calibrate the decision threshold on data that matches your target domain.
+
+---
+
+## Usage
+
+### Sample Audio
+The following sample was generated by **Gemini-3.8 Flash TTS** (AI-generated speech):
+
+<audio controls src="https://huggingface.co/Modotte/AIRealNet-Audio/resolve/main/assets/assistant.wav"></audio>
+
+### Quick Inference
+
+```python
+from transformers import pipeline
+
+pipe = pipeline(
+    "audio-classification",
+    model="Modotte/AIRealNet-Audio",
+    trust_remote_code=True
+)
+
+result = pipe(
+    "https://cdn-uploads.huggingface.co/production/uploads/677fcdf29b9a9863eba3f29f/C2llrmFhlWx-oryC9wF6f.wav"
+)
+print(result)
+```
+
+**Expected output:**
+```python
+[
+  {'score': 0.9979, 'label': 'AIVoice'},
+  {'score': 0.0021, 'label': 'HumanVoice'}
+]
+```
+
+| Label       | Meaning                  |
+|-------------|--------------------------|
+| `AIVoice`   | AI-generated / spoof audio |
+| `HumanVoice`| Real human speech        |
+
+Default decision threshold is **0.5**. You can adjust it according to your precision/recall needs.
+
+### Notes for Production Use
+1. Resample input to **16 kHz mono**.
+2. Segment long recordings into **12-second chunks** (the training chunk size).
+3. Aggregate chunk-level scores (mean, max, or calibrated fusion) as needed.
+
+---
+
+## Intended Use
+
+- Detection of AI-generated / spoofed speech on social media, messaging platforms, call centers, and research datasets.
+- Assistance for content moderators, journalists, fact-checkers, and platform trust-and-safety teams.
+- Research baseline for audio deepfake detection under a hyperspherical feature constraint.
+
+Not intended as the sole source of evidence in legal, forensic, or high-stakes verification scenarios without corroborating human analysis.
+
+---
+
+## Ethical Considerations
+
+- Training data construction followed the same privacy-first principles used for the original AIRealNet image model (no personal or sensitive recordings).
+- Users should treat model scores as one signal among many and always apply human review near the decision threshold.
+- The model card explicitly documents the dual-hypersphere design and the known length-bias failure mode so that downstream users understand both the strengths and the residual risks.
+
+---
+
+## How It Works
+
+1. Input audio is resampled to 16 kHz and segmented into 12-second chunks.
+2. A Wav2Vec encoder extracts frame-level features [T, 768].
+3. Features are mean-pooled, L1-normalized, and projected to 256 dimensions.
+4. The 256-dimensional vector is L1-normalized a second time and passed through a linear classification head.
+5. Softmax yields class probabilities; a threshold (default 0.5) produces the final binary decision.
+
+---
+
+## Future Work
+
+- Improve robustness to very short utterances and adversarial nano-edits.
+- Expand coverage to additional languages and emerging TTS / voice-conversion systems.
+- Investigate multi-modal (audio + video / metadata) detection.
+- Explore variable-length or streaming-friendly variants of the dual-hypersphere constraint.
+
+---
+
+## Citation
+
+```bibtex
+@misc{Modotte_AIRealNet_Audio_2025,
+    title  = {AIRealNet-Audio: Dual-Hypersphere Constrained Wav2Vec for Detecting AI-Generated vs Real Speech},
+    author = {Parvesh Rawal},
+    year   = {2025},
+    publisher = {Hugging Face},
+    url    = {https://huggingface.co/Modotte/AIRealNet-Audio}
+}
+```
+
+---
+
+## References
+
+- Yermakov, A., Cech, J., Matas, J., & Fritz, M. (2026). *Deepfake Detection that Generalizes Across Benchmarks* (GenD). Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV).  
+  [arXiv:2508.06248](https://arxiv.org/abs/2508.06248) · [GitHub](https://github.com/yermandy/GenD)
+- Microsoft / Facebook Wav2Vec 2.0 and related self-supervised speech models.
+- [AIRealNet](https://huggingface.co/Modotte/AIRealNet) — the image counterpart that motivated this audio extension.

assets/assistant.wav

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:976d73c3fe17f77ba1626c44bf3cf0f10a32d4c73e8057c6e426a4696fbee6da
+size 1384364

config.json

@@ -0,0 +1,130 @@
+{
+  "_name_or_path": "facebook/wav2vec2-base",
+  "activation_dropout": 0.0,
+  "adapter_attn_dim": null,
+  "adapter_kernel_size": 3,
+  "adapter_stride": 2,
+  "add_adapter": false,
+  "apply_spec_augment": true,
+  "architectures": [
+    "Wav2Vec2DualHypersphereForAudioClassification"
+  ],
+  "auto_map": {
+    "AutoConfig": "configuration.Wav2Vec2DualHypersphereConfig",
+    "AutoModelForAudioClassification": "modeling.Wav2Vec2DualHypersphereForAudioClassification"
+  },
+  "attention_dropout": 0.1,
+  "bos_token_id": 1,
+  "classifier_proj_size": 256,
+  "codevector_dim": 256,
+  "contrastive_logits_temperature": 0.1,
+  "conv_bias": false,
+  "conv_dim": [
+    512,
+    512,
+    512,
+    512,
+    512,
+    512,
+    512
+  ],
+  "conv_kernel": [
+    10,
+    3,
+    3,
+    3,
+    3,
+    2,
+    2
+  ],
+  "conv_stride": [
+    5,
+    2,
+    2,
+    2,
+    2,
+    2,
+    2
+  ],
+  "ctc_loss_reduction": "sum",
+  "ctc_zero_infinity": false,
+  "diversity_loss_weight": 0.1,
+  "do_stable_layer_norm": false,
+  "eos_token_id": 2,
+  "feat_extract_activation": "gelu",
+  "feat_extract_norm": "group",
+  "feat_proj_dropout": 0.1,
+  "feat_quantizer_dropout": 0.0,
+  "final_dropout": 0.0,
+  "freeze_feat_extract_train": true,
+  "hidden_act": "gelu",
+  "hidden_dropout": 0.1,
+  "hidden_size": 768,
+  "id2label": {
+    "0": "AIVoice",
+    "1": "HumanVoice"
+  },
+  "initializer_range": 0.02,
+  "intermediate_size": 3072,
+  "label2id": {
+    "AIVoice": "0",
+    "HumanVoice": "1"
+  },
+  "layer_norm_eps": 1e-05,
+  "layerdrop": 0.0,
+  "mask_channel_length": 10,
+  "mask_channel_min_space": 1,
+  "mask_channel_other": 0.0,
+  "mask_channel_prob": 0.0,
+  "mask_channel_selection": "static",
+  "mask_feature_length": 10,
+  "mask_feature_min_masks": 0,
+  "mask_feature_prob": 0.0,
+  "mask_time_length": 10,
+  "mask_time_min_masks": 2,
+  "mask_time_min_space": 1,
+  "mask_time_other": 0.0,
+  "mask_time_prob": 0.05,
+  "mask_time_selection": "static",
+  "model_type": "wav2vec2-dual-hypersphere",
+  "no_mask_channel_overlap": false,
+  "no_mask_time_overlap": false,
+  "num_adapter_layers": 3,
+  "num_attention_heads": 12,
+  "num_codevector_groups": 2,
+  "num_codevectors_per_group": 320,
+  "num_conv_pos_embedding_groups": 16,
+  "num_conv_pos_embeddings": 128,
+  "num_feat_extract_layers": 7,
+  "num_hidden_layers": 12,
+  "num_negatives": 100,
+  "output_hidden_size": 768,
+  "pad_token_id": 0,
+  "proj_codevector_dim": 256,
+  "tdnn_dilation": [
+    1,
+    2,
+    3,
+    1,
+    1
+  ],
+  "tdnn_dim": [
+    512,
+    512,
+    512,
+    512,
+    1500
+  ],
+  "tdnn_kernel": [
+    5,
+    3,
+    3,
+    1,
+    1
+  ],
+  "torch_dtype": "float32",
+  "transformers_version": "4.39.2",
+  "use_weighted_layer_sum": false,
+  "vocab_size": 32,
+  "xvector_output_dim": 512
+}

configuration.py

@@ -0,0 +1,30 @@
+from transformers import Wav2Vec2Config
+
+
+class Wav2Vec2DualHypersphereConfig(Wav2Vec2Config):
+    model_type = "wav2vec2-dual-hypersphere"
+
+    def __init__(
+        self,
+        classifier_proj_size=256,
+        final_dropout=0.1,
+        freeze_feat_extract_train=True,
+        **kwargs,
+    ):
+        # Ensure default classification metadata is set before calling super
+        if "num_labels" not in kwargs:
+            kwargs["num_labels"] = 2
+
+        if "id2label" not in kwargs:
+            kwargs["id2label"] = {0: "AI Voice (Fake)", 1: "Human Voice (Real)"}
+
+        if "label2id" not in kwargs:
+            kwargs["label2id"] = {"AI Voice (Fake)": 0, "Human Voice (Real)": 1}
+
+        super().__init__(**kwargs)
+        self.classifier_proj_size = classifier_proj_size
+        self.final_dropout = final_dropout
+        self.freeze_feat_extract_train = freeze_feat_extract_train
+
+
+__all__ = ["Wav2Vec2DualHypersphereConfig"]

model.safetensors

@@ -0,0 +1,3 @@
+version https://git-lfs.github.com/spec/v1
+oid sha256:219378a69e2a2f590676f89a2ce73f9ed8c12d0c3e2ab41db8884598c0be633b
+size 378302360

modeling.py

@@ -0,0 +1,124 @@
+from typing import Optional, Tuple, Union
+import torch
+import torch.nn as nn
+import torch.nn.functional as F
+from transformers import Wav2Vec2Model, Wav2Vec2PreTrainedModel
+from transformers.modeling_outputs import SequenceClassifierOutput
+from transformers.models.auto import AutoConfig, AutoModelForAudioClassification
+
+from .configuration import Wav2Vec2DualHypersphereConfig
+
+
+class Wav2Vec2DualHypersphereForAudioClassification(Wav2Vec2PreTrainedModel):
+    config_class = Wav2Vec2DualHypersphereConfig
+
+    def __init__(self, config: Wav2Vec2DualHypersphereConfig):
+        super().__init__(config)
+        self.num_labels = config.num_labels
+
+        self.wav2vec2 = Wav2Vec2Model(config)
+        proj_dim = getattr(config, "classifier_proj_size", 256)
+        self.projector = nn.Linear(config.hidden_size, proj_dim)
+        self.dropout = nn.Dropout(getattr(config, "final_dropout", 0.1))
+        self.classifier = nn.Linear(proj_dim, self.num_labels)
+
+        # Initialize weights and apply final processing
+        self.post_init()
+
+    def freeze_feature_extractor(self):
+        self.wav2vec2.feature_extractor._freeze_parameters()
+
+    def forward(
+        self,
+        input_values: Optional[torch.Tensor] = None,
+        attention_mask: Optional[torch.Tensor] = None,
+        output_attentions: Optional[bool] = None,
+        output_hidden_states: Optional[bool] = None,
+        return_dict: Optional[bool] = None,
+        labels: Optional[torch.Tensor] = None,
+    ) -> Union[Tuple, SequenceClassifierOutput]:
+        r"""
+        labels (`torch.LongTensor` of shape `(batch_size,)`, *optional*):
+            Labels for computing the sequence classification/regression loss.
+        """
+        return_dict = return_dict if return_dict is not None else self.config.return_dict
+
+        outputs = self.wav2vec2(
+            input_values,
+            attention_mask=attention_mask,
+            output_attentions=output_attentions,
+            output_hidden_states=output_hidden_states,
+            return_dict=return_dict,
+        )
+        hidden_states = outputs[0]
+
+        # 1. Temporal Pooling with Attention Mask support
+        if attention_mask is not None:
+            mask = self.wav2vec2._get_feature_vector_attention_mask(
+                hidden_states.shape[1], attention_mask
+            )
+            mask = mask.unsqueeze(-1).expand_as(hidden_states)
+            pooled_768 = (hidden_states * mask).sum(dim=1) / mask.sum(dim=1).clamp(min=1e-9)
+        else:
+            pooled_768 = hidden_states.mean(dim=1)
+
+        # 2. First Hypersphere Projection (L2 Normalize 768-D)
+        norm_pooled_768 = F.normalize(pooled_768, p=2, dim=-1)
+
+        # 3. Intermediate Projection & Dropout
+        projected_256 = self.dropout(self.projector(norm_pooled_768))
+
+        # 4. Second Hypersphere Projection (L2 Normalize 256-D)
+        norm_projected_256 = F.normalize(projected_256, p=2, dim=-1)
+
+        # 5. Final Classification Logits
+        logits = self.classifier(norm_projected_256)
+
+        loss = None
+        if labels is not None:
+            if self.config.problem_type is None:
+                if self.num_labels == 1:
+                    self.config.problem_type = "regression"
+                elif self.num_labels > 1 and (labels.dtype == torch.long or labels.dtype == torch.int):
+                    self.config.problem_type = "single_label_classification"
+                else:
+                    self.config.problem_type = "multi_label_classification"
+
+            if self.config.problem_type == "regression":
+                loss_fct = nn.MSELoss()
+                if self.num_labels == 1:
+                    loss = loss_fct(logits.squeeze(), labels.squeeze())
+                else:
+                    loss = loss_fct(logits, labels)
+            elif self.config.problem_type == "single_label_classification":
+                loss_fct = nn.CrossEntropyLoss()
+                loss = loss_fct(logits.view(-1, self.num_labels), labels.view(-1))
+            elif self.config.problem_type == "multi_label_classification":
+                loss_fct = nn.BCEWithLogitsLoss()
+                loss = loss_fct(logits, labels)
+
+        if not return_dict:
+            output = (logits,) + outputs[2:]
+            return ((loss,) + output) if loss is not None else output
+
+        return SequenceClassifierOutput(
+            loss=loss,
+            logits=logits,
+            hidden_states=outputs.hidden_states if hasattr(outputs, "hidden_states") else None,
+            attentions=outputs.attentions if hasattr(outputs, "attentions") else None,
+        )
+
+
+# Register model & config with Auto classes
+AutoConfig.register("wav2vec2-dual-hypersphere", Wav2Vec2DualHypersphereConfig)
+AutoModelForAudioClassification.register(
+    Wav2Vec2DualHypersphereConfig, Wav2Vec2DualHypersphereForAudioClassification
+)
+
+Wav2Vec2DualHypersphereClassifier = Wav2Vec2DualHypersphereForAudioClassification
+
+__all__ = [
+    "Wav2Vec2DualHypersphereConfig",
+    "Wav2Vec2DualHypersphereForAudioClassification",
+    "Wav2Vec2DualHypersphereClassifier",
+]

preprocessor_config.json

@@ -0,0 +1,11 @@
+{
+  "do_normalize": true,
+  "feature_extractor_type": "Wav2Vec2FeatureExtractor",
+  "feature_size": 1,
+  "max_length": 192000,
+  "padding_side": "right",
+  "padding_value": 0.0,
+  "return_attention_mask": true,
+  "sampling_rate": 16000,
+  "truncation": true
+}