---
license: cc-by-nc-sa-4.0
language:
- ru
library_name: transformers
base_model:
- AigizK/w2v-bert-2.0-bashkort-russian-omnivoice
tags:
- audio
- speech
- speech-emotion-recognition
- age-estimation
- gender-classification
- russian-stress
- custom-code
---

# TeraAnaly

Russian speech analysis with one frozen w2v-BERT encoder and independently callable age, gender, emotion, stress and CTC components. Decode audio once, run the encoder once, and reuse its output for any heads you need.

**Noncommercial use only for the TeraAnaly code and trained heads**, under CC-BY-NC-SA-4.0. The original encoder weights retain their upstream CC-BY-4.0 license. See [LICENSE.md](LICENSE.md).

## Files

| File | Contents |
| --- | --- |
| `modeling_tera_analy.py` | All model code, audio decoding, frontend and CTC alignment |
| `example_inference.py` | Example calling each head; edit the variables at the top |
| `config.json` | Architecture, labels, character vocabulary and source revisions |
| `encoder.safetensors` | Frozen encoder, about 2.32 GB |
| `age.safetensors` | Age component |
| `gender.safetensors` | Gender component |
| `emotion.safetensors` | Emotion component |
| `stress.safetensors` | Vowel stress ranker |
| `ctc.safetensors` | Native 20 ms character CTC component |
| `weights_manifest.json` | File sizes, SHA-256 checksums and component licenses |

## Hugging Face inference

```python
import json
import torch
from transformers import AutoModel

MODEL_ID = "TeraSpace/TeraAnaly"
AUDIO_PATH = "audio.wav"
TEXT = "Конечно, помню! Мы так здорово проводили время!"
DEVICE = "cuda" if torch.cuda.is_available() else "cpu"

torch.set_num_threads(4)
model = AutoModel.from_pretrained(MODEL_ID, trust_remote_code=True).to(DEVICE)
waveform = model.read_wav(AUDIO_PATH)  # One decode; mono, 16 kHz.
states = model.encoder(waveform)      # One frozen encoder pass.

result = {
    "age": model.age_head(states),
    "gender": model.gender_head(states),
    "emotion": model.emotion_head(states),
    "stressed_text": model.stress_head(states, TEXT),
}
print(json.dumps(result, ensure_ascii=False, indent=2))
```

Outputs are an integer age in years, the highest gender label, the highest emotion label, and a transcript with `+` before stressed vowels. Emotion labels are `neutral`, `angry`, `positive`, `sad`, `other`; demographic labels are `female`, `male`, `child` (`child` is an age category inherited from the teacher).

`stress_head(states, text)` calls our CTC component internally, pools vowel spans from layer 16, and selects a vowel separately in each Russian word. It requires the matching transcript. It marks single-vowel words too, preserves capitalization and punctuation, and collapses whitespace. It does not require Meta UnitY2, RuAccent, or the teacher models during inference.

Optional character alignment uses the same states:

```python
logits = model.ctc_head(states)
alignment = model.ctc_head.align(states, TEXT)
```

For a single dictionary directly, use `model(waveform, TEXT)`. This also runs the encoder once.

## Batch inference

Pass a list of mono waveforms and a matching list of transcripts. Recordings may have different lengths. The encoder runs once for the entire batch; padding is masked and every result stays in input order.

```python
audio_paths = ["first.wav", "second.wav"]
texts = ["Привет, Маша!", "Как дела?"]
waveforms = [model.read_wav(path) for path in audio_paths]
states = model.encoder(waveforms)

ages = model.age_head(states)            # list[int]
genders = model.gender_head(states)      # list[str]
emotions = model.emotion_head(states)    # list[str]
stressed_texts = model.stress_head(states, texts)
```

For all four outputs as a list of dictionaries, use `results = model(waveforms, texts)`. For a large collection, process it in smaller batches; `example_inference.py` does this using `BATCH_SIZE`.

One waveform still returns a dictionary of encoder states and scalar head outputs. A list (including a one-item list) returns a list of states and one output per recording. Empty lists return empty results. CTC also supports batches: `model.ctc_head(states)` returns a list of logits trimmed to each recording's frame count, and `model.ctc_head.align(states, texts)` returns a list of alignments. Stress computes CTC emissions and vowel scores in batches, with forced alignment and word selection performed separately per transcript.

For raw waveforms at different sampling rates, use `model.encoder(waveforms, sample_rate=[16000, 22050])`, or pass one integer rate for all clips. A dense `[batch, samples]` tensor is accepted for equally sized waveforms. For different lengths, pass the unpadded waveform list so that normalization and length detection use each recording's actual audio.

Encoder attention, demographic pooling and CTC convolutions mask padding independently. Frontend, pitch statistics and forced CTC decoding are still computed per recording. Batch memory use grows with the number of recordings and the longest clip; reduce the batch size for long audio, and group similar lengths when processing a dataset. Every clip retains the 40-second limit.

## Load individual components

Download `modeling_tera_analy.py` from this repository into your script folder, then import it. Each component has its own loader:

```python
from modeling_tera_analy import (
    TeraEncoder, AgeHead, GenderHead, EmotionHead, StressHead, CTCHead, read_wav,
)

MODEL_ID = "TeraSpace/TeraAnaly"  # A local model folder also works.
DEVICE = "cuda"

encoder = TeraEncoder.from_pretrained(MODEL_ID, device=DEVICE)
age = AgeHead.from_pretrained(MODEL_ID, device=DEVICE)
gender = GenderHead.from_pretrained(MODEL_ID, device=DEVICE)
emotion = EmotionHead.from_pretrained(MODEL_ID, device=DEVICE)
stress = StressHead.from_pretrained(MODEL_ID, device=DEVICE)

states = encoder(read_wav("audio.wav"))
age_years = age(states)
gender_label = gender(states)
emotion_label = emotion(states)
stressed_text = stress(states, "Привет, Маша!")
```

Load only the components you need. An age, gender or emotion loader reads its corresponding safetensors and configuration. The encoder loader reads only encoder weights and configuration. The stress loader reads stress and CTC weights; it does not load or rerun the encoder. `CTCHead.from_pretrained(...)` is available independently too.

`states` is a dictionary containing `hidden_states` indexed by layers 4, 8, 16 and 24, `recording_features`, `spectral_features`, `raw_features`, `frame_count` and `frame_seconds`. The existing trained heads also use frontend and prosody statistics; these are computed once by `encoder(...)` alongside the neural encoder. Pass the entire dictionary to a head.

## Local example and dependencies

Edit `MODEL_ID`, `AUDIO_PATH`, `TEXT`, `DEVICE` and `BATCH_SIZE` at the top of `example_inference.py`, then run it. For batch inference, set `AUDIO_PATH` to a list of WAV paths and `TEXT` to their matching transcript list. It prints one dictionary for a single path, or a list of dictionaries for a path list. There are no command-line arguments. Relative audio paths are resolved relative to that file; absolute paths work too. Supply your own WAV file; recordings are not bundled on Hugging Face.

For offline use, download the repository, set `MODEL_ID` to that folder, and import `TeraAnaly` or its individual components directly. **Local computation requires only Python's standard library and PyTorch**, including a built-in safetensors reader. Hugging Face downloads require `huggingface_hub`; `AutoModel` requires `transformers`. No `torchaudio`, NumPy, `safetensors`, or external audio program is used by the model code.

Tested with Python 3.10, PyTorch 2.9.0 and Transformers 5.15.1. This release uses float32 inference. CPU and one CUDA device are supported; clips are limited to 40 seconds. The reader accepts RIFF WAV with PCM 8/16/24/32-bit or IEEE float32 audio, averages channels and resamples to 16 kHz. Alternatively pass a mono PyTorch waveform to `encoder(waveform, sample_rate=...)`. Use Russian transcripts and spell out numbers.

## Training and scope

The shared encoder comes from [AigizK/w2v-bert-2.0-bashkort-russian-omnivoice](https://huggingface.co/AigizK/w2v-bert-2.0-bashkort-russian-omnivoice), revision `e4c4278904c6359e9ac769cf04575be68f99ed4e`, based on [facebook/w2v-bert-2.0](https://huggingface.co/facebook/w2v-bert-2.0). TeraAnaly training kept this encoder frozen. The unused upstream ASR adapter and ASR output layer are omitted from the export; retained encoder tensor values are unchanged.

Age and gender training used predictions from [audeering/wav2vec2-large-robust-24-ft-age-gender](https://huggingface.co/audeering/wav2vec2-large-robust-24-ft-age-gender), together with available speaker gender labels. Emotion training used compatible recorder labels from Russian emotional dialogues and predictions from [xbgoose/hubert-large-speech-emotion-recognition-russian-dusha-finetuned](https://huggingface.co/xbgoose/hubert-large-speech-emotion-recognition-russian-dusha-finetuned). The training pool also included Russian ewebis, epodcasts and studio recordings. Stress training used RuAccent-derived weak labels and frozen audio embeddings; the trained native CTC component supplies timings at inference.

Age, gender and emotion are model estimates. Stress agreement with automatic RuAccent labels measures agreement with those labels, rather than verified spoken stress accuracy. There is no independent human-labeled spoken-stress accuracy claim for this release.

The export was checked against the existing inference pipeline on a female and a male recording: all four returned outputs matched. Independent encoder loading produced identical hidden states, and separately loaded heads reproduced the bundled heads. Each sample used one encoder pass and stress called CTC internally once. Both local `AutoModel(..., trust_remote_code=True)` and local execution with optional dependencies disabled were checked. These checks establish export equivalence, not dataset-wide accuracy.

Batch checks compared mixed-length recordings against single inference, including reordered and repeated items, one-item batches, dense waveform tensors, mixed sampling rates and empty inputs. Returned age, gender, emotion and stressed text matched the single-recording results. The encoder, CTC emissions and vowel ranker each ran once per batch. Padding equivalence was also checked with IEEE float32 convolution kernels; CUDA's default TF32 kernels can produce small floating-point differences between single and batch computation.

## Attribution and license

Copyright 2026 TeraSpace for the new code and trained heads. These are licensed under [CC-BY-NC-SA-4.0](https://creativecommons.org/licenses/by-nc-sa/4.0/). Preserve attribution and distribute adaptations under the same license. The original encoder remains [CC-BY-4.0](https://creativecommons.org/licenses/by/4.0/), with attribution to its upstream authors; this exception does not grant commercial use of the TeraAnaly heads or code. Component details and teacher notices are in [LICENSE.md](LICENSE.md), and the full noncommercial license text is in [LICENSE](LICENSE).