oruk-loop-153M / NOTICE.md
NathanRoll's picture
oruk-loop-153M: weights, inference code, model card, licenses, results, media
58b0065 verified
|
Raw History Blame Contribute Delete
3.04 kB

Attribution and license scope

oruk-loop-153M is a derivative of Qwen3-ASR-1.7B (Qwen/Qwen3-ASR-1.7B-hf), © Alibaba Cloud / the Qwen team, distributed under the Apache License 2.0 (full text in LICENSE-QWEN). The following come from that model:

  • the architecture and configuration (config.json, apart from the oruk_loop block and auto_map);
  • the initial values of every looped weight: each shared weight started as the mean of the teacher's layers, and each per-depth adapter as a truncated SVD of that layer's difference from the mean;
  • the modules kept unchanged: the convolutional audio front-end, the encoder output LayerNorm, the audio projector and the final decoder RMSNorm;
  • the token table's exact rows and the initial values of its factorised rows;
  • tokenizer.json, tokenizer_config.json, chat_template.jinja, processor_config.json and the generation settings.

Oruk AI changed the following: each transformer stack was replaced by one shared looped block with per-depth adapters; the vocabulary was trimmed and factorised; the model was distilled from Qwen3-ASR-1.7B, weight-averaged and MWER fine-tuned; and the inference code and evaluation were written.

Licenses in this repository

  • model.safetensors (the oruk-loop-153M weights): PolyForm Small Business 1.0.0 or PolyForm Noncommercial 1.0.0, at the user's option; other commercial use needs a commercial license from Oruk AI. See LICENSE-WEIGHTS.md. Keep its Required Notice and the Qwen3-ASR attribution above when redistributing.
  • modeling_oruk_loop.py: MIT, see LICENSE.
  • Unmodified Qwen3-ASR files (tokenizer, chat template, processor configuration): Apache-2.0, see LICENSE-QWEN.
  • results/ and media/: CC BY 4.0, attributed to Oruk AI.

Training data credits

Training audio came from the training splits of the following corpora. Their copyrights and terms remain with their owners.

  • LibriSpeech (Panayotov et al.; OpenSLR 12): CC BY 4.0.
  • SPGISpeech (Kensho Technologies): Kensho Terms of Usage, which permit academic research and internal use.
  • GigaSpeech (SpeechColab): access for non-commercial research and educational purposes. Its card notes that speech recognition models trained on it may be eligible for commercial licensing.
  • VoxPopuli (Meta AI): CC0 1.0.
  • TED-LIUM release 3 (LIUM): CC BY-NC-ND 3.0.
  • AMI Meeting Corpus (University of Edinburgh et al.): CC BY 4.0.
  • Earnings-22 (Rev.com), train split of sanchit-gandhi/earnings22_split: CC BY-SA 4.0.
  • YODAS3 English (ESPnet / CMU WAVLab): CC BY 3.0, used in an early warm-up stage.
  • FLEURS en_us train (Google): CC BY 4.0, used in an early warm-up stage.

Several of these terms restrict commercial use of the data itself. Whether and how they bear on models trained on the data is for each user to assess for their own use. No endorsement by Alibaba Cloud, the Qwen team or the dataset providers is implied.