|
Download README.md from WhissleAI/arc-transcriber-space: direct link, hf CLI and curl.
- Browser
- Download file 9.4 kB
-
https://huggingface.co/WhissleAI/arc-transcriber-space/resolve/main/README.md
- Command line
-
hf download hf://WhissleAI/arc-transcriber-space/README.md
-
curl -L -o README.md https://huggingface.co/WhissleAI/arc-transcriber-space/resolve/main/README.md
9.4 kB
| license: cc-by-nc-4.0 | |
| language: | |
| - en | |
| library_name: arc-transcriber | |
| pipeline_tag: automatic-speech-recognition | |
| base_model: nvidia/stt_en_fastconformer_hybrid_large_streaming_multi | |
| tags: | |
| - speech-recognition | |
| - streaming | |
| - named-entity-recognition | |
| - entity-linking | |
| - space | |
| - moon | |
| - onnx | |
| # Arc Transcriber Space | |
| Arc Transcriber Space is a streaming English speech model for space and Moon-exploration apps. In a single pass it gives: | |
| - **Transcript words**, from a streaming FastConformer encoder with an RNNT decoder. | |
| - **Inline entity tags** on the words: lunar features, sites, celestial bodies, spacecraft, missions, crew, numbers and | |
| more. Each tag is checked by a span head. | |
| - **Name linking to the app's own list** (`names`), so "show me tycho" returns your `Tycho` entry. | |
| - **Long context**: a names list, previous text and session memory. | |
| - **Audio events and device-directed speech**. | |
| The encoder is an int8 ONNX graph (cache-aware streaming, 560 ms chunks). It runs on a CPU. | |
| ## Results | |
| Final numbers from the [Arc Audio report](https://whissleai.github.io/Outpost-Luna/arc-audio.html), which has the full | |
| details: how the test sets were built, how leaks were removed, and per-tag scores. | |
| **Test data.** The main benchmark is a NASA speech corpus collected by Team Outpost Luna from NASA public audio | |
| (Artemis audio and the podcasts Houston We Have a Podcast, Small Steps, Giant Leaps and Curious Universe). **This model | |
| was not trained on any of it**: it is a held-out test set here, and its training split is reserved for the next model. | |
| The podcast test set is 19 whole episodes (12.8 h, 10,410 segments); the Artemis test set is 15 whole recordings (1,141 | |
| utterances) kept out of every training set. Every test set is scored in full. | |
| ### Word error rate | |
| Streaming, against the NVIDIA FastConformer this model started from | |
| (`nvidia/stt_en_fastconformer_hybrid_large_streaming_multi`). Both models run streaming and are scored with the same text | |
| normalisation. Lower is better. | |
| | Test set | Size | NVIDIA base | Arc Transcriber Space | | |
| |---|---|---|---| | |
| | **NASA speech corpus (held out)** | | | | | |
| | Houston We Have a Podcast | 11 episodes | 10.0% | 7.2% | | |
| | Small Steps, Giant Leaps | 5 episodes | 9.7% | 6.4% | | |
| | Curious Universe | 3 episodes | 6.0% | 4.8% | | |
| | All podcast parts | 19 episodes, 12.8 h | 9.7% | 7.0% | | |
| | Artemis audio | 1,141 utterances | 28.5% | 12.6% | | |
| | **Other test sets** | | | | | |
| | Apollo mission audio | 500 utterances | 35.5% | 15.7% | | |
| | Open read speech (children's corpus) | 300 utterances | 42.0% | 18.5% | | |
| | Space questions (synthetic speech) | 300 utterances | 48.1% | 9.1% | | |
| | Common Voice 17 English | 2,000 clips | 12.6% | 17.1% | | |
| | CREMA-D (acted adult speech) | 600 clips | 3.7% | 1.9% | | |
| On Common Voice the base model is better (12.6% vs 17.1%): tuning for space speech costs some everyday dictation. Every | |
| other test set improves. | |
| ### Names: hearing and linking them | |
| The app's own path: the int8 ONNX encoder on a CPU, full test sets. Each utterance gets a names list made of its own | |
| names plus 20 similar-sounding decoys. On sets with no names, any link is a false link. The base model cannot link names. | |
| | Test set | Name recall | Names linked correctly | Linked to a wrong entry | Space-tag F1 | | |
| |---|---|---|---|---| | |
| | Artemis II crew audio (1,141 utterances) | 93.4% | 94.5% | 1.0% | 0.88 | | |
| | Space questions, synthetic speech (300) | 82.8% | 91.4% | 0.0% | 0.83 | | |
| | Open read speech, children's corpus (300) | – | 0.67 false links per 100 utterances (no names in this set) | – | – | | |
| | Apollo mission audio (500) | – | 0.4 false links per 100 utterances (no gold names in this set) | – | – | | |
| Inline tag precision / recall / F1 (gold tag count in brackets): | |
| | Tag | Artemis II crew audio (1,141) | Space questions (300) | | |
| |---|---|---| | |
| | ENTITY_LUNAR | .88 / .88 / .88 (584) | .88 / .81 / .84 (124) | | |
| | ENTITY_SPACECRAFT | .86 / 1 / .92 (6) | .83 / .83 / .83 (29) | | |
| | ENTITY_MISSION | – | .81 / .79 / .80 (43) | | |
| | ENTITY_CELESTIAL | .92 / .97 / .94 (34) | 1 / 1 / 1 (2) | | |
| | ENTITY_TERM | .82 / .83 / .82 (115) | 1 / 1 / 1 (2) | | |
| | ENTITY_PERSON | .22 / .94 / .36 (47) | .13 / 1 / .22 (1) | | |
| | NUMBER | .51 / .67 / .58 (162) | .88 / 1 / .93 (7) | | |
| | NUMBER_PERCENT | 0 / 0 / 0 (5) | 1 / 1 / 1 (13) | | |
| | NUMBER_TIME | .18 / .06 / .09 (36) | – | | |
| | NUMBER_MEASURE | 0 / 0 / 0 (14) | 0 / 0 / 0 (1) | | |
| | All space name tags | .88 / .88 / .88 (624) | .86 / .81 / .83 (198) | | |
| Weak spots: ENTITY_PERSON fires too often (low precision), and number kinds (time, measure) are rarely right. The four | |
| space name tags (lunar, spacecraft, mission, celestial), the ones linked to the app's names lists, are the strong ones. | |
| **Overlap with training.** Common Voice and CREMA-D are test only. Open read speech uses the corpus's own split (trained on | |
| its train split only). The space questions and Apollo test utterances are held out. Artemis II crew audio (a separate | |
| training source from the NASA corpus above) is split by whole recording: other recordings are in training, the 15 test | |
| recordings never are. | |
| ### Speed | |
| On CPU with 4 threads, the encoder and RNNT run at about 0.13× real time with a peak of about 2.3 GB RAM, model | |
| included. | |
| ## Install | |
| The code is in the [arc_transcriber](https://github.com/WhissleAI/arc_transcriber) package; use the `space` extra. | |
| ```bash | |
| pip install "arc-transcriber[space] @ git+https://github.com/WhissleAI/arc_transcriber" | |
| ``` | |
| A C library is also available (see `c/README.md` in the same repository). It uses the ONNX Runtime C API for the encoder: | |
| ```bash | |
| git clone https://github.com/WhissleAI/arc_transcriber && cd arc_transcriber/c | |
| make ARC_ORT=/path/to/onnxruntime # builds libarc + arc-nft2 | |
| ``` | |
| ## Use | |
| ```python | |
| import soundfile as sf | |
| from arc_audio import Arc | |
| arc = Arc.from_pretrained("WhissleAI/arc-transcriber-space", backbone="nft2", device="cpu", threads=4) | |
| NAMES = [ # what is on screen / in the current lesson, most relevant first | |
| {"name": "Tycho", "kind": "lunar_feature"}, | |
| {"name": "Apollo 12", "kind": "mission"}, | |
| {"name": "Ocean of Storms", "kind": "lunar_feature", "aliases": ["Oceanus Procellarum"]}, | |
| {"name": "Orion", "kind": "spacecraft"}, | |
| {"name": "Artemis II", "kind": "mission", "aliases": ["Artemis 2"]}, | |
| ] | |
| wav, sr = sf.read("examples/samples/sample_2.wav", dtype="float32") # 16 kHz mono | |
| out = arc.predict(wav, domain="assistant", transcript="nft2", names=NAMES) | |
| print(out["transcript"][0]["text"]) # "Show me the Apollo twelve landing site in the ocean of storms" | |
| for n in out["names"]: | |
| print(n["heard"], "->", n["entity"], n["state"]) # "Apollo" -> "Apollo 12" linked ; "ocean of storms" -> "Ocean of Storms" linked | |
| # streaming (microphone): feed 16 kHz float32 chunks, change the list at any time | |
| s = arc.stream(domain="assistant", transcript="nft2", names=NAMES) | |
| for chunk in mic_chunks(): # e.g. 100 ms each | |
| for update in filter(None, [s.feed(chunk)]): | |
| ... # partial / final transcript, names, events | |
| s.set_context(names=NEW_SCREEN_NAMES, previous_text="...") | |
| final = s.flush() | |
| ``` | |
| Each item in `out["names"]` has these fields: | |
| - `tag`: the entity type, e.g. `entity.mention.lunar_feature` | |
| - `heard`: the words in spoken form, e.g. `apollo twelve` | |
| - `entity`: your list entry's `name`, e.g. `Apollo 12` | |
| - `state`: one of | |
| - `linked`: act on `entity` | |
| - `abstain`: ambiguous or generic, so ask the user | |
| - `not_in_list`: a name the user said that is not in your list | |
| - `p`: confidence | |
| ## Names lists: read this before integrating | |
| Linking compares normalized text: | |
| - lower case; | |
| - digits become spoken words: `Apollo 12` → `apollo twelve`, `1969` → `nineteen sixty nine`; | |
| - `-` becomes a space; | |
| - **accented and other non-ASCII letters are dropped**. | |
| Rules for building lists: | |
| - Always give a `kind`. | |
| - Put the most relevant names first. | |
| - Keep lists short. | |
| - Add aliases for names that are not said the way they are written: `Artemis II` → `"Artemis 2"`, `S-IVB` → `"S 4 B"`. | |
| - Add ASCII aliases for non-ASCII names: `Schrödinger` → `"Schrodinger"`. | |
| The full guide is in [NAMES_LISTS.md](NAMES_LISTS.md). | |
| The transcript is in spoken form ("apollo twelve"). For display and routing, use `names[i].entity`, which is your | |
| list's own spelling. | |
| ## Notes | |
| - Input is 16 kHz mono audio. | |
| - **The model is built not to transcribe the app's own spoken answers**: text-to-speech playback leaking into the | |
| microphone. Synthetic TTS voices, especially female Kokoro-style voices, can therefore be partly or fully ignored. Test | |
| with real voices, not TTS. Real children's speech is transcribed normally. | |
| - The `examples/samples/` clips are synthetic speech made with Kokoro-82M (Apache-2.0). They contain no recording of a | |
| real person. | |
| ## Licence | |
| © 2026 Whissle Inc. The model weights are licensed under CC BY-NC 4.0 together with the Whissle terms in | |
| [LICENSE](LICENSE), for non-commercial use only. Commercial licences: ksingla@whissle.ai, https://whissle.ai. | |
| Third-party components are listed in [NOTICE](NOTICE): | |
| - a modified NVIDIA FastConformer (`nvidia/stt_en_fastconformer_hybrid_large_streaming_multi`, CC-BY-4.0); | |
| - a modified Moonshine Streaming decoder (MIT; its licence is kept in `nft2/decoder/LICENSE`). | |
| This model is not affiliated with or endorsed by NASA or NVIDIA. | |