GGUF
NeMo
nemo-speech.cpp
asr
speech-recognition
speech-to-text
transcription
ggml
llama.cpp
nvidia
nemotron
nemotron-3.5
cuda
cuda-12.8
cuda-binary
binary
prebuilt-binary
inference-server
turing
turing-gpu
t4
tesla-t4
kaggle
colab
Instructions to use masterofaudio2077/nemo-speech-cpp-cuda-t4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- NeMo
How to use masterofaudio2077/nemo-speech-cpp-cuda-t4 with NeMo:
# tag did not correspond to a valid NeMo domain.
- Notebooks
- Google Colab
- Kaggle
Update README.md
Browse files
README.md
CHANGED
|
@@ -1,6 +1,7 @@
|
|
| 1 |
---
|
| 2 |
license: other
|
| 3 |
library_name: nemo-speech.cpp
|
|
|
|
| 4 |
tags:
|
| 5 |
- asr
|
| 6 |
- gguf
|
|
@@ -12,7 +13,14 @@ tags:
|
|
| 12 |
---
|
| 13 |
# nemo-speech (CUDA build for Tesla T4)
|
| 14 |
|
| 15 |
-
Prebuilt `nemo-speech` server from NeMo-Speech.cpp, so Kaggle/Colab notebooks can skip the ~40 min compile.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 16 |
|
| 17 |
## Build
|
| 18 |
| | |
|
|
@@ -32,9 +40,9 @@ os.chmod("/tmp/nemo-bin/bin/nemo-speech", 0o755)
|
|
| 32 |
# libraries sit next to the binary
|
| 33 |
os.environ["LD_LIBRARY_PATH"] = "/tmp/nemo-bin/bin:" + os.environ.get("LD_LIBRARY_PATH", "")
|
| 34 |
```
|
| 35 |
-
Then run `/tmp/nemo-bin/bin/nemo-speech serve --asr-model <model.gguf> ...`.
|
| 36 |
|
| 37 |
## Limits
|
| 38 |
- Compiled for compute capability 7.5 only. It will not run on other GPUs (A100, L4, P100).
|
| 39 |
- Needs an NVIDIA driver with `libcuda.so.1` and a glibc/libstdc++ at least as new as the build machine's (see `ldd.txt`).
|
| 40 |
-
- Licence: follows the NeMo-Speech.cpp, ggml and CUDA licences
|
|
|
|
| 1 |
---
|
| 2 |
license: other
|
| 3 |
library_name: nemo-speech.cpp
|
| 4 |
+
base_model: nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B
|
| 5 |
tags:
|
| 6 |
- asr
|
| 7 |
- gguf
|
|
|
|
| 13 |
---
|
| 14 |
# nemo-speech (CUDA build for Tesla T4)
|
| 15 |
|
| 16 |
+
Prebuilt `nemo-speech` server from NeMo-Speech.cpp, so Kaggle/Colab notebooks can skip the ~40 min compile. It runs [NVIDIA Nemotron-3.5-Transcribe-30B-A4B](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B) (speech recognition) from the GGUF conversion [AtomicChat/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-GGUF](https://huggingface.co/AtomicChat/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-GGUF). This repo contains the binary only, not the model weights.
|
| 17 |
+
|
| 18 |
+
## Parent model
|
| 19 |
+
| | |
|
| 20 |
+
|---|---|
|
| 21 |
+
| Original model | [nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B) |
|
| 22 |
+
| GGUF weights used | [AtomicChat/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-GGUF](https://huggingface.co/AtomicChat/NVIDIA-Nemotron-3.5-Transcribe-30B-A4B-GGUF) |
|
| 23 |
+
| Inference engine | NeMo-Speech.cpp (ggml / llama.cpp based) |
|
| 24 |
|
| 25 |
## Build
|
| 26 |
| | |
|
|
|
|
| 40 |
# libraries sit next to the binary
|
| 41 |
os.environ["LD_LIBRARY_PATH"] = "/tmp/nemo-bin/bin:" + os.environ.get("LD_LIBRARY_PATH", "")
|
| 42 |
```
|
| 43 |
+
Then run `/tmp/nemo-bin/bin/nemo-speech serve --asr-model <model.gguf> ...`, with the GGUF files downloaded from the repo above.
|
| 44 |
|
| 45 |
## Limits
|
| 46 |
- Compiled for compute capability 7.5 only. It will not run on other GPUs (A100, L4, P100).
|
| 47 |
- Needs an NVIDIA driver with `libcuda.so.1` and a glibc/libstdc++ at least as new as the build machine's (see `ldd.txt`).
|
| 48 |
+
- Licence: the model is under NVIDIA's own licence (see the parent model page); the binary follows the NeMo-Speech.cpp, ggml and CUDA licences. Check them before redistributing.
|