R_REDDYX.XYZ
VoicePython★ 18,764⑂ 3,696 forksApache-2.0

NVIDIA-NeMo/Speech

NVIDIA NEMO SPEECH — FRAMEWORK FOR SPEECH AI AND MULTIMODAL LLMS

Scalable framework for ASR, TTS, and speech LLMs. Enables creating, customizing, and deploying speech models with ready checkpoints.

// KEY FEATURES

  • Nemotron 3.5 Transcribe: offline LLM for 40 languages up to 1 hour audio
  • Streaming ASR with controllable latency 80 ms–1 s on FastConformer
  • MagpieTTS v2607: 12 languages, Nemotron 3 VoiceChat — full-duplex dialogue
  • Runs on top of your Python/PyTorch/CUDA without replacing the stack

// DEEP DIVE: HOW IT WORKS

NeMo Speech 3.0 is a PyTorch framework for training and inference of speech models: ASR, TTS, speaker diarization/recognition, and speech LLMs. Architecture is modular: YAML configs, module classes, distributed training support via PyTorch Lightning. Checkpoints distributed via Hugging Face and NGC containers.

Key feature — Nemotron family: unified checkpoints for offline/streaming modes with latency control (Nemotron-Speech-Streaming, Parakeet-unified). MagpieTTS covers 12 languages. VoiceChat implements full-duplex on Nemotron Nano v2 backbone. Production-ready NIM microservices available.

Requires Python 3.12+, PyTorch 2.7+, NVIDIA GPU with CUDA for training (recommended for inference). Installs on top of existing environment without replacing PyTorch. Tested builds: Python 3.13 + PyTorch 2.11/CUDA 12.9 or 2.12/CUDA 13.2. Repository since 2026 focuses only on audio/speech/multimodality.

// USE CASES

Best for: Researchers and developers of Speech AI, ASR/TTS, multimodal LLMs

  • Train and deploy streaming ASR with sub-100 ms latency for call center
  • Generate speech in 12 languages via MagpieTTS for multilingual product
  • Run offline transcription of hour-long audio with speaker attribution and translation
#asr#deeplearning#generative-ai#machine-translation#neural-networks#speaker-diariazation#speaker-recognition#speech-synthesis#speech-translation#tts
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

OFFLINE SPEECH PROCESSING ON ONNXRUNTIME★ 16007LOCAL VOICE DICTATION FOR ALL OS★ 6802DOCKER WRAPPER KOKORO-82M TTS WITH OPENAI API★ 5706VIETNAMESE TTS WITH ON-DEVICE INSTANT VOICE CLONING★ 3432SMALL ENGLISH TTS MODEL 4.5M PARAMETERS★ 673LOCAL SPEECH-TO-SPEECH ASSISTANT ON GPU★ 448

← FULL CATALOG

NVIDIA-NeMo/Speech — Nvidia Nemo Speech — Framework For…