Lightricks/LTX-2
LTX-2: DIT MODEL FOR SYNCHRONIZED AUDIO AND VIDEO
LTX-2 is a DiT-based foundation model for generating synchronized audio and video. The package includes Python inference and LoRA training for production use.
// KEY FEATURES
- Synchronized audio and video generation in a single DiT model
- Two modes: fast DistilledPipeline and quality DFR with IC-LoRA
- FP8 quantization and CPU/disk offload support to save VRAM
- Includes Gemma4-12B text encoder, 22B transformer, VAEs and upscaler
// DEEP DIVE: HOW IT WORKS
LTX-2 uses a Diffusion Transformer (DiT) architecture to generate both video and audio simultaneously from text prompts. The pipeline loads a 22B parameter distilled transformer, Gemma4-12B text encoder, separate video and audio VAEs, and a latent spatial upscaler. Two inference paths exist: DistilledPipeline for speed and DFR (Diffusion Fidelity Rendering) for production quality, which adds IC-LoRA detailing passes and extra keyframes.
Unlike models that handle video and audio separately, LTX-2 unifies them in a single foundation model. The DFR mode reuses the same distilled transformer but applies a spatial detailing pass with generated keyframes, increasing VRAM and runtime. The package uses uv for dependency management and supports neighborhood attention via natten on Linux/CUDA, falling back to Triton or eager implementations elsewhere. Models total ~66 GB and require Hugging Face authentication.
Requires GPU with sufficient VRAM (22B transformer plus VAEs); quantization fp8-cast and offload to CPU or disk mitigate memory pressure. The repository is gated on Hugging Face, needing a read token with gated repo scope. Linux with CUDA enables the fastest natten backend; Windows and macOS fall back to slower decoding. No API keys beyond HF auth mentioned.
// USE CASES
Best for: ML engineers and researchers for creating audiovisual content
- Create synchronized videos with speech and sound effects from text descriptions
- Generate production-quality clips via DFR mode with keyframe detailing
- Train LoRA adapters to fine-tune the model for specific styles or characters
// QUICK START
git clone https://github.com/Lightricks/LTX-2.git
cd LTX-2
uv sync --extra natten
hf auth loginCommands are taken from the repository README — check the current version on GitHub before running.
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram