R_REDDYX.XYZ
MultimodalPython★ 10,312⑂ 1,649 forksApache-2.0

Lightricks/LTX-2

LTX-2: DIT MODEL FOR SYNCHRONIZED AUDIO AND VIDEO

LTX-2 is a DiT-based foundation model for generating synchronized audio and video. The package includes Python inference and LoRA training for production use.

// KEY FEATURES

  • Synchronized audio and video generation in a single DiT model
  • Two modes: fast DistilledPipeline and quality DFR with IC-LoRA
  • FP8 quantization and CPU/disk offload support to save VRAM
  • Includes Gemma4-12B text encoder, 22B transformer, VAEs and upscaler

// DEEP DIVE: HOW IT WORKS

LTX-2 uses a Diffusion Transformer (DiT) architecture to generate both video and audio simultaneously from text prompts. The pipeline loads a 22B parameter distilled transformer, Gemma4-12B text encoder, separate video and audio VAEs, and a latent spatial upscaler. Two inference paths exist: DistilledPipeline for speed and DFR (Diffusion Fidelity Rendering) for production quality, which adds IC-LoRA detailing passes and extra keyframes.

Unlike models that handle video and audio separately, LTX-2 unifies them in a single foundation model. The DFR mode reuses the same distilled transformer but applies a spatial detailing pass with generated keyframes, increasing VRAM and runtime. The package uses uv for dependency management and supports neighborhood attention via natten on Linux/CUDA, falling back to Triton or eager implementations elsewhere. Models total ~66 GB and require Hugging Face authentication.

Requires GPU with sufficient VRAM (22B transformer plus VAEs); quantization fp8-cast and offload to CPU or disk mitigate memory pressure. The repository is gated on Hugging Face, needing a read token with gated repo scope. Linux with CUDA enables the fastest natten backend; Windows and macOS fall back to slower decoding. No API keys beyond HF auth mentioned.

// USE CASES

Best for: ML engineers and researchers for creating audiovisual content

  • Create synchronized videos with speech and sound effects from text descriptions
  • Generate production-quality clips via DFR mode with keyframe detailing
  • Train LoRA adapters to fine-tune the model for specific styles or characters

// QUICK START

git clone https://github.com/Lightricks/LTX-2.git
cd LTX-2
uv sync --extra natten
hf auth login

Commands are taken from the repository README — check the current version on GitHub before running.

#generative-ai#ltx#ltx-2
Open on GitHub →

New repositories every 30 minutes

REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.

Join on Telegram
← Full catalog·Full index

// SIMILAR REPOSITORIES

EMBEDDED VECTOR DB FOR MULTIMODAL AI★ 11947REMOVING AI WATERMARKS AND METADATA★ 6403OPEN SOURCE AI WORKSPACE FOR CREATORS★ 14387MULTIMODAL DECISION MODEL★ 918OPEN CREATIVE STUDIO ON COMFYUI★ 233PROFESSIONAL AI ENGINE FOR VISUAL MEDIA★ 29106

← FULL CATALOG

Lightricks/LTX-2 — Ltx-2: Dit Model For Synchronized Audio…