Multimodal: open-source projects on GitHub
143 repositories on Multimodal, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
phi - Simple Phi-4 Launch
Run the powerful multimodal Phi-4 model directly on your Mac with Apple Silicon — no cloud, no complex setup.
★ 280 · Jupyter NotebookNAVIT Any‑Format Vision Transformer
Implementation of the Patch n’ Pack paper: NaViT – a Vision Transformer that works with any aspect ratio and resolution without padding.
★ 276 · PythonLongVT — Think About Video Foundations
A model that learns to 'think' about long videos through built‑in tools for CVPR 2026.
★ 274 · Python🔍 LIGHTHOUSE: VIDEO MOMENT RETRIEVAL
User-friendly library for reproducible video moment retrieval and highlight detection. Also supports audio. Published at EMNLP2024 Demo, ICASSP 2025 and ICASSP 2026.
★ 274 · PythonMULTIMODAL DECISION MODEL
OneJev — multimodal System One model for calibrated answers to questions about screens, photos, video and text in one forward pass.
★ 272 · PythonLLAMA_ROS ROS 2 LLM
The project combines llama.cpp and llava.cpp with ROS 2, allowing the use of GGUF models for real-time robotics.
★ 268 · C++OPTIMIZING PROMPTS FOR SEEDANCE 2.5
More than 100 verified Seedance 2.5 prompts with real video previews. Built-in Agent Skill optimizes prompts, creates storyboards, and generates videos using Seedance mod
★ 268 · TypeScriptBENCHMARKS FOR CODE FROM LLM
Awesome-Code-Benchmark — an overview of key benchmarks for evaluating language models' coding abilities.
★ 250SCREENMIND - YOUR SCREEN MEMORY
Remembers your screen, searches by content, and chats with AI locally.
★ 249 · PythonUNIMMAD ANOMALY DETECTION IN MULTIMEDIA MODEL
UniMMAD offers a unified approach to anomaly detection in multimedia data by combining images, audio, and text. Using mixture‑of‑experts, it performs feature decompressio
★ 248 · PythonPIE — MODEL SERVING ON RUST
Programmable engine for high-performance LLM and diffusion model serving. Written in Rust, supports batching, KV caching, and dynamic adapter loading.
★ 240 · RustAWESOME AGENTIC WORLD MODELING
The project demonstrates the foundations of agentic world modeling: capabilities, laws, and further development in TypeScript.
★ 235 · TypeScriptCELESTE-PYTHON: Multimodal AI Primitives
Type-safe primitives for multimodal AI: work with any models and providers via a unified interface.
★ 228 · PythonComfyUI Fal API: 1400+ Models
Production-ready custom nodes for ComfyUI, providing access to 1400+ fal.ai models with automatic updates, caching, and cost control.
★ 227 · PythonDSH-CREW: PLUGIN FOR CLAUDE CODE AND CODEX
dsh-crew is a DeepSeek Harness plugin to run agents from Claude Code and Codex. Supports native subagents with progress tracking, worker sessions with presets, and a mult
★ 215 · JavaScriptGRACE: INT4 QWEN3-VL DISTILL
Qwen3-VL in INT4 with quantization-aware distillation – ready for deployment.
★ 213 · Jupyter NotebookLOCAL MULTIMODAL CHAT
Self-hosted chat with local LLMs (Ollama/OpenAI), supporting PDF RAG, image chat, and voice input via Whisper.
★ 209 · PythonFUSILLI MULTIMODAL FUSION
Fusilli — Python package with ready-made multimodal fusion pipelines: from data loading to model training and evaluation.
★ 205 · PythonMGM-OMNI: FUTURE AI VOICE
MGM-Omni enables scaling large language models for personalized long-form speech, ensuring coherent synthesis for hours ahead.
★ 202 · PythonNOOBOT CHEAP AI AGENT
Self‑hosted workspace for AI agents with low cost: support for tool calling, MCP, multi‑model routing and sandboxed execution. Enables creating multi‑agent workflows and
★ 195 · JavaScriptOpenSearch-VL: Multimodal Search
Open recipe for training deep multimodal search agents with data curation, visual tools, and RL.
★ 190 · PythonSystem 1 Model Framework
Framework to train, test, and serve System 1 decision models on language and multimodal platforms.
★ 137 · PythonLLM Intern Skill
This project provides a set of LLM-based tools for improving resumes, matching jobs, fact-checking experience, and preparing for interviews.
★ 96 · MarkdownNew projects on this topic in Telegram
Telegram →