Vision Language Model: open-source projects on GitHub
28 repositories on Vision Language Model, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
RUN VLM LOCALLY ON MAC
mlx-vlm runs and fine-tunes vision-language models (LLaVA, Florence-2, Idefics) on Apple Silicon via MLX. Everything works locally on your Mac without GPU servers.
★ 5,847 · PythonMLX-VLM: RUN VLM ON MAC
Package for running and fine‑tuning Vision Language Models on Mac via MLX. 4.7K★
★ 4,787 · PythonLMMS-EVAL - ONE-STOP MODALITY EVAL
Unified toolkit for evaluating multimodal models on text, image, video, and audio tasks. 4169 stars on GitHub.
★ 4,169 · PythonOGAM Offline AI Swiss Knife
OGAM — chat, vision, speech, and image generation offline on your device without keys or network data.
★ 3,380 · TypeScriptQWEN-VL SERIES FINETUNE OPEN SOURCE
Open-source implementation for fine-tuning Qwen-VL series models from Alibaba Cloud in Python. Allows easy adaptation of multimodal models for custom tasks.
★ 1,983 · PythonMAGE: AI AGENT FOR PC CONTROL
Microsoft released Mage — an agent for GUI control via flow matching. The model edits images and performs computer tasks without fine-tuning.
★ 1,782 · PythonOVIS: Visual-Text Sync
A modern MLLM architecture for structural alignment of visual and text embeddings.
★ 1,539 · PythonTHEPIPE: DATA FROM DOCUMENTS FAST
ThePipe uses vision‑language models to extract data from PDF, Word and PowerPoint. It recognises tables, text and images, turning them into ready‑to‑use structures. Suppo
★ 1,530 · PythonVISION AND LANGUAGE: TOP MODELS
A collection of architectures of well-known vision‑language models with explanations and links.
★ 1,368 · MarkdownVisual Toolkit for LLM
A library for text models that lets them 'see' images: recognize content, answer questions, restore UI, automate GUI, and integrate with agents.
★ 1,262 · Pythonvllm-mlx LLM Server
OpenAI and Anthropic compatible server for Apple Silicon that runs LLMs and vision models with continuous batching and MCP tool calling, achieving 400+ tokens/s.
★ 1,260 · PythonMULTIMODAL FUTURE IS HERE
Fully open-source framework for training multimodal models — accessible to everyone. Python.
★ 1,050 · PythonVISRAG: RAG WITHOUT PARSING
VisRAG provides RAG without text parsing, using VLMs to directly extract knowledge from images and documents.
★ 991 · PythonGROUNDING LMM: SEE AND UNDERSTAND
CVPR 2024 🔥 Grounding Large Multimodal Model (GLaMM) – the first model that generates natural language answers tied to object segmentation masks. It combines scene under
★ 969 · PythonDSH Vision Toolkit
The project provides a powerful toolkit for working with images in pure text models: simple installation, recognition of inserted images, multi‑image Q&A, UI recovery fro
★ 945 · TypeScriptMULTIMODAL DECISION MODEL
Valen-Decision is a multimodal System One decision model. It takes text, images, and video, returning probabilities over supplied candidates and bounding box coordinates.
★ 918 · PythonMOSS-VL: 11B MODEL FOR VIDEO
The open-weight MOSS-VL series of 11-billion-parameter models is designed for understanding long-form and streaming video in real time. It supports multimodal input and e
★ 833 · PythonDOC7: Markdown Conversion
Convert any documents into clean Markdown, preserving structure and visual elements, ready for AI models.
★ 718 · Go4B Grounding Foundation Model
A 4B parameter foundation model unifying detection, referring, pointing, OCR, GUI, and layout grounding. Achieves SOTA across 34 benchmarks and improves robot generalizat
★ 439 · PythonOLYMPUS UNIFIES CV TASKS
The official Olympus code — a universal task router for CV, presented at CVPR 2025. Routes data to the best models.
★ 427 · PythonCOLPALI-COOKBOOKS — COOKING MULTIMODAL RAG
A collection of recipes for training and adapting the ColPali model for your multimodal search tasks.
★ 357COLETTE Multimodal RAG
Local multimodal RAG for searching and interacting with any technical documents via a web interface.
★ 302 · HTMLAESTHETIC HTML/CSS GENERATION FROM PROMPTS
AesCode turns prompts into editable HTML/CSS infographics. Uses image generators as aesthetic guidance and cross-modal rewards with RL (Qwen3) for verifiable code. 225★ o
★ 281 · PythonVISUAL PRIMITIVES FOR AI THINKING
Archival snapshot of the Thinking-with-Visual-Primitives project — 254 stars. Study of visual primitives in the context of AI thinking.
★ 280 · MakefileVIDORE: BENCHMARK VISION RETRIEVAL
Official ViDoRe benchmark for Vision Document Retrieval evaluation. Implements metrics and code from ColPali paper. Allows comparing models for visual document search.
★ 278 · PythonMULTIMODAL DECISION MODEL
OneJev — multimodal System One model for calibrated answers to questions about screens, photos, video and text in one forward pass.
★ 272 · PythonMEDICAL_IMAGE_ANALYSIS - AI ANALYSIS OF MEDICAL IMAGES
The project uses foundation models for medical image analysis, providing high accuracy and speed of diagnosis.
★ 243 · PythonGRACE: INT4 QWEN3-VL DISTILL
Qwen3-VL in INT4 with quantization-aware distillation – ready for deployment.
★ 213 · Jupyter NotebookNew projects on this topic in Telegram
Telegram →