R_REDDYX.XYZ

Vision Language Model: open-source projects on GitHub

28 repositories on Vision Language Model, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.

// RELATED TOPICS

Multimodal 143LLM 2094VLM 50Llava 15Local AI 82Apple Silicon 56Mlx 47Computer Use 62Deepseek 229RAG 533Retrieval 25Pytorch 98Search 47Large Language Models 154
Neywa-Labs/mlx-vlm

RUN VLM LOCALLY ON MAC

mlx-vlm runs and fine-tunes vision-language models (LLaVA, Florence-2, Idefics) on Apple Silicon via MLX. Everything works locally on your Mac without GPU servers.

★ 5,847 · Python
Blaizzy/mlx-vlm

MLX-VLM: RUN VLM ON MAC

Package for running and fine‑tuning Vision Language Models on Mac via MLX. 4.7K★

★ 4,787 · Python
EvolvingLMMs-Lab/lmms-eval

LMMS-EVAL - ONE-STOP MODALITY EVAL

Unified toolkit for evaluating multimodal models on text, image, video, and audio tasks. 4169 stars on GitHub.

★ 4,169 · Python
off-grid-ai/OGAM

OGAM Offline AI Swiss Knife

OGAM — chat, vision, speech, and image generation offline on your device without keys or network data.

★ 3,380 · TypeScript
2U1/Qwen-VL-Series-Finetune

QWEN-VL SERIES FINETUNE OPEN SOURCE

Open-source implementation for fine-tuning Qwen-VL series models from Alibaba Cloud in Python. Allows easy adaptation of multimodal models for custom tasks.

★ 1,983 · Python
microsoft/Mage

MAGE: AI AGENT FOR PC CONTROL

Microsoft released Mage — an agent for GUI control via flow matching. The model edits images and performs computer tasks without fine-tuning.

★ 1,782 · Python
ATH-MaaS/Ovis

OVIS: Visual-Text Sync

A modern MLLM architecture for structural alignment of visual and text embeddings.

★ 1,539 · Python
emcf/thepipe

THEPIPE: DATA FROM DOCUMENTS FAST

ThePipe uses vision‑language models to extract data from PDF, Word and PowerPoint. It recognises tables, text and images, turning them into ready‑to‑use structures. Suppo

★ 1,530 · Python
gokayfem/awesome-vlm-architectures

VISION AND LANGUAGE: TOP MODELS

A collection of architectures of well-known vision‑language models with explanations and links.

★ 1,368 · Markdown
Anionex/agent-vision-toolkit

Visual Toolkit for LLM

A library for text models that lets them 'see' images: recognize content, answer questions, restore UI, automate GUI, and integrate with agents.

★ 1,262 · Python
waybarrios/vllm-mlx

vllm-mlx LLM Server

OpenAI and Anthropic compatible server for Apple Silicon that runs LLMs and vision models with continuous batching and MCP tool calling, achieving 400+ tokens/s.

★ 1,260 · Python
EvolvingLMMs-Lab/LLaVA-OneVision-2

MULTIMODAL FUTURE IS HERE

Fully open-source framework for training multimodal models — accessible to everyone. Python.

★ 1,050 · Python
OpenBMB/VisRAG

VISRAG: RAG WITHOUT PARSING

VisRAG provides RAG without text parsing, using VLMs to directly extract knowledge from images and documents.

★ 991 · Python
mbzuai-oryx/groundingLMM

GROUNDING LMM: SEE AND UNDERSTAND

CVPR 2024 🔥 Grounding Large Multimodal Model (GLaMM) – the first model that generates natural language answers tied to object segmentation masks. It combines scene under

★ 969 · Python
Anionex/dsh-vision-toolkit

DSH Vision Toolkit

The project provides a powerful toolkit for working with images in pure text models: simple installation, recognition of inserted images, multi‑image Q&A, UI recovery fro

★ 945 · TypeScript
Valen-Team/Valen

MULTIMODAL DECISION MODEL

Valen-Decision is a multimodal System One decision model. It takes text, images, and video, returning probabilities over supplied candidates and bounding box coordinates.

★ 918 · Python
OpenMOSS/MOSS-VL

MOSS-VL: 11B MODEL FOR VIDEO

The open-weight MOSS-VL series of 11-billion-parameter models is designed for understanding long-form and streaming video in real time. It supports multimodal input and e

★ 833 · Python
magicrew/doc7

DOC7: Markdown Conversion

Convert any documents into clean Markdown, preserving structure and visual elements, ready for AI models.

★ 718 · Go
groundingpi/GroundingPI

4B Grounding Foundation Model

A 4B parameter foundation model unifying detection, referring, pointing, OCR, GUI, and layout grounding. Achieves SOTA across 34 benchmarks and improves robot generalizat

★ 439 · Python
yuanze-lin/Olympus

OLYMPUS UNIFIES CV TASKS

The official Olympus code — a universal task router for CV, presented at CVPR 2025. Routes data to the best models.

★ 427 · Python
tonywu71/colpali-cookbooks

COLPALI-COOKBOOKS — COOKING MULTIMODAL RAG

A collection of recipes for training and adapting the ColPali model for your multimodal search tasks.

★ 357
jolibrain/colette

COLETTE Multimodal RAG

Local multimodal RAG for searching and interacting with any technical documents via a web interface.

★ 302 · HTML
microsoft/AesCode

AESTHETIC HTML/CSS GENERATION FROM PROMPTS

AesCode turns prompts into editable HTML/CSS infographics. Uses image generators as aesthetic guidance and cross-modal rewards with RL (Qwen3) for verifiable code. 225★ o

★ 281 · Python
ailuntx/Thinking-with-Visual-Primitives

VISUAL PRIMITIVES FOR AI THINKING

Archival snapshot of the Thinking-with-Visual-Primitives project — 254 stars. Study of visual primitives in the context of AI thinking.

★ 280 · Makefile
illuin-tech/vidore-benchmark

VIDORE: BENCHMARK VISION RETRIEVAL

Official ViDoRe benchmark for Vision Document Retrieval evaluation. Implements metrics and code from ColPali paper. Allows comparing models for visual document search.

★ 278 · Python
OmniJev/OneJev

MULTIMODAL DECISION MODEL

OneJev — multimodal System One model for calibrated answers to questions about screens, photos, video and text in one forward pass.

★ 272 · Python
Event-AHU/Medical_Image_Analysis

MEDICAL_IMAGE_ANALYSIS - AI ANALYSIS OF MEDICAL IMAGES

The project uses foundation models for medical image analysis, providing high accuracy and speed of diagnosis.

★ 243 · Python
ForeverBlue816/GRACE

GRACE: INT4 QWEN3-VL DISTILL

Qwen3-VL in INT4 with quantization-aware distillation – ready for deployment.

★ 213 · Jupyter Notebook

New projects on this topic in Telegram

Telegram →
Vision Language Model — 28 open-source GitHub projects | REDDYX AI