R_REDDYX.XYZ

VLM: open-source projects on GitHub

50 repositories on VLM, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.

// RELATED TOPICS

LLM 2094Multimodal 143Agent 547Llama 106Qwen 106Reinforcement Learning 51RAG 533Vision Language Model 28Deepseek 229GPT Oss 20Openai 690Benchmark 54Pytorch 98MCP 1601
huggingface/transformers

🤗 Transformers

A framework for defining machine learning models in text, vision, audio, and multimodal models, for inference and training.

★ 160,482 · Python
bytedance/UI-TARS-desktop

Multimodal AI Stack

Open multimodal AI agent stack that unites cutting‑edge models and infrastructure for building autonomous agents.

★ 33,596 · TypeScript
sgl-project/sglang

SGLang: LLM Framework

SGLang is a high-performance framework for serving large language and multimodal models.

★ 27,854 · Python
StarTrail-org/PixelRAG

PixelRAG: Pixel Search

PixelRAG replaces traditional web parsing with direct search in the pixel representation of pages, providing scalable and accurate data extraction.

★ 10,946 · Python
RunanywhereAI/runanywhere-sdks

Local AI SDK Set

Production-ready toolkit for running AI models on a local device.

★ 10,350 · C++
oumi-ai/oumi

OUMI EASY LLM TUNING

OUMI lets you quickly fine‑tune (SFT/RL), evaluate, and deploy Qwen, Gemma, and any open agentic LLM/VLM without extra complexity.

★ 9,433 · Python
qualcomm/GenieX

GENIEX: LLMs on Devices

Run cutting-edge multimodal models on GPU, NPU, and CPU — from PC to mobile and IoT devices.

★ 8,530 · Rust
JuneYaooo/rea-video

FRAME-BY-FRAME VIDEO BREAKDOWN

The rea-video project reverse-engineers video frame by frame: identifies what was filmed, AI-generated, or coded. Outputs edit details, captions, audio specs, and a repro

★ 8,505 · Python
qualcomm/nexa-sdk

NEXA-SDK AI Everywhere Always

Run cutting‑edge LLMs and VLMs on GPU, NPU, and CPU — from PC to mobile and IoT devices. Support for models like GPT‑OSS, Granite‑4, Qwen‑3‑VL, and more.

★ 8,094 · Rust
om-ai-lab/VLM-R1

VLM-R1: Smart Image Recognition

The project demonstrates how reinforcement learning improves image understanding by multimedia models.

★ 6,046 · Python
OpenBMB/UltraRAG

UltraRAG: RAG pipeline framework

Low-code MCP framework for creating complex and innovative RAG pipelines.

★ 5,549 · Python
EvolvingLMMs-Lab/lmms-eval

LMMS-EVAL - ONE-STOP MODALITY EVAL

Unified toolkit for evaluating multimodal models on text, image, video, and audio tasks. 4169 stars on GitHub.

★ 4,169 · Python
SoMarkAI/OXR

Open-Source Document Parser

Open-source document parsing model achieving state-of-the-art accuracy and speed through recursive self-improvement post-training.

★ 3,830 · Python
SkyworkAI/Skywork-R1V

SKYWORK-R1V VISION LANGUAGE AI

The Skywork-R1V model series combines vision and language for complex reasoning. Available in Python and already has over 3k stars.

★ 3,173 · Python
xlang-ai/OSWorld

OSWorld: BENCHMARK FOR AI AGENTS

OSWorld is a benchmark for evaluating multimodal AI agents in real computer environments. Open tasks in real operating systems.

★ 2,911 · Python
modelscope/evalscope

EVALSCOPE: SIMPLE LLM TESTING

Flexible Python framework for evaluating the performance of large models (LLM, VLM, AIGC). ⭐2874

★ 2,874 · Python
om-ai-lab/OmAgent

OMAGENT: THE MULTIMODAL AGENT

OmAgent — Python framework for multimodal agents on Gemini, GPT-4 and other LLMs. Enables rapid prototyping and deployment via Gradio, supports RAG and rapid chatbot temp

★ 2,670 · Python
heshengtao/comfyui_LLM_party

LLM Agent Framework in ComfyUI

Framework unites LLM agents, Omost, GPT‑Sovits, ChatTTS, and OCR in ComfyUI with support for Feishu, Discord, and all popular LLMs.

★ 2,390 · Python
2U1/Qwen-VL-Series-Finetune

QWEN-VL SERIES FINETUNE OPEN SOURCE

Open-source implementation for fine-tuning Qwen-VL series models from Alibaba Cloud in Python. Allows easy adaptation of multimodal models for custom tasks.

★ 1,983 · Python
yueliu1999/Awesome-Jailbreak-on-LLMs

COLLECTION OF JAILBREAK LLM METHODS

Awesome-Jailbreak-on-LLMs is a collection of cutting‑edge jailbreak methods for large language models. The repository contains papers, code, datasets, evaluations, and ef

★ 1,977
Core-Mate/OpenGUI

OpenGUI: Control Android via GUI

OpenGUI is a framework for creating AI agents that can see, plan, and control real Android applications via their interface.

★ 1,811 · Kotlin
llmsresearch/paperbanana

PaperBanana: Scientific Visualization Generation

An open-source implementation and extension of Google Research PaperBanana for automating the creation of academic figures, diagrams, and scientific visualizations.

★ 1,648 · Python
NVIDIA-AI-Blueprints/video-search-and-summarization

GPU-Accelerated AI Video Analytics

A set of reference architectures for building GPU-accelerated vision agents and AI-based video analytics applications. ⭐ 1,486 stars, Python.

★ 1,486 · Python
gokayfem/awesome-vlm-architectures

VISION AND LANGUAGE: TOP MODELS

A collection of architectures of well-known vision‑language models with explanations and links.

★ 1,368 · Markdown
Tencent/AngelSlim

ANGELSLIM: COMPRESSING MODELS WITHOUT LOSS

A tool for model compression focusing on convenience and efficiency. 1257 stars on GitHub.

★ 1,257 · Python
EvolvingLMMs-Lab/NEO

NEO: Vision-Language Models

A series of multimodal Vision-Language models built from scratch. 839 stars on GitHub.

★ 926 · Python
Valen-Team/Valen

MULTIMODAL DECISION MODEL

Valen-Decision is a multimodal System One decision model. It takes text, images, and video, returning probabilities over supplied candidates and bounding box coordinates.

★ 918 · Python
Purewhiter/mobilegym

MOBILEGYM: ANDROID AGENT TRAINING

MobileGym is a browser‑hosted Android emulator for research on mobile GUI agents. The platform provides verifiable evaluation and highly parallel online reinforcement‑lea

★ 866 · Python
OpenMOSS/MOSS-VL

MOSS-VL: 11B MODEL FOR VIDEO

The open-weight MOSS-VL series of 11-billion-parameter models is designed for understanding long-form and streaming video in real time. It supports multimodal input and e

★ 833 · Python
MigoXLab/dingo

DINGO: AI MODEL EVALUATION

Dingo is a tool for comprehensive evaluation of data, model, and AI application quality in Python. It enables measuring reliability and performance of systems.

★ 768 · Python
taco-group/SparkVSR

SPARKVSR: INTERACTIVE VIDEO SUPER-RESOLUTION

A new video super-resolution method via sparse keyframe propagation. ECCV 2026 paper, 675 stars on GitHub.

★ 750 · Python
TIGER-AI-Lab/VLM2Vec

VLM2Vec: Vector Models for Multimodality

The library implements the VLM2Vec method for training Vision‑Language models that produce effective multimodal embeddings. Presented at ICLR 2025.

★ 704 · Python
huggingface/optimum-intel

OPTIMUM-INTEL - ACCELERATE INFERENCE WITH INTEL

Optimum Intel provides Intel tools to accelerate AI model inference, simplifying deployment and boosting performance.

★ 630 · Jupyter Notebook
zenstory-ai/video-recap-skills

VIDEO RECAP WITH CLAUDE CODE

The project provides Claude Code skills for creating Chinese narration recaps from video files. It exports the result as an editable JianYing/CapCut draft. It runs locall

★ 626 · Python
RobotecAI/rai

RAI: Agent for Robots

RAI is a framework for physical AI robots on ROS 2 and Python, supporting voice interaction, scenarios, and logs.

★ 616 · Python
Fabric-Project/Fabric

FABRIC CREATIVE 3D CODING

Fabric is a Node‑based environment for creative coding, 3D graphics and image processing, inspired by Quartz Composer. In Swift it uses Metal for fast rendering and suppo

★ 606 · Swift
gokayfem/ComfyUI_VLM_nodes

ComfyUI_VLM_nodes — Powerful Multimodal AI

A set of custom nodes for ComfyUI: from working with VLM and LLM to music generation and creative prompts.

★ 593 · Python
NVIDIA-NeMo/Automodel

AUTOMODEL — DISTRIBUTED LLM/VLM TRAINING

Native PyTorch library for distributed LLM/VLM training with out-of-the-box Hugging Face support. ⭐ 576 stars on GitHub.

★ 576 · Python
awwaiid/ghostwriter

REMARKABLE2 BECOMES VISION-LLM INTERFACE

Project ghostwriter turns reMarkable2 into handwritten interface for vision-LLMs: ChatGPT, Claude, Gemini. Write by hand on E Ink — model sees, understands, replies. Writ

★ 573 · Rust
worldbench/awesome-vla-for-ad

VLA MODELS FOR CARS

A collection of Vision-Language-Action models for autonomous driving: an overview of past, present, and future directions.

★ 527 · HTML
hybridgroup/yzma

YZMA: Local LLM in Go

Integration of llama.cpp into Go applications for local inference with hardware acceleration. 496 stars on GitHub.

★ 496 · Go
xlang-ai/OSWorld-V2

OSWorld-V2 - AGENT REAL TASK TEST

OSWorld 2.0 evaluates AI agents' ability to complete long real computer tasks using only Python.

★ 421 · Python
OpenEnvision/Awesome-Multimodal-Modeling

AWESOME MULTIMODAL MODELING

A collection of resources on multimodal modeling: MLLM, UMM, NMM, 336 ★.

★ 336
vlm-run/vlmrun-cookbook

VLMRUN-COOKBOOK - VLM NOTEBOOKS

A collection of Jupyter notebooks showing how to quickly get started with the VLM Run API: loading models, inference, and multimodal output.

★ 313 · Jupyter Notebook
JosefAlbers/phi

phi - Simple Phi-4 Launch

Run the powerful multimodal Phi-4 model directly on your Mac with Apple Silicon — no cloud, no complex setup.

★ 280 · Jupyter Notebook
EvolvingLMMs-Lab/LongVT

LongVT — Think About Video Foundations

A model that learns to 'think' about long videos through built‑in tools for CVPR 2026.

★ 274 · Python
OmniJev/OneJev

MULTIMODAL DECISION MODEL

OneJev — multimodal System One model for calibrated answers to questions about screens, photos, video and text in one forward pass.

★ 272 · Python
mgonzs13/llama_ros

LLAMA_ROS ROS 2 LLM

The project combines llama.cpp and llava.cpp with ROS 2, allowing the use of GGUF models for real-time robotics.

★ 268 · C++
gokayfem/ComfyUI-fal-API

ComfyUI Fal API: 1400+ Models

Production-ready custom nodes for ComfyUI, providing access to 1400+ fal.ai models with automatic updates, caching, and cost control.

★ 227 · Python
Somnusochi/VLM-AutoYOLO

Auto-labeling YOLO Training

Platform for automatic image and video labeling using VLM NVIDIA LocateAnything-3B, with manual correction and one-click YOLO model training.

★ 31 · Python

New projects on this topic in Telegram

Telegram →
VLM — 50 open-source GitHub projects | REDDYX AI