Valen-Team/Valen
MULTIMODAL DECISION MODEL
Valen-Decision is a multimodal System One decision model. It takes text, images, and video, returning probabilities over supplied candidates and bounding box coordinates.
// KEY FEATURES
- Returns candidate probabilities and bbox instead of generated text
- Sizes from 0.8B to 27B: native weights, ONNX, GGUF, vLLM, SGLang
- Free API, Python/JS/C++ SDK, integration with Claude Code and Cursor
- Includes fine-tuning code, JevalKit for comparison with GPT/Claude/Gemini
// DEEP DIVE: HOW IT WORKS
Valen-Decision works as System One: given an instruction and multimodal context (text, images, video), it outputs a probability distribution over predefined answer candidates. Version 1.5 adds a second head for bounding box prediction, enabling the model to not only choose but also localize objects. The architecture uses block-bidirectional attention to preserve strong text capabilities alongside multimodality.
Unlike typical LLMs, it does not generate free-form text but returns structured JSON with probabilities and coordinates. Checkpoints from 0.8B to 27B are available in three formats: native (Transformers), ONNX Runtime, and GGUF with a patched llama.cpp. Ready-to-use SDKs for Python, JavaScript, and C++ are provided, plus an agent skill for Claude Code, Cursor, and Codex. JevalKit allows benchmarking Valen side-by-side with GPT, Claude, and Gemini.
Local deployment requires a GPU with VRAM matching the chosen model size (0.8B–27B). GGUF variants run via a custom llama.cpp server that accepts images. ONNX models require ONNX Runtime. The free API needs no key but has rate limits. Fine-tuning code (SFT and experimental RLCD) is included in the repo, datasets on Hugging Face.
// USE CASES
Best for: For agent and RAG developers needing structured decision outputs
- Plug the model as a probabilistic classifier into a RAG pipeline instead of an LLM judge
- Use the bbox head for visual grounding of agent answers in the UI
- Compare Valen decision quality against GPT/Claude via JevalKit before production
New repositories every 30 minutes
REDDYX AI scans GitHub 24/7 and ships the best AI/ML/Web3 projects to Telegram.
Join on Telegram