LLM Security: open-source projects on GitHub
36 repositories on LLM Security, picked by the REDDYX AI agent. Each has a summary, key features and a breakdown. Sorted by stars.
// RELATED TOPICS
Ready LLM templates
Ready cloud templates for RAG, AI‑pipeline, and search with live data, Docker‑compatible and synchronized with SharePoint, S3, Kafka, and PostgreSQL.
★ 59,750 · Jupyter NotebookBINARY DEFENSE AGAINST AI AGENTS
BLA is an adversarial defense for binaries, Electron, JS, Python, and Wasm. It poisons context, exhausts resources, and induces hallucinations in reverse-engineering agen
★ 5,675 · TypeScriptGISKARD-OSS - LLM AGENT EVALUATION
Library for evaluating and testing LLM agents with open-source code.
★ 5,417 · PythonAI SYSTEM SECURITY
A platform for testing the security of AI ecosystems: scanning infrastructure, MCP, agents, and evaluating jailbreak attacks.
★ 3,790 · PythonAI‑ENGINEER HQ: SCIENCE CORE
The project contains Jupyter notebooks with methods for training models, RAG pipelines on LangChain and data structure algorithms for AI agents. It allows quickly creatin
★ 3,780 · Jupyter NotebookCapability-based sandbox proxy
Capability-based sandboxes with fine-grained policy. Proxy access directly in the agent's execution context, with zero configuration and zero latency.
★ 2,407 · Rustbeelzebub - AI-Driven Low-Code Virtualization
Beelzebub is a low-code framework for safely deceiving systems, using AI to virtualize environments.
★ 2,238 · GoADR: AI Agent Security in Uber
ADR provides observability, security testing, and threat detection for enterprise AI agents, already used at Uber.
★ 2,135 · PythonAI SAFETY HOT DIGEST
AI Safety HOT collects daily AI safety news, papers & incidents, jailbreaks, prompt injection, alignment and evals. On the platform — website, daily digest, RSS and MCP.
★ 2,125 · TeXAI-AGENT FOR SECURITY
0 — open-source AI security agent that autonomously detects, exploits and fixes vulnerabilities in the stack. Research preview from Swiss Applied AI & Cybersecurity Resea
★ 1,356 · TypeScriptCybermes Security Agent
Autonomous framework for offensive security, bug bounty, and red teaming, based on Hermes Agent with advanced reasoning and orchestration of multiple LLMs.
★ 1,026 · PythonMEDUSA: AI Security Scanner
AI-first scanner with 79 analyzers and 40,000+ threat detection rules for AI/ML, LLM agents, and MCP servers.
★ 1,018 · PythonPIPELOCK PROTECTS AI AGENTS
AI agent firewall for MCP security and egress traffic. Scans HTTP, MCP, A2A and WebSocket traffic in real time for exfiltration, SSRF and prompt injection, emitting media
★ 993 · GoSHIP-SAFE SECURITY AGENTIC
The ship-safe CLI scanner looks for CI/CD vulnerabilities, agent permission risks, MCP injections, secrets, and dependencies flagged by DMCA.
★ 872 · JavaScriptPROTECTION OF AI AGENTS FROM THREATS
safelabs-eval — framework for red‑teaming and evaluation of AI agent protection based on the ASI01–ASI10 taxonomy inspired by OWASP. Written in Python, it supports CrewAI
★ 829 · PythonISOLATED ENVIRONMENT FOR AI AGENTS
Each AI coding agent gets its own isolated machine with root access, Docker, and systemd. Built‑in active defense automatically detects and neutralizes threats.
★ 786 · GoTRACEKIT: AUDIT AI AGENT ACTIONS
Tracekit provides tamper-evident tracing of AI agent actions with policy gates and cross-checks. See what the agent did and why — critical for safety and compliance.
★ 726 · PythonARCJET: AI SECURITY
Protect your AI agents from prompt injections, data theft, and tool abuse with a simple TypeScript module.
★ 701 · TypeScriptAI firewall for MCP
Open AI-agent firewall for MCP protection: outbound request control, DLP, SSRF and prompt injection defense.
★ 636 · GoAPI RELAY SECURITY SCANNER
Local scanner for AI API relays and LLM proxies. 620 ⭐ on GitHub.
★ 620 · PythonAGENT OPFOR ATTACK EMULATION
Open‑source framework for simulating attacks on AI agents and MCP servers. Written in TypeScript and already has 377⭐.
★ 586 · TypeScriptCODE-ON-INCUS – ISOLATED AGENTS
Each AI agent gets its own container with root, Docker, and systemd, while active protection automatically neutralizes threats.
★ 530 · PythonOPEN-PROMPT-INJECTION
Open-Prompt-Injection — benchmark for evaluating attack and defense against prompt injections in LLMs. Contains datasets,test scripts,defense metrics. Implemented in Pyth
★ 514 · PythonSECURITY FOR AI CODERS - CAPS
Prismor blocks dangerous commands, secret leaks, and prompt injections in Claude Code, Cursor, and Windsurf in real time.
★ 494 · PythonAGENTHOUND AGENT HUNT
Offensive security framework for AI agent infrastructure: reconnaissance, credential looting, model exfiltration, poisoning, and attack‑path analysis.
★ 464 · GoPrompt Hacking Course
The most complete course on prompt engineering and LLM attacks: from basics to advanced hacking and defense techniques.
★ 394 · Jupyter NotebookLLMVault: OWASP LLM Training
Learn to protect AI systems: practical Prompt Injection, RAG, and agent vulnerabilities in one open training framework.
★ 356 · PythonPROTECTIVE LIST OF AGENT SKILLS
Collected materials on protecting AI‑agent skills: attacks, defenses, frameworks, and benchmarks. The list helps researchers and developers assess tool‑use vulnerabilitie
★ 287AGENT-AUDIT SECURITY SCANNER
Static security scanner for LLM agents. Detects prompt injection, audits MCP configs, and performs taint analysis. Includes 51 rules mapped to OWASP Agentic Top 10 (2026)
★ 245 · PythonDOBERMAN-CORE — PROTECTION FOR AI AGENTS
Doberman-Core intercepts every input, output, and tool call during AI-agent operation, blocking unsafe actions. It works as middleware, checking authorization and applyin
★ 243 · PythonRED-TEAMING AI AGENTS IN PYTHON
Humanbound — adversarial testing engine, SDK, and CLI for AI agents. Runs locally or via Humanbound Platform. 218 stars on GitHub.
★ 239 · PythonGUARDANA AI MODEL PROTECTION
Guardana scans model artifacts, live endpoints, MCP servers, and agent traces for security threats. It generates reproducible evidence for release decisions.
★ 225 · PythonGate Private Data Boundary
Gate creates a deterministic barrier between your data and AI, ensuring privacy without losing utility.
★ 214 · RustARGUS: AI AGENT SECURITY RED-TEAM TESTING
Open-source framework for red-team testing AI agents. 261 probes mapped to OWASP LLM Top 10, MITRE ATLAS, and NIST AI RMF. Browser, MCP, and mock payment support included
★ 210 · PythonADRIAN: AI AGENT SECURITY
Adrian monitors and blocks dangerous AI agent actions in real time, preventing malicious tool calls and prompt injections.
★ 203 · PythonDEFENSE MEMORY AI-AGENTS
Provides runtime defense against memory AI-agent poisoning (OWASP ASI06). Drop-in for LangChain, OpenAI Agents, AutoGen, CrewAI and mem0.
★ 203 · PythonNew projects on this topic in Telegram
Telegram →