Agent Security Regression Harness

Overview

Agentic applications are increasingly able to retrieve untrusted content, call tools, access sensitive data, interact with MCP servers, and execute multi-step tasks. Small changes to prompts, model configuration, tool permissions, memory behavior, retrieval sources, or approval flows can silently reintroduce security failures.

The OWASP Agent Security Regression Harness is an open source project for making agent security testing repeatable, automatable, and actionable. It helps builders and defenders define abuse-case scenarios, execute them against agent systems, assert expected security outcomes, and integrate those checks into local development and CI pipelines.

The project is intended for application security engineers, AI platform teams, red teams, developers building agents, and organizations adopting MCP-connected systems.

What the project provides

The project will provide a Python-based regression harness and CLI for executing agent security test scenarios. It will include a reusable scenario specification format, a policy assertion engine for validating security invariants, execution traces for debugging and auditability, and machine-readable outputs for CI and reporting.

Initial work will focus on scenario libraries for common agent and MCP failure modes, along with reference integrations for API-driven agents, selected agent frameworks, and MCP-based workflows.

Security areas covered

Initial scenarios will focus on goal hijack, prompt and context injection, unsafe tool execution, unauthorized outbound actions, sensitive data disclosure, privilege escalation, identity misuse, insecure memory handling, session isolation, approval bypass, and malicious or shadow MCP interactions.

Project scope

This project is intentionally scoped for an Incubator launch. The first year focuses on a narrow, usable core rather than broad framework coverage.

The project is vendor-neutral, model-neutral, and designed for local and CI-based use. The initial implementation prioritizes practical integrations, reproducible traces, and a stable scenario format over benchmark-style scoring.