Security testing for chat, RAG, agent, and MCP systems.
AI Gate is ShakerScan’s AI application security-testing workflow. It sends probe packs to a configured AI target, records transcripts, and turns detector and judge results into findings with the same evidence-first review model as Scan. In the current open-source release, AI Gate is a preview surface.
AI Gate is labeled preview in the open-source README. The APIs work with configured targets; demo calibration fixtures are optional and are not required to use it.
Who it is for. Teams shipping LLM features who need repeatable, transcript-backed evidence about how a chatbot, RAG pipeline, agent, or MCP server behaves under adversarial input.
What problem it solves
AI features fail in ways ordinary DAST does not see: prompt injection, sensitive-data disclosure, unsafe tool use, approval bypass, and retrieval leakage. Ad-hoc red teaming finds some of these once but leaves nothing repeatable behind.
AI Gate runs the same probes the same way each time and keeps transcripts and findings next to your other security evidence, so an AI regression is reviewed like any other finding.
How it works
- 01Open AI Gate, add a target, and describe its request and response mapping for chat APIs, RAG APIs, agent traces, MCP traces, or embeddable widgets.
- 02Test connectivity, then select a probe pack and scan profile. Production AI targets require explicit confirmation.
- 03ShakerScan sends the probes, records transcripts, and applies deterministic detectors. An optional AI judge can use the model provider you configure.
- 04Review transcripts and findings after the scan. Export an AI red-team report as JSON or Markdown, or export test cases in promptfoo, PyRIT, or garak formats.
- 05Review longitudinal probe history and use bounded replay to re-run failed or errored probes against the same target.
What you get
- Findings with source AI Gate, severity, and transcript evidence.
- An exportable AI red-team report and evidence pack per completed scan.
- A shared scenario catalog, a learning guide, and test-case exports for other AI evaluation tools.
- AI asset inventory, target connectivity tests, MCP live-readiness checks, and runtime-risk summaries.
Example workflow
# Connectivity check for a configured AI targetcurl -X POST http://localhost:8080/ai/targets/{target_id}/test \ -H "Content-Type: application/json" \ -d '{"prompt":"Connectivity check. Reply safely."}' # After a scan completes: AI red-team report and test-case exportcurl "http://localhost:8080/scans/{scan_id}/ai-redteam-report?format=markdown"curl "http://localhost:8080/ai/test-cases/export?format=promptfoo"
Security and authorization
- Run active AI workflows only against systems you own or are authorized to test. Probes can trigger provider safety systems, consume paid tokens, and appear in the target’s logs.
- Production AI targets require explicit confirmation before a scan is queued.
- Transcripts can contain sensitive prompts and responses. They are retained in your deployment; treat them as sensitive evidence.
Limitations
- AI Gate is a preview surface. Probe packs, detectors, and judges are not complete coverage of AI risk, and findings still require human review.
- Detector and judge quality depends on the target’s response shape and, when enabled, the AI judge provider you configure.
- ShakerScan does not claim that a passing AI Gate run makes a model or agent jailbreak-proof.
Get started
- 01Install ShakerScan and open AI Gate in the web UI.
- 02Add an AI target with its request and response mapping, then test connectivity.
- 03Select a probe pack and scan profile and start the scan on an authorized target.
- 04Review transcripts and findings, then export the report or test cases as needed.