Use Case
Test RAG behavior before customers rely on it.
RAG failures often depend on runtime retrieval behavior, tenant boundaries, document lifecycle, and citations. ShakerScan turns those checks into release evidence.
Sample gate decision
Cross-tenant retrieval leakage
The target returned another tenant name or document reference during a RAG probe. Policy blocks the release until retrieval isolation is fixed.
Connect a RAG API, widget, or trace target.
Run RAG-focused probes against retrieval, citation, and tenant-boundary behavior.
Review transcript-backed evidence and remediation guidance.
Gate releases that leak data, cite wrong sources, or recall deleted content.
Why ShakerScan
The output is a release control, not just a report.
ShakerScan is built around release evidence: a tested target, a policy result, a verifier command, and an approval path when risk needs human review.
Signed evidence
Evidence hashes and AI Gate attestations bind the decision to the target, environment, policy, probe pack, and release scope when signing is configured.
CI-verifiable decision
GitHub Actions or the shakerscan CLI can verify that the decision matches the expected repo, commit, branch, environment, target, policy, and evidence hash.
Approval workflow
When an eligible workflow is approved, scoped approval tokens record the reason, audience, expiry, and decision path instead of bypassing the gate silently.
Checklist
RAG test checklist
Test retrieval with tenant-specific and deleted-source canaries.
Check citation integrity and source mismatch behavior.
Retest after index changes, document deletion, or prompt changes.
Use least-privilege credentials for testing.
Limitations
What this page does not claim
ShakerScan does not replace human security review, threat modeling, or a scoped penetration test.
AI Gate decisions depend on the configured target, probe pack, policy, scan profile, and available evidence.
Production targets require authorization, safe scope, rate limits, and operational approval.
FAQ
Is ShakerScan an AI pentesting replacement?
No. ShakerScan is a verifiable security gate for release workflows. It complements deeper manual testing by producing repeatable runtime evidence and CI-verifiable allow, block, or needs_approval decisions.
Can ShakerScan scan any target?
No. Targets must be owned by the customer or explicitly authorized. Production scans should use safe profiles, rate limits, and defined scope.