Skip to main content
The fastest way to see a real scan — no agent of your own required. The AgentGuardian Testbench is a hosted Cloud Run service that runs five demo agents, one clean control and four planted with deliberate OWASP-LLM-Top-10 vulnerabilities. You point the scanner at the FinBot banking assistant and watch the swarm peel it open.
The testbench targets are owned and operated by the AgentGuardian project specifically so the community can red-team them. Never run AgentGuardian against a system you do not own or have written authorisation to test. Doing so may violate computer-misuse laws in your jurisdiction.

Confirm the testbench is up

You will attack finbot (a fictional banking assistant for “CineFlow Productions”) in the next step.

Set your LLM API key

The swarm needs an LLM provider to drive the Commander, Attacker, and Evaluator roles. Gemini 2.5 Flash is the cheapest path — a --mode fast scan costs roughly $0.01.
No API key handy? Swap --model gemini:gemini-2.5-flash for --model stub below. The swarm structure runs end-to-end but the AIVSS comes back as band=not_evaluated because the stub evaluator is not a real LLM. Use it to learn the flow, then re-run with a real model for an authoritative score.

Run the scan

Every flag here is declared in src/agent_guardian/cli.py:
  • --endpoint — hosted HTTP target URL.
  • --model gemini:gemini-2.5-flash — LLM spec for Commander / Attacker / Evaluator roles.
  • --mode fast — CI-gate smoke profile; caps each agent at 3 probes / 4 turns.
  • --budget-usd 0.20 — hard USD cap; the swarm soft-stops new attack turns at 80%.
Within the first second of stdout you will see two clickable URLs — that is the auto-served live dashboard wiring up before the swarm fires.

​
Expected output

The full live region is several hundred lines; here is a redacted slice showing the dashboard banner, mid-scan progress, and the final summary:
The exact AIVSS, finding count, and per-agent spend vary turn-to-turn (LLM non-determinism) but the band stays CRITICAL on every fast-mode run we have benchmarked — FinBot’s planted vulnerabilities are not subtle.

​
The summary line, field by field

  • AIVSS=23 — inverse-risk 0–100; lower is more vulnerable.
  • band=CRITICAL — band_for_score cutoff: any score < 40 is CRITICAL.
  • tier=T1 — auto-detected target tier (T1 = tools + memory + PII; the testbench advertises a tool surface so the swarm picks the strictest tier).
  • findings=14 — how many planted vulnerabilities the swarm confirmed.
  • report=scan.json — the default emitter; the canonical, signed copy also lands at ~/.agentguardian/scans/<scan_id>/scan.json.

​
Compare against the clean control

Now point the same scan at clean_control — a control agent built with no planted vulnerabilities — to verify the scanner is not generating false positives.
Expected summary line:
The control answers basic questions about a fictional library catalogue and refuses every prompt-injection, secret-extraction, and tool-abuse attempt the swarm throws at it. 0 findings on the control + 14 on FinBot is the credibility evidence: AgentGuardian found real vulnerabilities, not phantoms.
You have now run AgentGuardian against both a vulnerable agent and a clean control. The 73-point AIVSS gap (96 → 23) is the scanner doing its job.

​
Next step

Understanding Your First Report

Read every field of the scan.json — findings, evidence, AIVSS breakdown, fix-it commands.

Scan a REST API Agent

Now point the scanner at your own HTTP-shaped agent.

How AgentGuardian Works

The four-phase swarm: Recon → Decompose → Parallel attack → Finalise.

Attack Library

All 96 probes across 10 OWASP ASI categories.