Skip to main content

Documentation Index

Fetch the complete documentation index at: https://docs.agentguardian.io/llms.txt

Use this file to discover all available pages before exploring further.

​
What this gives you

A PR check that fails when the AgentGuardian AIVSS score drops below your floor, with every finding annotated inline in GitHub’s Security tab via the official github/codeql-action/upload-sarif@v3 action.

​
When to add this

  • The first time an LLM agent lands in main and needs a regression gate.
  • On every release branch, before tagging.
  • Before any change that touches the agent’s system prompt, tool surface, or memory layer.

​
Wire it up

Drop in the workflow

Create .github/workflows/agent-guardian.yml with the YAML below. The permissions: security-events: write block is required by github/codeql-action/upload-sarif@v3 to publish annotations to Code Scanning.
.github/workflows/agent-guardian.yml
name: AgentGuardian Red Team Scan

on:
  pull_request:
  push:
    branches: [main]

permissions:
  contents: read
  security-events: write   # required for codeql-action/upload-sarif

jobs:
  redteam:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Set up Python
        uses: actions/setup-python@v5
        with:
          python-version: "3.12"

      - name: Install AgentGuardian
        run: pip install agent-guardian

      - name: Run scan
        env:
          GEMINI_API_KEY: ${{ secrets.GEMINI_API_KEY }}
        run: |
          agent-guardian scan \
            --framework langgraph \
            --framework-ref my_app.graph:graph \
            --model gemini:gemini-2.5-flash \
            --mode full \
            --budget-usd 0.10 \
            --output sarif \
            --output-path scan.sarif \
            --fail-under 70

      - name: Upload SARIF to GitHub Code Scanning
        if: always()
        uses: github/codeql-action/upload-sarif@v3
        with:
          sarif_file: scan.sarif

Pick a target

Replace my_app.graph:graph with the dotted reference to your real framework-native object. Supported --framework values: adk, autogen, crewai, langgraph, openai_agents, strands.For a hosted HTTP agent, swap the framework flags for --endpoint https://my-agent.example.com/chat and set AGENT_GUARDIAN_AUTH_BEARER from a repo secret.

Add the provider secret

Repo Settings → Secrets and variables → Actions → New repository secret. Add the key matching your --model choice: GEMINI_API_KEY, OPENAI_API_KEY, ANTHROPIC_API_KEY, or use --model stub for an offline smoke check (note: stub runs are non-authoritative — they always fail --fail-under, see below).

Open a PR

Push the workflow on a branch and open a pull request. The AgentGuardian Red Team Scan / redteam check appears in the PR conversation and the SARIF findings appear under Security → Code scanning after the run completes.
--fail-under only gate-passes on --mode full. A fast or smart scan returns exit code 1 even if the numeric AIVSS clears the floor — fast/smart scans are designed as iteration smoke checks, not release gates. This is enforced in cli.py:3114–3129. Use --mode full on the workflow that gates merges.

​
The full flow

​
Expected output on a PR

A passing run prints the scan summary to the job log and exits 0: 
scan ag_2026_8f3a91cd done: AIVSS=82 band=low_risk tier=T2 findings=3 report=scan.sarif
A failing run prints the gate decision to stderr and exits 1: 
scan ag_2026_8f3a91cd done: AIVSS=58 band=elevated_risk tier=T2 findings=11 report=scan.sarif
--fail-under 70: FAILED -- AIVSS 58 < floor 70
In both cases the SARIF is uploaded (the upload step uses if: always()), so every finding shows up in the PR’s Files changed → Annotations lane and under Security → Code scanning alerts.

​
How to interpret the exit code

The CLI uses six exit codes, defined verbatim in cli.py:83–89. The gate condition you wire into the workflow should care about exactly two of them — 0 (pass) and 1 (gate failed). The rest are signals that something else went wrong and the scan never produced a verdict.
Exit codeConstantMeaningWhat to do in CI
0EXIT_OKScan completed and AIVSS ≥ --fail-under.Merge.
1EXIT_FAIL_UNDERScan completed and AIVSS < --fail-under, or the scan was non-authoritative (--mode fast/smart, or --model stub).Block merge. Read the SARIF annotations to triage.
2EXIT_CONFIGBad invocation — unknown flag, conflicting target modes, malformed --contract.Fix the workflow YAML. Not a security regression.
3EXIT_TARGET_UNREACHABLEThe pre-scan reachability probe for --endpoint mode failed.Check that the agent is up before the scan step (e.g. add a curl --retry health check).
4EXIT_LLM_PROVIDERThe LLM provider returned an unrecoverable error (rate limit, auth, network).Re-run the workflow. Check the provider secret.
5EXIT_SANDBOXThe sandboxed code adapter refused to load the target.Inspect the job log; fix the target reference.
130EXIT_USER_INTERRUPTThe runner cancelled the job (timeout, manual cancel).Re-run; consider raising --budget-usd if the scan is timing out.
Cap the scan’s spend with --budget-usd so a runaway provider can never cost more than you’ve budgeted per PR. The swarm soft-stops new attack turns at 80% of the cap and reserves the remainder for the report emission step.

​
Tuning the floor

Start permissive on the first PR (--fail-under 60) and tighten as you land mitigations. A reasonable progression:
  • First two weeks — --fail-under 60. Catches catastrophic regressions only; lets the team see what a real swarm finds without blocking every merge.
  • Steady state — --fail-under 70. Matches the low_risk band boundary; rejects merges that introduce a medium-severity ASI01/ASI02 finding.
  • Hardened release branch — --fail-under 80. Matches the safe / low_risk boundary; only ships when the agent has no high-severity open findings.
Band cutoffs are defined in src/agent_guardian/models/severity.py (function band_for_score).

​
Next step

Reports

Open the scan.sarif and the signed scan.json produced by every run.

First scan

Run the same command locally against the bundled vulnerable demo before you wire it into CI.

Attack library

See the 96 probes across 10 ASI categories the gate is exercising.

Contributing

Add a GitLab or Jenkins variant — same CLI, same exit codes.