Methodology · v0.1.0 · Updated 2026-09-09
How every grade is computed.
AgentTrust grades come from 8 static rules, a permission manifest, and documentary provenance — nothing else. No ML, no black box, no hidden signals. Everything below is reproducible with one command.
The rule suite
Eight rules, two OWASP lists.
Each finding carries a rule ID, a CWE, a remediation, and file:line evidence. Capability checks evaluate de-stringed code, so string literals and prose never confer capabilities.
| Rule | What it detects | Severity | OWASP |
|---|---|---|---|
| AT-SEC-001 | Direct prompt concatenation (injection via template) | Critical | LLM01 |
| AT-SEC-002 | Hardcoded credential or API secret | Critical | LLM02 |
| AT-SEC-003 | Unbounded dynamic shell execution | Critical | ASI02 |
| AT-SEC-004 | eval() / Function constructor invocation | Critical | ASI05 |
| AT-SEC-005 | Unrestricted recursive file deletion | High | ASI02 |
| AT-SEC-006 | Unfiltered SSRF / arbitrary network egress | High | LLM06 |
| AT-SEC-007 | Raw secret leakage in debug logging | Medium | LLM02 |
| AT-COMP-001 | Missing human-in-the-loop gate for critical actions | High | ASI09 |
OWASP references: LLM Top 10 2025 (LLM01–LLM10) and Agentic Top 10 published 2025-12-09 (ASI01–ASI10).
Scoring
Five dimensions, fixed weights.
| Dimension | Weight | What moves it |
|---|---|---|
| Security | 35% | Critical −25, high −15, medium −8, low −3 per finding |
| Permissions | 25% | Excessive −45, broad −25, moderate −10; shell, deletion, unapproved email subtract; enforced human approval adds |
| Provenance | 15% | License, lockfile, security policy, changelog, signatures add documentary points |
| Reliability | 15% | Reliability/compliance findings subtract; human gates add |
| Stability | 10% | Lockfile and changelog presence |
Grades: A 90+ · B 75+ · C 60+ · D 40+ · F below 40. Finding IDs are deterministic hashes, so repeat scans produce byte-stable SARIF.
Limits
What the scanner cannot see.
Static analysis only. No code executes, no payloads fire, no runtime behavior is observed. Findings need human triage; counts may include test and example code. Attack analysis is heuristic relabeling of findings; workflow eval parses suites without executing them. Trust Cards are evidence input — never a certification, audit, or compliance verdict.
Why our own repo grades F
It contains its own test ammunition: a deliberately vulnerable fixture, unit tests with attack strings, and automation scripts that shell out by design. The shipped CLI grades A. Grade the artifact, not the test bench.
Reproduce any grade
npx @eulogik/agenttrust scan https://github.com/owner/repo --quiet
Registry snapshots record the upstream commit sha, so any historical grade can be re-verified exactly.