A Proposed Evidence-Led Assessment Method for AI Systems

AI assessments often begin with a generic control list and end without showing whether controls address the actual use case. We propose an Evidence–Boundary–Behavior method, or EBB, for teams evaluating an AI system before release or material change. This is an original assessment methodology synthesized from established risk and application-security guidance. It reports no experiments, survey results, benchmarks, or claims of effectiveness.

EBB treats assurance as a set of bounded claims. Each claim names an owner, a system boundary, an unacceptable outcome, supporting controls, a way to challenge those controls, and evidence that can be retained. The aim is to make disagreements and unknowns visible rather than compressing them into a single maturity score.

First, frame intended users, affected parties, permitted purposes, prohibited outcomes, human authority, and the release decision. Second, map models, data sources, retrieval stores, prompts, tools, identities, providers, operators, and downstream consumers. Mark where trust or control ownership changes.

Third, create abuse narratives. For each important asset or decision, describe a plausible actor, precondition, action sequence, and consequence. Include direct and indirect prompt injection, sensitive-data exposure, poisoned sources, excessive agency, dependency compromise, resource exhaustion, and ordinary model error. Fourth, translate narratives into control claims such as: “a retrieved document cannot grant tool authority.” Avoid claims that merely restate a product feature.

Fifth, design evidence and challenges. For every claim, identify design evidence, configuration evidence, a repeatable test, expected safe behavior, observation points, and a named reviewer. Tests should include realistic multi-step paths and negative cases, not only prohibited keywords. Sixth, decide and monitor: classify residual risk, dependencies, expiry conditions, rollback triggers, and signals that require reassessment. A change to model, system prompt, retrieval corpus, tool permission, or provider can invalidate evidence.

The core record lists claim ID, use case, boundary, abuse narrative, control owner, test procedure, evidence, result, limitations, residual risk owner, and expiry trigger. A companion diagram shows identity and data paths. A decision page lists accepted, deferred, and blocked risks. Teams can map records to NIST AI RMF functions without presenting the mapping as certification.

For prioritization, use consequence and exposure bands rather than an apparently precise numeric risk score. A customer-facing agent with write access deserves deeper testing than an isolated summarizer, even if both use the same model. Reassessment depth should follow the changed boundary and potential consequence.

EBB is a proposed process, not a validated predictor of incidents. Test results are specific to available versions, prompts, data, tools, identities, and test coverage. Generative behavior can vary, and an assessment cannot enumerate every adversarial path or establish legal compliance. Evidence may become stale after silent provider changes. Independent legal, privacy, safety, accessibility, and domain review may be required. Organizations should pilot the method, document where it fails, and adapt it to their risk appetite.

NIST AI RMF supplies the Govern, Map, Measure, and Manage risk functions. The NIST Generative AI Profile adds generative-AI considerations. OWASP’s LLM guidance supplies application abuse categories. EBB’s claim record, boundary sequence, and expiry-trigger structure are our proposed synthesis; they should not be attributed to those sources.