AI Agents & Orchestration

41 MIN READ

Requirements Engineering for AI Agents

Requirements Engineering for AI Agents means judging trajectories, not single outputs—trace-based evaluation and autonomy ladders make that testable.

Requirements Engineering for AI Agents fails the moment a team writes it the way it writes requirements for deterministic software; because an agent doesn’t produce one correct output, it produces a distribution of possible trajectories, and no fixed acceptance criterion can enumerate all of them. Most portfolios discover this only after a pilot ships: the agent “worked” in the demo and then drifted, over-reached, or stalled in production because nobody specified what “worked” meant beyond the happy path. Requirements discipline is the readiness gate that separates a governed portfolio capability from a stalled pilot.

Where this article sits

Journey stage 4 of 7: Pilots

readiness → use-cases → roi → pilots → kpis → operationalize → scale

this articlelinkedjourney stagepillar

Your trail so far

The articles you visit light up on this map.

What Is Requirements Engineering for AI Agents, and Why Does It Break Classic Requirements Decomposition?

Requirements engineering for AI agents specifies an agent’s goal, its permitted autonomy, the tools and data it may touch, and the acceptance criteria that judge nondeterministic, multi-step behavior; rather than a fixed mapping from one input to one correct output (Requirements engineering for artificial intelligence systems: A systematic mapping study). That study establishes requirements engineering for AI systems as a research area distinct from classic requirements engineering, and the distinction is not academic: it changes what a Story’s “done” column has to say.

The Determinism Assumption in Acceptance Criteria

Classic acceptance criteria assume a deterministic mapping from input to output, which is why a requirement can enumerate the exact correct result and a test can check the system against that single answer. Deterministic specification works because the same input always produces the same output: a payment API either charges the correct amount or it doesn’t, and the criterion is binary. Agentic systems break that assumption. An agent given the same goal and the same starting state can take different tool sequences, ask different clarifying questions, or reach the goal by a different path on two separate runs: each of which might be individually acceptable, and none of which matches a single enumerated “correct” trace. The systematic mapping study’s central finding is that this probabilistic behaviour is what forces requirements engineering for AI systems to diverge from classic requirements engineering rather than merely extend it: elicitation, specification, and validation methods built for input-output determinism don’t transfer cleanly to a system whose output is a distribution over acceptable behaviours. The practical break shows up first in the Story column labeled “acceptance criteria”: a criterion written as “returns result X” has nothing to check against when the agent might legitimately produce X, X’, or X” depending on which tool call it chose first.

Goals, Autonomy, and Tool Access

An agent requirement has four parts, the goal it pursues, the autonomy it’s permitted, the tools and data it can reach, and the acceptance criteria that judge the run, and all four must be specified before a team can call the agent “requirements-ready.” The goal states the outcome in business terms, the same way a user story states an outcome. Autonomy states how much independence the agent has to reach that goal: whether it can execute directly, must propose and wait for approval, or falls somewhere on a graduated scale between the two. Tool and data access states which systems the agent may call and which knowledge sources it may read, because an agent’s effective behaviour is bounded as much by what it can touch as by what it’s told to do. Acceptance criteria state how a run is judged once it finishes, goal achieved, autonomy bound respected, tools used validly, replacing the single expected result with a permitted space of behaviour. Nondeterminism is why all four have to be written explicitly instead of assumed: a team that skips the autonomy or tool-access fields is implicitly granting the agent whatever access the underlying platform exposes, which is rarely the access anyone intended.

Agent Requirements in the SAFe Hierarchy

Agent requirements decompose across the same four tiers a portfolio already uses for any other investment, with each tier of the SAFe Requirements Model carrying a different kind of agent-specific artifact. An Epic states the strategic outcome and the measurable benefit an agentic capability is expected to deliver. A Capability translates that outcome into solution intent; what the system must be able to do to realize it. A Feature narrows solution intent into usable, deliverable behaviour a team can build in a Program Increment. A Story makes one agent interaction or one control testable, with acceptance criteria a team can actually verify at the end of an iteration. This scaled decomposition is the backbone the rest of this guide maps onto: every later section, acceptance modeling, autonomy boundaries, context requirements, security controls, attaches to one of these four levels, and naming the level up front is what keeps an autonomy decision made at the Epic from evaporating by the time the work reaches a Story.

How Agent Nondeterminism Forces a New Acceptance-Criteria Model

Trace-based evaluation judges an agent by scoring the full sequence of tool calls and decisions it took to reach an outcome, not by comparing a single output to a single expected answer; because when behavior is a distribution over trajectories, “done” has to mean an acceptable trajectory, not a fixed result. That single shift is the hardest requirements problem agentic systems introduce, and getting it wrong is expensive: teams that keep writing single-answer acceptance criteria end up either failing agents that behaved correctly or passing agents that reached the right answer by an unacceptable path.

Single-Answer Correctness to Trajectory Quality

Single-answer correctness fails for agent behaviour because a multi-step agent doesn’t map one input to one output: it plans, calls tools, observes results, and adjusts, and any of several tool sequences can land on an acceptable outcome. Trace-based evaluation replaces the single check with a judgment on the whole run: did the agent call valid tools in a sensible order, did its intermediate decisions stay within its permitted scope, and did it recover sensibly when a tool call failed. Outcome reliability is the metric that captures this at the acceptance-criteria level, the rate at which an agent reaches an acceptable end state across repeated runs of the same task, and it replaces single-answer correctness as the bar nondeterministic behavior is actually judged against. A Story that states “the agent books the meeting” is not requirements-complete; a Story that states “the agent books the meeting, and across 20 repeated runs of the same request reaches an acceptable outcome at or above the agreed reliability threshold” is.

What Trace-Based Evaluation Scores

Trace-based evaluation scores three things across a run: tool-use validity (did every tool call fall inside the agent’s permitted scope and use valid parameters), decision quality (did each step follow logically from the state before it, rather than looping, guessing, or ignoring available information), and outcome correctness (did the run end in a state that satisfies the goal). InfoQ’s practitioner analysis of evaluating AI agents makes the structural point directly: agents are systems, not models, and single-turn accuracy metrics don’t capture how a multi-turn system actually fails in production (Evaluating AI Agents in Practice: Benchmarks, Frameworks, and Lessons Learned).

A trace that scores well on outcome correctness but poorly on tool-use validity is still a requirements failure, the agent got lucky, not compliant, which is why acceptance criteria have to name all three dimensions rather than collapsing to “did it work.”

Reliability Across Repeated Runs

Reliability across repeated runs is what turns trace scoring into a countable acceptance gate for agent reliability: instead of running a task once and checking the trace, a team runs the same task N times and requires the outcome-reliability rate to clear an agreed threshold before the Story is accepted. This matters because a single successful trace proves the agent can succeed, not that it reliably will: an agent that reaches the acceptable outcome 6 times out of 10 has a materially different production risk profile than one that reaches it 9 times out of 10, even though both produced at least one passing trace in review. Setting the threshold is a requirements decision, not a testing afterthought: a customer-facing refund agent needs a far higher reliability bar than an internal draft-generation agent, and that difference belongs in the Story’s acceptance criteria, not in a tester’s judgment call after the fact.

Completion, First-Pass Approval, and Rework Rate

The AgentCenter task-specification pattern turns “works well enough” into countable Story-level gates by pairing behaviour fields, title, description, acceptance criteria, priority, with measurable outcomes: completion rate, first-pass approval rate, rework rate, and quality score. This structure resembles the machine-readable capability manifest documented in the GSA-TTS AI Agent Specification Template, which formalizes an agent’s permitted actions as an auditable table rather than free-text instructions GSA-TTS AI Agent Specification Template (AI Agent Specification Template). Completion rate states how often the agent finishes the task at all. First-pass approval rate states how often the output is accepted without a human correction cycle: the sharpest signal of whether the acceptance criteria were specified tightly enough in the first place. Rework rate is the inverse: how often a human has to intervene and redo part of the agent’s work, which converts directly into the labor cost the agent was supposed to remove. Quality score rounds out the set with a graded judgment of output quality where binary pass/fail doesn’t capture nuance. Naming all four in the Story’s acceptance criteria, rather than leaving them as dashboard metrics collected after the fact, is what makes “works well enough” a testable requirement instead of a subjective call made after deployment.

Required context, the field that specifies which knowledge sources and retrieval scope the agent may draw on when producing its output, belongs to the data-and-context requirements covered later; here it matters only as a reference: an acceptance criterion that names completion rate without also constraining required context is measuring an agent against a moving target.

Encoding Behavioural Constraints as Testable Rules

A reusable agent acceptance criterion states four conditions together: the goal was met, the agent stayed within its autonomy bounds, its tool calls were valid, and the outcome reproduced across N runs above the agreed reliability threshold. Constitutional AI is the canonical example of turning a behavioural constraint into something testable rather than aspirational: instead of relying on human-labeled examples of harmful output, the method encodes a list of rules or principles that the system checks its own outputs against, converting a vague “be harmless” expectation into rules a reviewer, or an automated harness, can actually evaluate against a trace Constitutional AI (Constitutional AI: Harmlessness from AI Feedback).

The same logic applies below the safety layer to ordinary business constraints. “The agent should be helpful but not overstep” is not a testable rule; “the agent may draft a refund but must not execute one above $500 without approval” is, because it names an action class, a threshold, and a required response. Score the trace, not the single answer: a reviewer checking a Story’s acceptance criteria should be able to point at a specific step in a logged trace and say which rule it satisfied or violated, rather than forming an overall impression of whether the run “felt right.” Rework rate falls when constraints are written this precisely, because ambiguous constraints are exactly what generates disputes about whether an agent’s output should count as a pass.

Mapping Agent Requirements Onto a Scaled Epic-Capability-Feature-Story Hierarchy

Each level of a scaled requirements hierarchy needs a distinct agent-requirement artifact; strategic outcome and hypothesis at Epic, solution intent at Capability, usable system behaviour at Feature, and a single testable interaction at Story. Collapsing agent requirements into one undifferentiated document is exactly how autonomy and safety intent gets lost between the level where it’s decided and the level where code gets written.

Epic-Level: The Epic Hypothesis Statement

The Epic Hypothesis Statement is the concrete artifact for specifying an agent investment at the Epic level, capturing the hypothesis, the leading indicators that will confirm or refute it, and the minimum viable product that tests it before full-scale investment Epic Hypothesis Statement (Epic Hypothesis Statement – Scaled Agile Framework). For an agent Epic, the hypothesis states the business outcome the agent is expected to produce and the mechanism by which it produces it: not “an AI agent will help with customer support” but “an agent that drafts first-response replies will reduce average response time by a stated margin while keeping first-pass approval above an agreed floor.” Framing the hypothesis this specifically is what makes it falsifiable, and a hypothesis that can’t be falsified isn’t a requirement, it’s a hope.

Leading Indicators and MVP

Leading indicators for an agent Epic are the metrics teams check before the full rollout proves or disproves the hypothesis; completion rate, first-pass approval rate, and rework rate from the earliest Stories, tracked against the thresholds set in the hypothesis rather than waited on until a full quarter of production data accumulates. The MVP for an agent Epic is deliberately narrow: a single agent interaction, in a single Feature, tested against real inputs at low autonomy, rather than a fully autonomous system rolled out across every use case the Epic eventually envisions.

The narrower the MVP, the faster the leading indicators produce a real signal, and the cheaper it is to kill a bad hypothesis before it consumes a full Program Increment of Feature-level investment across multiple teams.

Encoding Autonomy Intent at Portfolio Level

Autonomy intent has to be written into the Epic Hypothesis Statement itself, not left for Feature or Story authors to infer, because the portfolio-level decision about how much independence this agent is ultimately meant to have shapes every later Feature’s scope. An Epic that states “this agent will eventually operate with minimal human review” implies a different security posture, a different tool-access footprint, and a different acceptance-criteria bar than one that states “this agent stays advisory indefinitely.” Writing that intent down at the Epic gives every downstream Capability and Feature owner a portfolio-level anchor to check their own autonomy decisions against, rather than each team independently guessing how much latitude the business actually wants the agent to have.

Solution Intent to Usable Behaviour

Solution intent at the Capability level translates the Epic’s hypothesis into what the system must be capable of doing, and the Feature level narrows that further into usable, deliverable behaviour a single team can build inside a Program Increment. Where the Epic states the business outcome, the Capability states the functional shape that achieves it; for a customer-support agent Epic, the Capability might state that the system must be able to classify incoming requests, retrieve relevant account context, and draft a response within a bounded autonomy level. The Feature then breaks that Capability into a concrete, buildable slice: “classify and route incoming requests” becomes one Feature, “draft a response from retrieved context” becomes another, each with its own acceptance criteria and its own autonomy and tool-access footprint appropriate to its narrower scope. This is where required-context and tool-permission requirements (covered later) first become concrete, because a Feature is specific enough to name the actual data sources and actual tools a team will build against.

Making One Agent Interaction Testable

A Story makes a single agent interaction or a single control testable by a team within one iteration, which means it inherits the Feature’s autonomy and tool-access scope and narrows it to one verifiable behaviour with acceptance criteria a reviewer can check against a trace. “The agent drafts a response from retrieved account context” is a Feature-level statement; “given a billing question with a known account, the agent drafts a response that cites the correct plan tier, stays within its advisory autonomy bound, and reaches an acceptable outcome across repeated runs above the reliability threshold” is a Story. The difference is testability: a Story that can’t be checked against a specific trace by the end of an iteration isn’t sized correctly, whatever level of the hierarchy it claims to sit at.

Preserving Autonomy with Traceability Links

Autonomy and safety requirements specified at the Epic are routinely diluted or dropped by the time work reaches a Story, unless traceability links carry that intent down explicitly through each hop, Epic to Capability, Capability to Feature, Feature to Story, so a reviewer can trace any Story’s acceptance criteria back to the portfolio-level intent that authorized it. Research on requirements engineering in large-scale agile system development documents the underlying tension directly: requirements engineering at scale requires upfront, detailed analysis that has to coexist with agile flow, and organizations that let flow win without preserving that upfront analysis lose exactly the kind of cross-cutting intent that autonomy boundaries represent (Requirements engineering challenges and practices in large-scale agile system development).

A worked example makes the failure mode concrete. An Epic states that a support agent must stay advisory, no unattended execution, for its first two quarters. That intent has to appear explicitly in the Capability’s solution intent, again in each Feature’s autonomy field, and again in every Story’s acceptance criteria, or a team three hops removed from the Epic will build a Story that quietly grants execution rights because nothing in front of them said otherwise. The traceability link isn’t bureaucratic overhead; it’s the only mechanism that stops a portfolio-level safety decision from being silently overwritten by a team-level implementation choice.

Hierarchy Level	Agent-Requirement Artifact	What It Fixes
Epic	Epic Hypothesis Statement	Business outcome, leading indicators, MVP scope, portfolio-level autonomy intent
Capability	Solution intent	Functional shape the system must exhibit to realize the hypothesis
Feature	Usable, deliverable behaviour	Concrete tool access, required context, and autonomy scope for one buildable slice
Story	Testable interaction	One agent interaction or control with trace-checkable acceptance criteria

Specifying Autonomy and Human-in-the-Loop Boundaries as Acceptance Criteria

Agent autonomy levels replace ambiguous “agentic” language with a measurable expectation by defining a small ladder of operating points and tying each one to the oversight it requires, so a requirement states exactly how much independence an agent has been granted rather than gesturing at a vague degree of self-direction. Vague autonomy language is where most agent requirements quietly fail: not because nobody thought about safety, but because “the agent should be mostly autonomous but still safe” can’t be checked against anything.

Autonomy Levels and Required Oversight

A workable autonomy ladder ties each level to a specific oversight mechanism rather than a general comfort level: an agent that only drafts and waits for a human to send requires no runtime intervention capability at all, because nothing executes without review; an agent that executes low-risk, easily reversible actions and notifies a human afterward requires real-time notification and a rollback path; an agent that executes consequential, hard-to-reverse actions requires a pre-execution approval gate rather than a post-hoc notification. Harvard Business Review’s analysis of agentic AI system design makes the underlying principle explicit: companies that succeed with agentic AI design workflows around outcomes and appoint accountable owners rather than treating oversight as a bolt-on control layered over a system built without it Harvard Business Review (Designing a Successful Agentic AI System).

From “Agentic” to Measurable Independence

“Agentic” stops being a slogan the moment a requirement states which action classes the agent may execute unattended, which require notification, and which require prior approval; three concrete buckets replace one adjective. A procurement agent, for example, might be permitted to draft purchase requests unattended, execute purchases under a stated dollar threshold with post-hoc notification, and require prior approval for anything above that threshold. Writing the ladder this way turns a design conversation about “how autonomous should this be” into a requirements decision with a specific, auditable answer for every action the agent can take.

Mapping Action Classes to Approval Gates

Mapping action classes to approval gates means every distinct type of action an agent can take gets its own entry on the ladder, rather than one blanket autonomy setting applied to the whole agent. A single support agent might have three action classes in play at once: answering informational questions (unattended), issuing account credits under a threshold (notify-after), and closing accounts (approve-before). Treating autonomy as a single scalar collapses those three risk profiles into one number and inevitably sets it either too loose for the riskiest action or too tight for the safest one; mapping it per action class avoids both failure modes.

Tool-Permission Boundaries as Governed Interfaces

Tool-permission boundaries name which tools and data stores an agent may access and model that access as a governed interface rather than an instruction embedded in a prompt, which is what makes change control and audit tractable. A prompt that tells an agent “only use the read-only customer API” is a suggestion the model can ignore or misinterpret under adversarial input; a tool registry that only exposes the read-only endpoint to that agent’s runtime is a boundary the model cannot cross regardless of what the prompt says. Fraunhofer’s work on concept-guided LLM agents for human-AI safety codesign demonstrates this pattern directly, using a structured system-model graph to guide an agent’s reasoning within predefined safety concepts rather than trusting instruction-following alone to hold the boundary (Concept-Guided LLM Agents for Human-AI Safety Codesign).

Treating tool access this way also makes it a first-class requirement rather than an implementation detail: a Feature’s acceptance criteria should name the exact tool set an agent may call, because a tool added later without updating the requirement is a scope change that bypassed review.

Writing a Human-in-the-Loop Acceptance Criterion

A human-in-the-loop acceptance criterion names four things together, the action class, the required approver role, the intervention or rollback path if the action goes wrong, and the audit record that proves the gate was honored, converting “a human should check important actions” into a testable Story requirement. “Action class: refund issuance above $500. Required approver: team lead. Intervention path: refund can be reversed within 24 hours via the payments console. Audit record: approval timestamp and approver ID logged against the transaction” is a criterion a reviewer can verify against a trace; “a human should be in the loop for large refunds” is not.

Writing the criterion at this level of specificity also surfaces gaps before they become incidents: a team that tries to fill in “intervention path” and discovers there isn’t one has found a real requirements gap, not a hypothetical one, and can fix it before the Story reaches production rather than after an agent takes an action nobody can undo.

one question · 10 seconds

Quick one while it is in front of you: what part of specifying this agent is actually giving your team trouble?

Data and Context Requirements: Specifying What an Agent Needs to Know

What an agent is allowed to know is a requirement as binding as what it’s allowed to do, which means knowledge sources, retrieval scope, freshness windows, and grounding expectations belong in the specification rather than living implicitly inside a prompt that nobody reviews as a requirements artifact. Software-engineering research on agentic systems is converging on the same point from the specification-inference angle: the central difficulty in building trustworthy agentic workflows is deciphering developer intent precisely enough that the agent’s inputs, not just its instructions, are traceable to that intent (Agentic AI for Software: thoughts from Software Engineering community).

Required Context as First-Class Requirement

Required context specifies the knowledge sources, retrieval scope, freshness window, and grounding expectation an agent needs to perform its task, and the AgentCenter task-specification pattern’s dedicated “required context” field is the practitioner anchor for treating this as a requirement rather than an implementation detail left to whichever retrieval pipeline happens to be wired up. A Story that specifies a goal and acceptance criteria but leaves required context implicit is incomplete in the same way a Story that leaves tool access implicit is incomplete: the agent’s actual behaviour depends as much on what it can retrieve as on what it’s told to do, and an unspecified retrieval scope means the requirement can’t predict what the agent will actually say.

Knowledge Sources, Scope, and Freshness

Knowledge sources name the specific systems an agent may retrieve from, a product catalog, a policy document set, a customer’s account history, and scope bounds how far into those systems the agent may reach, because “search the knowledge base” is not the same requirement as “search the current-quarter pricing pages of the knowledge base.” Freshness states how current the retrieved information must be before the agent may rely on it: an agent answering pricing questions from a document set that’s six months stale is producing wrong answers with high confidence, and freshness is the requirement that catches that failure mode before it reaches a customer. Grounding, data freshness, and context scope together form the substrate specification a Feature owner has to fill in before a team can reasonably estimate the Story.

Grounding Acceptance Criteria

A grounding acceptance criterion states that the agent must answer from its approved knowledge sources and signal uncertainty rather than fabricate an answer when those sources don’t cover the question; turning “don’t make things up” from an aspiration into something a trace review can check. A reviewer checking a grounding criterion looks for two things in the trace: did every substantive claim in the agent’s output trace back to a retrieved passage, and did the agent explicitly flag the cases where retrieval came back empty or irrelevant rather than filling the gap with a plausible-sounding guess. An agent that grounds correctly 95% of the time but silently fabricates the other 5% has failed the requirement, because the failure mode is precisely the one grounding was written to catch.

Governing RAG Sources as Permissioned Inputs

Retrieval-augmented generation sources and their access permissions belong in the specification as declared, testable inputs rather than implicit prompt content, because a RAG source an agent can reach is functionally equivalent to a tool it can call: it shapes behaviour and it needs the same change-control discipline. Governing RAG sources as permissioned inputs means naming, in the Feature or Story, exactly which indexes or documents the agent’s retrieval layer is allowed to query, the same way tool-permission boundaries name which APIs it may invoke. Doing this keeps data dependencies visible at the Feature level, before a team commits to building against a knowledge source that might change scope, get deprecated, or turn out to carry information the agent shouldn’t have access to at its assigned autonomy level.

How Much Retrieved Context Should a Story Budget For?

A context budget states the maximum volume of retrieved passages an agent may carry into a single run, separate from which sources it may query or how fresh they must be, because a retrieval scope that’s correctly bounded can still flood the agent’s working context with more passages than it can weigh coherently. Teams that leave this unspecified typically discover the failure late: a retrieval layer that returns everything remotely relevant produces a context window crowded enough that the agent’s grounding degrades even though every individual passage came from an approved source. Naming a context budget in the Story, a maximum passage count or token allocation for retrieved material, distinct from the model’s overall context window, gives a reviewer something concrete to check when a grounding failure traces back to an overloaded rather than an incorrect retrieval.

Writing Agent Stories: From Spec-Driven Development to Story-Level Acceptance

Spec-driven development is the practice of authoring an explicit, executable specification before any code or agent behaviour is generated, so the specification, not the prompt that invokes an AI system, is the source of truth a team builds and tests against. ThoughtWorks documents spec-driven development as one of the defining AI-assisted engineering shifts of 2025, describing it as a paradigm that uses well-crafted requirement specifications, aided by AI coding agents, to generate executable code and behaviour rather than treating a prompt as a disposable instruction (Spec-driven development: Unpacking one of 2025’s key new AI-assisted engineering practices).

Spec-Driven Development as Source of Truth

A spec written before generation states the agent’s goal, autonomy bound, tool scope, and acceptance criteria in a form an engineer or an AI coding agent can act on directly, and keeping that spec as the durable artifact, rather than the prompt used to produce a particular run, is what preserves traceability when the underlying model or tooling changes. Behaviour-Driven Development scenarios are the natural bridge format here: a Given/When/Then scenario written against agent behaviour (“Given a billing question with a known account, when the agent is asked to respond, then it drafts a reply grounded in the account’s current plan and stays within its advisory autonomy bound”) translates a spec-driven artifact directly into something a Story can carry as an acceptance scenario, checkable against a logged trace the same way a BDD scenario is checkable against a test run.

Turning Specs Into SAFe Stories

Turning a spec into a backlog Story means stating the user or business outcome, attaching the agent-specific acceptance criteria this guide has built section by section, goal achieved, autonomy bound respected, tool scope honoured, trace check passed, and linking the Story upward to the Feature that scoped its tool access and required context. The reusable sequence is: outcome, executable spec, Story with acceptance criteria, trace-based test, review gate. Each step produces an artifact the next step consumes, which is what keeps the chain auditable end to end rather than collapsing into an ad hoc conversation between a product owner and whoever happens to be tuning the prompt that week. Multi-agent systems built to operationalize this pattern, such as AgileCoder, show the same structure from the execution side: coordinated agents that consume Agile Methodology artifacts directly rather than treating a codebase as an undifferentiated blob of instructions produce more traceable, more reviewable output than a single agent working from a loose natural-language brief Agile Methodology (AgileCoder: Dynamic Collaborative Agents for Software Development based on Agile Methodology).

LLM-Assisted Story Drafting and Its Limits

LLMs can draft a first-pass user story from an initial requirement, and empirical work on user story generation through prompt engineering shows they do this competently for well-scoped functional requirements while still needing human refinement to catch missing edge cases, ambiguous acceptance criteria, and priority calls that depend on business context the model doesn’t have (Enhancing User Story Generation in Agile Software Development Through Open AI and Prompt Engineering). A multi-agent approach to requirements elicitation and analysis tested across four real-world projects found meaningful variation between models on this exact task, one model returning the fastest drafts, another producing higher-similarity user stories on complex requirements, which is itself a requirement: a team adopting LLM-assisted story drafting has to validate model choice against its own requirement complexity rather than assuming any general-purpose model performs identically (AI based Multiagent Approach for Requirements Elicitation and Analysis). The practical pattern is to let an LLM produce the draft Story from the spec, it’s genuinely fast at converting a spec into story-shaped prose, and then route every draft through the same human review a hand-written Story would get, checking specifically for the four agent-specific acceptance criteria this section built: an LLM asked to draft a story rarely fills in autonomy bounds or tool scope on its own unless the spec it drafted from already specified them, which is exactly why spec-driven development has to precede story generation rather than follow it.

Security and Risk Requirements for Agentic Systems: OWASP-LLM as Story-Level Controls

The OWASP Top 10 for LLM Applications gives requirements engineers a concrete risk taxonomy that converts directly into agent security requirements; prompt injection, excessive agency, insecure output handling, and sensitive-information disclosure each become an explicit, testable control rather than a general “be secure” instruction layered on after the fact.

OWASP Top 10 as Security Taxonomy

Each OWASP-LLM risk maps to a specific control class a Story can carry: prompt injection requires input validation that treats retrieved and user-supplied content as untrusted regardless of how authoritative it looks; excessive agency requires the tool-permission boundaries and autonomy ladder this guide has already specified, since the OWASP category is precisely the failure mode those two mechanisms exist to prevent; insecure output handling requires that anything the agent produces which gets executed downstream, a generated command, a generated query, passes through the same validation a human-authored equivalent would; sensitive-information disclosure requires that required-context and grounding rules explicitly exclude data the agent’s requester isn’t authorized to see, not merely data the prompt didn’t ask for.

Prompt Injection and Excessive Agency

Prompt injection is the risk that content an agent retrieves or receives contains instructions designed to override its actual task, and the control isn’t a smarter prompt: it’s architectural: the agent’s tool-permission boundary has to hold even when the content it’s processing tells it to do something outside that boundary, because a boundary that only holds against well-behaved input isn’t a boundary. Excessive agency is the risk that an agent has been granted more autonomy or more tool access than its task actually requires, and the requirements-side control is the autonomy ladder and tool-permission boundary specified earlier in this guide: an agent scoped to draft-only for refunds cannot exhibit excessive agency on refunds no matter what an injected instruction tells it to do, because the capability to execute one simply isn’t present.

Insecure Output Handling and Data Leakage

Insecure output handling names the risk of treating an agent’s output as trusted the moment it’s downstream of a validation step meant for human-authored input: a generated SQL fragment or shell command needs the same sanitization a human-typed one would get, not an exemption because an AI system produced it. Data leakage names the risk that an agent’s grounding or context retrieval surfaces information the requester isn’t authorized to see, which is why required-context specifications need to carry access-control scope, not just topical scope: “retrieve from the policy knowledge base” is incomplete without “scoped to the requester’s own account and permission tier.”

S2C-SAFe and IEC 62443-4-1 Compliance

Security compliance can be engineered into a scaled agile framework rather than bolted on after delivery, which is the evidence S2C-SAFe provides directly: the extension makes SAFe compliant with the IEC 62443-4-1 secure-development standard by integrating security requirements into the framework’s existing artifacts instead of adding a parallel compliance process (How to Integrate Security Compliance Requirements with Agile Software Engineering at Scale?). For agent requirements specifically, this means the OWASP-LLM controls above don’t need a separate security backlog; they attach to the same Feature and Story artifacts already carrying the agent’s functional and autonomy requirements, reviewed by the same teams at the same cadence. SAFe’s own advanced-topics guidance treats security and compliance the same way, as a discipline integrated into existing competencies rather than a bolt-on gate at the end of the pipeline (SAFe Advanced Topics – Scaled Agile Framework).

Governing Tool Use as Auditable Interfaces

The Model Context Protocol is the concrete tool-integration surface these story-level controls attach to: an open standard for wiring agents to tools and data sources, where each MCP server an agent is permitted to call is an enumerable, governable requirement rather than an implicit capability buried in a system prompt. Governing tool use this way makes change control and audit tractable in a way prompt-embedded instructions never can: a security review can enumerate exactly which MCP servers a given agent’s Story authorizes, check that list against the OWASP-LLM controls above, and flag a scope change the moment a new server is added, rather than discovering the expanded access only after an incident. Distinguishing deterministic automation from agentic decision-making matters here too: routing a request through a fixed, auditable script rather than granting an agent open-ended tool access is often the safer and cheaper control, and requirements engineers should default to that distinction before reaching for agency at all: a judgment this guide develops fully next.

When Not to Use an Agent: Deterministic Automation vs Agentic Decisions

The first requirements decision for any candidate agent task is whether an agent is warranted at all, and the honest answer is often no: agents are for judgment-heavy, variable, context-dependent steps, and deterministic automation is cheaper, safer, and easier to verify for everything else.

The Agent-vs-Automation Decision Checklist

A four-criterion checklist decides whether a task warrants an agent: variability of input (does the correct response genuinely depend on interpreting context that varies case to case, or is it the same handful of branches every time), consequence of error (what happens if the system gets it wrong), reversibility (can a wrong action be undone cheaply, or is it permanent), and oversight cost (how much human review does running this as an agent actually require versus running it as a script). Judgment-heavy tasks, the ones where the correct action depends on interpreting variable context rather than applying a fixed rule, are the class of work where agency pays for the nondeterminism it introduces; routine, low-variance tasks are not, and dressing them up as “agentic” only adds evaluation overhead, retry latency, and an expanded security surface a deterministic script never carries.

Agency justification is the requirements artifact that operationalizes this checklist: a recorded, one-paragraph case for why a given step needs an agent rather than a script, owned alongside the Story it authorizes. Absent a justification, the default is a script, per the Agile Practice Guide’s broader principle that teams should choose the lightest process capable of delivering the outcome reliably rather than defaulting to the most sophisticated available tool (Agile Practice Guide).

How Misplaced Agency Harms Flow

Flow efficiency, the share of a Story’s lead time spent on active work rather than waiting, is the metric a misplaced agent quietly degrades: agent steps add evaluation and retry latency on top of whatever work they’re actually performing, and that latency shows up in the value stream as flow-efficiency loss even when the agent eventually reaches the correct output. A deterministic script either runs correctly or fails loudly and immediately; an agent that’s marginally uncertain about a step may retry, second-guess, or wait for a clarifying signal, and each of those adds wall-clock time a script spends zero time on. Positioning agency as a requirements gate, clear the checklist, write the justification, only then scope the Story as agentic, stops that flow cost from being paid on tasks that never needed the agent’s judgment in the first place.

Roles and Ownership of Agent Requirements in an Enterprise Delivery Organization

An agentic build needs four distinct competencies, AI product manager, system architect, AI governance specialist, and security engineer, and the practical move for most delivery organizations is mapping those competencies onto existing roles rather than creating four new job titles.

Competency Set Mapped to SAFe Roles

Product Management owns Feature intent for an agentic capability the same way it owns Feature intent for any other capability, translating the Epic Hypothesis Statement’s business outcome into the Feature-level behaviour a team builds. The Release Train Engineer owns flow, facilitating the train-level cadence where agent requirements get planned, sequenced, and de-conflicted across the teams building interdependent pieces of an agentic system. The System Architect owns solution intent and the autonomy boundaries that flow from it, translating the portfolio-level autonomy decision recorded in the Epic Hypothesis Statement into the tool-permission boundaries and required-context specifications a team can actually build against. The AI governance specialist is the one role in this set that carries genuinely distinct accountability rather than an extension of an existing one, compliance and oversight requirements, the OWASP-LLM control mapping, and the traceability audit that confirms autonomy intent survived from Epic to Story, work that doesn’t naturally sit inside Product Management, the RTE’s flow mandate, or the System Architect’s solution-intent scope. SAFe’s knowledge base documents Product Management, the Release Train Engineer, and the System Architect as established roles with exactly these mandates, giving a delivery organization existing job descriptions to extend rather than roles to invent from nothing System Architect (SAFe Knowledge Base – Scaled Agile Framework).

A RACI from Epic to Story

A lightweight RACI keeps ownership unambiguous at every hop from Epic to Story: Product Management is accountable for the Epic Hypothesis Statement and responsible for the Feature-level outcome it authorizes; the System Architect is accountable for the autonomy boundary and tool-permission scope at Capability and Feature; the AI governance specialist is consulted at every level where a compliance or oversight requirement attaches and accountable for the traceability audit; the RTE is informed at every level and accountable for the cadence that gets agent requirements reviewed on the same rhythm as everything else in the train. Discussions of redesigning work around AI systems reach the same conclusion from the organizational-design side: giving an agentic capability an unambiguous chain of accountable owners, rather than treating it as a shared responsibility nobody specifically owns, is what determines whether governance holds under delivery pressure (Human Robot Agent: Redesigning Work in the Age of AI). Avoiding role proliferation matters practically, not just organizationally: a fifth or sixth new title for every agentic capability a portfolio ships is a staffing cost most organizations can’t sustain past the second or third pilot, while extending four existing roles with a defined slice of new accountability scales with the portfolio instead of against it.

Measuring Requirements Quality and Agent Performance: Maturity and Trace-Based Evidence

Requirements engineering maturity for AI-based systems remains low and inconsistently applied across the industry, which is precisely the finding that makes explicit measurement of agent requirements quality a necessity rather than a nice-to-have (How mature is requirements engineering for AI-based systems? A systematic mapping study on practices).

Requirements-Quality Measures for Agent Specs

Three measures gauge whether an agent’s requirements are actually complete rather than superficially thorough: completeness of stated autonomy bounds (does every action class the agent can take have an explicit autonomy entry, or are some left to default assumptions), presence of trace-based acceptance criteria (does the Story name the reliability threshold and the trace-check dimensions this guide has built, or does it still read like a single-answer criterion with an AI label attached), and traceability coverage from Epic down to Story (can a reviewer follow the autonomy and safety intent recorded at the Epic through every Capability and Feature hop to the Story that finally implements it, without a gap). A Story can pass every functional test and still fail all three of these measures, which is exactly the gap the maturity study identifies at the industry level: the practices exist, but consistent application of them does not.

Performance Metrics as Epic Leading Indicators

Completion rate, first-pass approval rate, rework rate, and quality score, the AgentCenter behaviour-outcome fields introduced earlier as Story-level acceptance signals, also feed back upward as the leading indicators an Epic Hypothesis Statement committed to tracking. Research on AI-driven assistants inside scaled agile software development shows these outcomes are measurable in practice, not just in principle: AI assistance integrated into a scaled agile development process produces countable effects on team productivity and delivery outcomes that a portfolio can track the same way it tracks any other Epic-level leading indicator (The Potential of AI-Driven Assistants in Scaled Agile Software Development). Wiring the same four metrics into both the Story-level acceptance gate and the Epic-level hypothesis check closes a loop that’s otherwise easy to leave open: without that link, a Story can pass its own local acceptance criteria indefinitely while the Epic’s actual business hypothesis quietly goes unconfirmed.

Reading Rework and Approval Signals

A rising rework rate or a falling first-pass approval rate signals that an agent’s requirements were left too loose: the acceptance criteria didn’t constrain behaviour tightly enough, so the agent produces output a human has to routinely correct. The opposite pattern, a completion rate that’s high but a quality score that stays flat or declines while rework stays unusually low, often signals the reverse problem: requirements written so tight, or an autonomy bound set so narrow, that the agent is technically completing tasks without doing anything a looser specification wouldn’t have done just as well by hand, meaning the automation isn’t actually earning its evaluation and oversight cost. Reading these two signals together, rather than optimizing either one in isolation, is what tells a team whether to loosen a requirement, tighten one, or conclude the agency-justification checklist from earlier in this guide would have said no in the first place.

Non-Functional Requirements for AI Agents: Latency, Cost, and Reliability Budgets

An agent’s non-functional envelope needs three explicit budgets, a latency target, a cost-per-task ceiling, and a reliability target expressed as a service-level objective on trajectory success rate, because none of the functional acceptance criteria this guide has built so far say anything about whether the agent is fast enough, cheap enough, or dependable enough to be worth running in production.

Latency, Cost-per-Task, and Reliability Budgets

A latency budget states the maximum acceptable time between a request and a completed agent response, and for multi-step agents that call several tools in sequence, that budget has to account for the full chain rather than any single call. A cost-per-task ceiling states the maximum acceptable spend per completed task, with token cost as the dominant, genuinely variable line item for LLM-driven agents; unlike a fixed-cost script, an agent’s spend per task can swing meaningfully with how many tool calls, retries, and context-window tokens a given run consumes. InfoQ’s practitioner analysis names exactly this pattern: operational constraints, latency, cost per task, and token efficiency, are first-class evaluation targets, not afterthoughts, because they determine whether a technically capable agent is viable at enterprise scale at all (Evaluating AI Agents in Practice: Benchmarks, Frameworks, and Lessons Learned). A reliability target expressed as a service-level objective on trajectory success rate, not merely system uptime, closes the set: an agent can be up 99.9% of the time and still fail its actual job most of those runs if the SLO only measures availability instead of measuring whether trajectories actually reach acceptable outcomes.

NFR Budget	What It Bounds	Where It Attaches
Latency target	Maximum time from request to completed response, across the full tool-call chain	SAFe Feature, as a constraint/enabler requirement
Cost-per-task ceiling	Maximum spend per completed task, denominated in token cost	SAFe Feature, tied to Epic measurable benefit
Reliability SLO	Trajectory success rate across repeated runs, not uptime	SAFe Feature, paired with Story-level outcome reliability

Tying Operating Cost to Epic Benefit

Cost-per-task has to tie back to the Epic Hypothesis Statement’s measurable benefit, because an agent that “works” but costs more per task than the value it produces is a requirement failure the functional acceptance criteria will never catch on their own. A support-drafting agent that reliably produces accurate replies at a token cost exceeding what a human agent’s time would have cost for the same volume has passed every functional and trace-based check in this guide while failing the one that actually justified building it. Distinguishing NFRs from functional acceptance criteria matters precisely because they fail independently: a Story can meet its outcome-reliability threshold and its tool-scope requirements while blowing through its cost or latency budget, and only an NFR check written into the Feature, separate from, but reviewed alongside, the functional criteria, catches that failure before it reaches production at scale.

Who Watches the Budgets Once the Agent Is Live?

A budget written into a Story is a one-time acceptance check unless someone owns watching it after deployment, which is why latency, cost-per-task, and the reliability SLO each need a monitoring owner and an alert threshold, not just a target value in a requirements document. Production traffic drifts from the traffic a Feature was tested against, prompt lengths grow, tool calls that used to succeed on the first try start retrying, a knowledge source doubles in size, and any of those drifts can push a budget that passed acceptance testing into breach without a single line of the agent’s logic changing. Naming the monitoring owner and alert threshold alongside the budget itself, the same way the RACI names an accountable owner for autonomy and tool-permission scope, keeps a slow budget breach from surfacing only when a stakeholder notices the agent got expensive or slow.

Operationalizing Agent Requirements: A Maturity Path for Enterprise Portfolios

Requirements discipline is what converts a promising agent pilot into dependable portfolio capability, and the AI agent maturity path from one to the other runs through three stages: an isolated pilot, governed Feature-level delivery with explicit autonomy and human-in-the-loop acceptance, and a portfolio-level operating model for agents.

Staged Maturity Path: Pilot to Portfolio

Roughly two-thirds of enterprises adopting generative AI remain stuck in isolated pilots that never scale, precisely because scaling requires an operating model and most organizations never build one; Scaled Agile’s framing of AI-Empowered Agility argues that the organizations that do scale are the ones that already have a Lean-Agile framework capable of anchoring AI initiatives to strategic goals and measured outcomes rather than treating each pilot as an isolated experiment AI-Empowered Agility (Beyond the Pilot: Achieving True AI-Empowered Agility – Scaled Agile Framework). The staged path this guide has built section by section maps directly onto that argument: an isolated pilot has a goal and maybe a demo, but rarely has explicit autonomy bounds, trace-based acceptance, or Epic-to-Story traceability; governed Feature-level delivery adds all three, along with the required-context and NFR budgets that keep the agent’s behaviour and its operating cost both bounded; a portfolio-level operating model extends governance across every agentic capability at once, with the RACI and role-mapping this guide covered earlier applied consistently rather than reinvented per pilot.

Onboarding Agents as Team Members

Treating an agent like a new team member who needs a defined role, scoped access, and a clear standard for what “good work” means is the framing Harvard Business Review’s guidance on onboarding AI agents applies directly, and it’s exactly the requirements discipline this guide has built from a different angle: a defined role is the goal and autonomy bound, scoped access is the tool-permission boundary and required context, and a clear standard for good work is the trace-based acceptance criteria Harvard Business Review (Create an Onboarding Plan for AI Agents). An agent that joins a portfolio without an onboarding-equivalent set of requirements is functionally the same as a new hire given system access on day one with no job description and no manager; technically capable of doing something, accountable to no clearly stated standard for what that something should be.

Governing Autonomy by Principle at Scale

Principle-based decision-making governs agent autonomy by pre-agreed principles rather than per-case approval, which is what preserves flow at scale once a portfolio has more than a handful of agentic capabilities running at once; research on realising benefits in a scaled agile environment finds that principle-based approaches outperform purely rule-based ones precisely because they adapt to the iterative, evolving nature of agile delivery instead of requiring a new rule for every edge case (Principle-based decision-making: realising benefits in a scaled agile environment). Applied to agent autonomy specifically, this means a portfolio states principles once, when to prefer deterministic automation over agency, how autonomy ladders scale with action consequence, what a human-in-the-loop criterion must always name, and lets every Epic Hypothesis Statement, Feature, and Story inherit those principles instead of re-litigating them from scratch. That’s the difference between a portfolio that governs ten agentic capabilities with the same discipline it governed the first one, and a portfolio where every new agent requires its own bespoke safety review because nothing from the last one carried forward.

The natural next step for an organization reading this as a checklist against its own portfolio is a requirements-maturity assessment: locating where the organization actually sits on this path, isolated pilots, governed Features, or a working portfolio operating model, and prioritizing the specific gaps, whether that’s missing traceability links, unwritten autonomy ladders, or an Epic Hypothesis Statement that was never revisited once the pilot shipped.

Summary

Requirements engineering for AI agents is not classic requirements engineering with an AI label attached: it is a distinct discipline built around a single structural fact: agent behaviour is a distribution over trajectories, not a fixed mapping from input to output, and every artifact this guide has covered exists to make that distribution governable.

Trace-Based Acceptance Is the Load-Bearing Mechanism

Every other requirement in this guide, autonomy ladders, tool-permission boundaries, required context, NFR budgets, OWASP-LLM controls, ultimately gets checked the same way: against a logged trace, scored for tool-use validity, decision quality, and outcome reliability across repeated runs, rather than against a single expected answer. A portfolio that adopts trace-based acceptance as its default acceptance model has already solved the hardest part of agent requirements engineering, because every other artifact in this guide, the Epic Hypothesis Statement’s leading indicators, the Story’s human-in-the-loop criterion, the Feature’s required-context specification, exists to feed that trace with the right evidence rather than to replace it. Teams that skip straight to writing autonomy ladders or security controls without first committing to trace-based acceptance end up with well-specified requirements that nobody can actually check against a running system, because the single mechanism that would let a reviewer verify any of them against real behaviour was never put in place.

Traceability From Epic to Story Is What Makes Autonomy Decisions Stick

The second load-bearing mechanism is traceability: an autonomy bound, a security control, or a cost ceiling decided at the Epic or Feature level only survives to production if it’s carried explicitly through every hop to the Story that implements it, because nothing about SAFe’s decomposition process preserves intent automatically. A portfolio that has done the work of writing the Epic Hypothesis Statement, the autonomy ladder, the tool-permission boundary, and the trace-based acceptance criteria, but has no traceability link connecting them, has produced a set of well-written documents sitting in different places, not a governed agentic capability. The organizations that convert agent pilots into dependable portfolio capability are the ones that treat traceability as load-bearing infrastructure rather than paperwork: every Story a reviewer opens should trace cleanly back to the portfolio-level decision that authorized it, with nothing lost at any hop in between.

Anonymous. Counted, not tracked.

Where is your organisation with this right now?

What is the hardest part where you are?

agility at scale