AI Governance & Ethics

44 MIN READ

NIST AI Risk Management Framework (AI RMF): Complete Implementation

The NIST AI Risk Management Framework has no legal force, yet regulators treat it as the default standard, covering four functions and a six-phase rollout.

Can a voluntary framework function as a mandatory standard? The NIST AI Risk Management Framework carries no legal force, yet federal agencies, sector regulators, and ISO certification bodies now treat it as the reference architecture every serious AI governance program has to reckon with; and organizations that skip it discover the gap only when a regulator, auditor, or incident forces the question.

What is the NIST AI Risk Management Framework?

The NIST AI Risk Management Framework (AI RMF 1.0) is a voluntary framework published in January 2023 by the National Institute of Standards and Technology for identifying, assessing, and managing AI risk across the full system lifecycle, structured around two components: the AI RMF Core and AI RMF Profiles. Voluntary status sounds like an opt-out. In practice, the framework closes in on organizations from three directions at once, and none of them require a mandate to bite.

NIST AI RMF definition, scope, and authority basis

The AI RMF gives organizations a structured vocabulary and process for AI risk rather than a checklist of controls to pass or fail. It was developed in an open, transparent, multidisciplinary, and multistakeholder manner over an 18-month period, in collaboration with more than 240 contributing organizations spanning private industry, academia, civil society, and government (AIRC).

That authority basis matters because it distinguishes the AI RMF from a vendor whitepaper or a single agency’s internal policy. Consensus development means the categories and subcategories inside the framework reflect friction points that surfaced repeatedly across hundreds of organizations: not one team’s theory of what AI risk should look like. Released on January 26, 2023, the framework built directly on public comment periods, workshops, and multiple draft cycles (OECD.AI). The two components that resulted, the Core and the Profiles, carry distinct jobs, and conflating them is the most common misreading of the document.

The AI RMF Core: Four Functions Structure

The AI RMF Core organizes AI risk management into four functions, Govern, Map, Measure, and Manage, each broken into categories and subcategories that define specific outcomes an organization should be working toward. It is the part of the document practitioners cite most often, because it is where the framework stops being philosophy and starts being a checklist of governance outcomes.

The Core does not prescribe how to achieve those outcomes; it defines what a mature program produces. A team building an AI Risk Register, a risk tolerance statement, or a bias-testing pipeline is executing Core subcategories whether or not it labels the work that way. This separation between outcome and method is deliberate: it lets a five-person startup and a multinational bank both use the same Core without the document dictating either one’s operating model.

AI RMF Profiles: Current and Target State Documentation

AI RMF Profiles are the mechanism organizations use to document where their governance practices stand today against where they need to be, expressed against the same Core functions and categories. A Profile is not a score; it is a structured gap map that turns the Core’s abstract categories into a concrete comparison an organization can act on.

Profiles matter because the Core alone offers no starting point for prioritization; every organization would otherwise treat every category as equally urgent. A completed Current State and Target State Profile pair tells a governance team exactly which categories carry the widest gap. That gap becomes the input every later implementation phase depends on. NIST has since published sector- and technology-specific Profiles, including one scoped to generative AI, extending the same current-versus-target logic to narrower contexts.

Voluntary vs de facto mandatory status

Voluntary on paper does not mean optional in practice: Executive Order 14110 directed federal agencies to use the AI RMF for AI safety and security, sector regulators including the SEC and the Federal Reserve Board reference it in AI supervisory guidance, and ISO/IEC 42001 was explicitly designed to be compatible with it. Each of those three facts operates independently, which is what makes the combined effect hard to opt out of.

A vendor selling into federal agencies inherits the Executive Order’s requirement indirectly, because the agency’s own AI procurement standards trace back to it. A bank’s AI model risk function inherits the SEC and Federal Reserve reference indirectly, because examiners increasingly ask whether AI-specific risk categories exist in the model risk inventory at all. An organization pursuing ISO/IEC 42001 certification inherits the AI RMF’s vocabulary directly, since the certifiable standard was built to map onto it; none of these paths requires a statute naming the AI RMF; they arrive through adjacent legal and audit pressure instead.

Relationship to existing risk management frameworks

The AI RMF is AI-specific and lifecycle-oriented, explicitly addressing sociotechnical risks, fairness, transparency, accountability, that generic enterprise risk management frameworks were never built to capture. A traditional ERM framework treats risk as a financial or operational exposure with a dollar value attached; the AI RMF treats risk as something that can also degrade trust, fairness, and legal standing without ever showing up as a line-item loss.

That gap is not cosmetic. A model that produces disparate outcomes across demographic groups can operate for months without tripping any financial risk threshold, while accumulating regulatory and reputational exposure the entire time. The AI RMF’s Map and Measure functions exist specifically to help that category of risk emerge before it becomes a financial one. Yoshua Bengio has described the AI RMF as an important first step toward internationally harmonized AI risk governance: a characterization that fits the framework’s role as connective tissue between US practice and the emerging international standards landscape, including ISO/IEC 42001.

Who the framework is designed for

The AI RMF targets AI actors across the full lifecycle, designers, developers, deployers, evaluators, and end users, rather than a single functional role, which is why chief risk officers, AI governance leads, compliance teams, and enterprise architects all read different sections of the same document for different purposes. A chief risk officer reads Govern for accountability structure. An engineering lead reads Map and Measure for the technical controls their models need to pass. A compliance officer reads the Profile mechanism to build the audit trail regulators expect.

This breadth is a deliberate design choice, not scope creep. AI risk does not originate in a single function: a biased dataset (data science), an unclear escalation path (governance), and an unmonitored production deviation (operations) are three separate failure points that would fall through the cracks of a framework aimed at only one audience. National Institute of Standards and Technology guidance documents typically address a narrower technical audience; the AI RMF’s decision to span roles is itself a signal of how the agency views AI risk as cross-functional by nature.

Key outputs the framework produces for an organisation

Implementing the AI RMF produces four concrete artifacts an organization did not have before: a documented AI risk tolerance statement, a Current State and Target State Profile pair, a Core-aligned governance structure with named accountability, and a repeatable cycle for identifying and treating new AI risk as systems change. None of these artifacts exist by default in most organizations building or buying AI today.

The absence of a risk tolerance statement is the most common gap, because it forces a decision, how much AI risk the organization will accept, that many leadership teams have never explicitly made. Once written down, that statement becomes the reference point every subsequent Map, Measure, and Manage decision gets checked against, which is what turns AI governance from ad hoc judgment calls into a consistent, defensible practice.

NIST AI RMF Core Functions: GOVERN, MAP, MEASURE, MANAGE

The four AI RMF Core functions, Govern, Map, Measure, and Manage, operate as a continuous cycle rather than a linear sequence, with Govern providing the accountability layer that the other three depend on to function at all. Treating them as four sequential project phases is the single most common structural mistake organizations make when they first read the document, and it is worth naming before walking through each function on its own.

NIST AI RMF GOVERN Function

Govern establishes the organizational practices, policies, processes, and culture that make AI risk management possible; organizational roles and responsibilities, AI risk management policy, documented risk tolerance, and team-level practices are its four core category areas. Without Govern, the other three functions happen inconsistently across teams or do not happen at all, because no one is accountable for whether they happen.

The category that separates functioning Govern implementations from paper ones is risk tolerance. A documented statement, signed at the executive level, tells every downstream team what “acceptable AI risk” means in specific, applicable terms rather than as a vague aspiration. Teams operating without that statement resolve ambiguous risk calls inconsistently, which is precisely the failure mode Govern exists to close.

NIST AI RMF MAP Function

Map identifies and categorizes AI risk in context, covering the intended use of a system, the populations it affects, the deployment environment, and a prioritized inventory of the risks that combination creates. Map is where risk stops being generic and starts being specific to what a given system actually does and who it touches.

A single model architecture can carry different risk profiles depending on context: a recommendation engine used for retail product suggestions carries different stakes than the same architecture used for credit eligibility screening. Map forces that context onto the record before any measurement work begins. Skip it, and teams end up measuring the wrong things later. Contextual categorization under Map also captures legal and regulatory context, giving downstream Measure and Manage work a documented reason for the priority order it follows.

MEASURE function: analysis, assessment, and metrics

Measure analyzes, assesses, and monitors the risks Map identifies, producing quantitative evidence, bias testing, performance evaluation, fairness metrics, robustness testing, and ongoing shift monitoring, that Govern decisions can actually be checked against. Measure is the function most implementation programs underfund, because it requires tooling investment that Govern and Map largely do not.

Fairness metrics under Measure are not a single number; they typically combine several distinct statistical tests run against a model’s outputs across protected groups, checked against monitoring for concept drift as production data shifts away from the distribution a model was trained on. A model that passes fairness testing at launch and is never measured again is a governance program with a Measure function that exists on paper only. The gap between organizations that treat Measure as a one-time bottleneck and organizations that treat it as continuous instrumentation is the single clearest predictor of whether an AI RMF program produces real risk reduction.

MANAGE function: prioritisation, response, and monitoring

Manage prioritizes and implements risk response; treatment plans for identified risks, decisions to accept residual risk, incident response procedures, and a continuous improvement loop that feeds findings back into Govern. Manage is where an AI risk register stops being a document nobody reads and starts changing what teams actually build.

A risk register with hundreds of identified findings and no resourced treatment plans is not evidence of a mature program; it is evidence that Map and Measure are outpacing Manage. The corrective move is to size Manage capacity to match Map and Measure output before scaling risk identification further; otherwise the gap between risks found and risks treated widens every quarter, and audit exposure grows with it. Manage-function findings, including near-miss incidents, are also the primary input that should trigger a Govern policy update, closing the loop the four functions are built around.

NIST AI RMF MEASURE Function

Measure’s second job, beyond producing metrics, is generating the evidence artifact that governance decisions and external audits actually rely on. A quantitative fairness report, a deviation-monitoring dashboard, and a documented explainability output are not just technical byproducts; they are the paper trail that turns a governance claim into something a regulator or board member can independently verify.

Organizations that build Measure tooling only to satisfy internal engineering curiosity, without designing outputs for an external audience, typically have to rebuild their reporting layer the first time a regulator or customer due-diligence team asks for evidence. Treating Measure outputs as audit artifacts from the start, with version history, timestamps, and named ownership, avoids that rework and shortens the time it takes to answer an external risk questionnaire.

How the four functions interact as a cycle

Govern sets the policy context, Map identifies what risks exist within that context, Measure quantifies those risks with evidence, and Manage responds to them; after which Manage’s findings feed back into Govern to update policy, closing a loop rather than terminating a project. Govern (culture and accountability), Map (context and risk identification), Measure (risk analysis and evaluation), and Manage (risk response and monitoring) is the framework’s own one-line summary of the sequence (OECD.AI).

The cycle framing has a practical consequence: there is no finish line for AI RMF implementation. A system that passed every Map and Measure check at launch can drift into new risk territory as its training data ages, as its user population changes, or as regulation shifts underneath it. Programs that treat AI RMF adoption as a project with a completion date typically discover the gap during the first incident that occurs after the “project” was declared done.

How to Implement the NIST AI Risk Management Framework

Implementing the NIST AI Risk Management Framework runs through six sequential phases, inventory, profile creation, governance structure, Map execution, Measure tooling deployment, and Manage process embedding, each producing a specific deliverable before the next phase can proceed. Peer-reviewed case-study evidence supports exactly this staged approach: Swaminathan and Danks develop a six-step process tailored to the specific risk profile of facial recognition technology when applying the AI RMF to surveillance use cases AI RMF (Semantic Scholar).

Phase 1: AI System Inventory and Classification

The first implementation phase catalogues every AI system in production and development, classifies each by risk tier using AI RMF risk characterization criteria, and assigns a named system owner, producing an AI System Registry as the phase’s deliverable. Nothing downstream is possible without this step; Map, Measure, and Manage all operate on a per-system basis, and a system missing from the registry is a system no later phase will ever touch.

The classification step is where organizations most often underscope the exercise. It limits the inventory to internally built models and misses AI embedded in purchased software. A registry that excludes third-party AI creates a governance blind spot that widens every time procurement approves a new vendor tool with an embedded model. Treating the registry as a living document, updated at procurement intake, not just at annual review, closes that gap before it compounds.

NIST AI RMF Profiles

Phase 2 uses AI RMF Profiles to document current state, what governance practices already exist, against target state, what practices should exist, and prioritizes the resulting gap for the highest-risk systems first. This phase converts the abstract four-function structure into a concrete, system-specific comparison that the rest of the implementation sequence can act on.

Current State Profile

A Current State Profile records what governance practices exist today for a given system, function by function, rather than what a policy document claims should exist. Most first-pass Current State Profiles reveal more gaps than teams expect, because policy documents and lived practice diverge more than governance leads assume before they run the exercise.

Building the Current State Profile candidly, including categories where the answer is “nothing exists yet”, is what makes the Target State comparison useful. A Current State Profile inflated to look better than the underlying reality produces a Target State gap analysis that understates the real work ahead, which shows up later as missed deadlines rather than a candid starting assessment.

Target State Profile

A Target State Profile defines the governance practices a system should have given its risk tier, expressed in the same categories as the Current State Profile so the two can be directly compared. High-risk systems typically warrant a Target State that covers every Core category in depth; lower-risk systems can target a leaner subset without under-governing the risk that actually exists.

The comparison between the two Profiles, not either Profile alone, is the deliverable that drives prioritization. A system with a wide Current-to-Target gap on a high-risk classification should outrank a system with a narrow gap on a low-risk classification, regardless of which team asks first, and making that prioritization rule explicit prevents implementation resources from following organizational politics instead of actual risk.

Phase 3: Governance Structure, Roles, Policies, and Risk Tolerance

Phase 3 establishes the accountability structure the Govern function requires, an AI Risk Committee for executive oversight, a policy layer for AI ethics, and operational AI Risk Officers, and publishes the risk tolerance statement and governance policy that give the rest of the program authority. Without a named structure, Govern-function activity has no owner and tends to stall the moment its original champion changes roles.

AI Risk Committee and Executive Oversight

The AI Risk Committee is the executive body that reviews AI risk at the portfolio level, approves the organization’s risk tolerance statement, and signs off on residual risk acceptance for the highest-tier systems. Committee membership typically spans risk, legal, engineering, and business leadership, because AI risk decisions cut across all four functions simultaneously.

A committee that meets quarterly but never reviews specific systems, only aggregate metrics, misses the decisions that actually require executive judgment, such as accepting residual bias risk on a high-stakes system after remediation has hit diminishing returns. Giving the committee a standing agenda item to review named systems, not just dashboard totals, keeps its oversight connected to real decisions instead of becoming a reporting formality.

AI Ethics Board and AI Risk Officers

Where the AI Risk Committee sets policy and accepts risk at the executive level, an AI Ethics Board and operational AI Risk Officers translate that policy into system-level practice: the Ethics Board resolving ambiguous fairness and use-case questions, and Risk Officers embedded closer to development teams executing Map and Measure activities day to day.

This two-layer structure exists because executive committees cannot realistically review every system-level decision, and development teams cannot realistically set organization-wide risk tolerance on their own. Risk Officers who sit inside or adjacent to engineering teams catch context that a quarterly committee review would never surface, while the Ethics Board provides a consistent forum for the judgment calls too ambiguous for either group to resolve alone.

Phase 4: MAP Execution for Each AI System

Phase 4 runs the Map function against every system in the registry, contextual characterization of intended use and affected populations, risk identification against AI RMF risk categories, and prioritization, producing a Risk Register entry for each system as the deliverable. This is the phase where organization-wide policy from Phase 3 meets system-specific reality for the first time.

Running Map consistently across dozens or hundreds of systems requires a repeatable template, not a bespoke exercise for each one; without standardization, risk registers become incomparable across systems and prioritization collapses into guesswork. A shared Map template, the same categories, the same rating scale, applied by every team, is what lets a governance lead credibly say one system carries higher priority risk than another, rather than relying on whichever team wrote the most alarming description.

Phase 5: MEASURE tooling and metrics setup

Phase 5 deploys the technical stack Measure requires, bias testing, performance monitoring, and explainability tooling, and establishes a KPI dashboard, producing a Measurement Framework as the deliverable. This is typically the most resource-intensive implementation phase, because it is the first phase requiring engineering build capacity rather than policy and process work.

Bias Testing Tools: Fairlearn and IBM AI Fairness 360

Fairlearn, built by Microsoft, and IBM AI Fairness 360 are the two bias-testing toolkits organizations most commonly deploy to run fairness metrics against model outputs across protected groups before and after deployment. Both compute standard fairness statistics, demographic parity and equalized odds among them, against a model’s predictions, giving Measure a quantifiable basis instead of a subjective sense that a model “seems fair.”

one question · 10 seconds

Right now, what is actually stalling your NIST AI RMF rollout?

Deploying either toolkit is a starting point, not an endpoint; fairness metrics require a defined threshold and a documented remediation path before a bias test produces anything actionable. A team that runs Fairlearn once at launch and never again has built a bias-testing capability without a bias-monitoring practice, which is the gap that later becomes visible as an undetected fairness regression in production.

Performance Monitoring: Evidently AI and Arize AI

Evidently AI and Arize AI are the platforms organizations typically deploy to track model performance and concept drift continuously in production, feeding the Measure function’s ongoing-monitoring requirement rather than its one-time-evaluation requirement. Drift monitoring catches the case fairness testing at launch cannot: a model whose training-time performance held but whose real-world input distribution has shifted since deployment.

The value of continuous monitoring compounds over a system’s lifetime, because the risk that a static launch-time assessment misses grows the longer a system stays in production without re-evaluation. Explainability tooling, most commonly SHAP (SHapley Additive exPlanations), rounds out the Measure stack by attributing individual predictions to specific input features; evidence that becomes essential the first time a stakeholder asks why a model made a specific consequential decision.

Phase 6: MANAGE Process and Continuous Improvement

Phase 6 documents risk treatment plans for every identified risk, establishes an incident response process, implements re-validation triggers, and sets a review cadence, producing Risk Treatment Plans and an Incident Response Playbook as the deliverable. This is the phase that determines whether the previous five phases produce risk reduction or simply produce documentation.

An Incident Response Playbook needs a working definition of what counts as an AI incident before it can be useful: a definition broad enough to capture fairness regressions and shift, not just outright system failures. Re-validation triggers close the loop with Phase 2’s Profiles: a material change to a system’s training data, use case, or deployment context should automatically trigger a fresh Current State assessment rather than waiting for the next scheduled annual review.

NIST AI RMF Best Practices for Enterprise AI Governance

Ten NIST AI RMF practices account for most of the difference between enterprise governance programs that function and ones that stall, ranked by regulatory impact and implementation feasibility: AI system inventory, a documented risk tolerance statement, independent review, continuous monitoring, and stakeholder engagement lead the list. NIST states plainly that the Playbook is neither a checklist nor a set of steps to be followed in its entirety; organizations may borrow as many or as few suggestions as apply to their industry and use case (AIRC Playbook), which is the explicit license for distilling the document’s 200-plus suggested actions down to the ones that carry the most weight.

AI system inventory as the foundational governance control

An AI System Inventory is the practice every other AI RMF control depends on, because an organization cannot govern, measure, or remediate risk in a system it does not know exists. The reliable way to catch what engineering self-report misses is a procurement-intake checklist triggered the moment a vendor contract is signed, flagging any embedded AI capability before the tool ever reaches a business unit rather than waiting for a team to self-identify what it bought. Governance programs that skip inventory tend to build sophisticated controls for a handful of visible flagship systems while dozens of smaller, quietly deployed models go entirely ungoverned.

Sizing the inventory exercise correctly matters as much as running it at all. A single governance analyst cannot realistically canvas every business unit, so ownership should split between a central inventory lead who maintains the registry and unit-level points of contact who know what their teams have actually deployed or bought. Disputes over whether a given tool counts as “AI”, a rules engine with a machine-learned scoring step, a SaaS product with an embedded recommendation feature, need a documented tiebreaker, typically the AI Risk Committee or a delegated risk officer, so classification disagreements get resolved once rather than re-litigated every audit cycle. Running the inventory as an annual full sweep with quarterly spot-checks in between catches drift without demanding a continuous full-time headcount.

Risk tolerance statements for AI decision-making

An AI Risk Tolerance Statement only does its job once it is ratified at the executive level: a draft that never gets signed off invites forum-shopping, where a team facing an ambiguous risk call escalates to whichever reviewer approved a similar call last time rather than checking the call against a standard the whole organization is bound by. That gaming pattern is what separates a statement that exists on paper from one that actually governs decisions.

Ratification closes the loophole by giving every team the same answer regardless of which reviewer they ask. A published, ratified statement also creates an audit trail: when a risk call is later questioned, the organization can point to the specific tolerance threshold the decision was checked against, rather than reconstructing an ad hoc judgment call after the fact.

Independent review and challenge of model outputs

Independent review establishes a challenge function, reviewers separate from the team that built a model, because model developers cannot objectively assess the fairness, robustness, or risk of their own work with the same rigor an outside reviewer applies. This is the practice most often skipped under delivery pressure, precisely because it adds a review cycle that a self-assessment does not.

The challenge function does not need to be a separate department to be effective; a rotation where teams review each other’s high-risk systems against a shared rubric captures most of the independence benefit without the overhead of a dedicated review organization. What matters is that the reviewer has no stake in the model shipping on schedule: that single condition is what separates a genuine challenge function from a rubber-stamp review.

Continuous monitoring over point-in-time assessment

Continuous monitoring only functions as a practice if it has a named owner and a clear escalation path, not just a dashboard someone occasionally checks: an owner assigned per system, a defined SLA for acting once an alert fires, and a monthly cadence for reviewing systems that have not tripped any threshold at all.

That SLA needs teeth: an alert that sits unactioned for weeks because no one owns the follow-up is functionally the same as no monitoring at all. Continuous monitoring does not require re-running a full assessment on a fixed schedule; it requires automated deviation and performance signals that trigger a targeted re-assessment the moment a threshold is crossed, with the monthly no-threshold-tripped review acting as a backstop for risks the automated signals were not tuned to catch.

Stakeholder engagement including affected communities

The AI RMF explicitly calls for engaging the communities a system affects in its risk assessment, not only the technical teams that built it, because the people experiencing a system’s consequences frequently see risks that internal reviewers structurally cannot. Joy Buolamwini’s and Virginia Eubanks’s work has repeatedly demonstrated that populations affected by algorithmic systems emerge fairness and equity problems long before those problems register in an internal technical review.

Operationalizing this practice means building a feedback channel for affected users into the Map and Measure functions directly, not treating stakeholder input as a one-time consultation before launch. A complaint or appeal mechanism tied back to the Risk Register turns affected-community feedback into a Measure-function signal instead of a customer-service ticket that never reaches the governance program.

Joy Buolamwini

Joy Buolamwini’s Gender Shades study tested commercial facial-analysis systems from IBM, Microsoft, and Face++ against a benchmark deliberately built to span a wide range of skin tones and genders, and found error rates for darker-skinned women running many times higher than for lighter-skinned men. The finding did more than expose one dataset’s flaw: it became the reference case the field cites when a fairness claim needs subgroup evidence before anyone accepts it, and it helped found the Algorithmic Justice League as a standing audit body rather than a one-off research paper.

That finding maps directly onto the AI RMF’s Measure function: disaggregated evaluation, not a single top-line metric, is what the framework’s fairness-testing guidance calls for. Programs that report a single accuracy or fairness score to their governance committee, without a subgroup breakdown underneath it, are structurally reproducing the exact reporting gap Buolamwini’s research identified as the mechanism that let biased systems ship undetected.

How the NIST AI RMF Addresses Human Oversight and Algorithmic Bias

Human oversight and algorithmic bias are the two interpretations of the NIST AI RMF that governance leads search for most specifically, and the framework addresses both through named subcategories rather than a single dedicated section; oversight through Govern 3 accountability structures and Manage response checkpoints, bias through Map, Measure, and NIST SP 1270’s taxonomy. Reading the framework for either topic in isolation misses how tightly the two are wired together in practice.

Where human oversight lives in the RMF: GOVERN 3 accountability and MANAGE response checkpoints

Human oversight in the AI RMF is not a single control; it lives in Govern 3’s workforce and accountability categories and in Manage 2 and Manage 4’s documented response plans, which build human decision checkpoints directly into how the organization responds to identified risk. The framework’s own description of trustworthy AI includes “accountable and transparent” as a named characteristic, which is the conceptual anchor these subcategories operationalize.

In practice, this means oversight shows up as human-in-the-loop approval gates for consequential decisions, documented override and kill-switch authority, logging every time oversight is invoked, and periodic calibration reviews that check for automation bias: the tendency to over-trust model output simply because it came from a model. A governance program that treats “a human can technically intervene” as sufficient, without logging how often that intervention actually happens, cannot demonstrate the accountability Govern 3 calls for.

RMF risk-proportionate oversight vs EU AI Act Article 14 prescriptive duty

The AI RMF treats human oversight as risk-proportionate guidance calibrated to a system’s context, while the EU AI Act’s Article 14 imposes a prescriptive human-oversight duty specific to high-risk AI systems: a structural difference that matters for any organization operating under both regimes simultaneously. The AI RMF asks what level of oversight a given risk profile warrants; Article 14 specifies oversight requirements a qualifying system must meet regardless of an organization’s own risk calculus.

Organizations subject to both frameworks typically use the AI RMF’s flexible categories to build the oversight program, then check the result against Article 14’s specific requirements for any system that falls under EU high-risk classification. Building oversight controls to the stricter of the two standards from the outset, rather than a US-only baseline retrofitted later, avoids the rework that comes from discovering a gap only after an EU deployment is already underway.

NIST SP 1270 bias taxonomy: systemic, computational, and human-cognitive bias

NIST SP 1270 organizes algorithmic bias into three categories, systemic, computational and statistical, and human-cognitive, giving governance teams a shared vocabulary for a problem that is otherwise easy to treat as a single undifferentiated risk. Each category has a distinct root cause and a distinct remediation path, which is why lumping them together produces bias mitigation efforts that miss most of the actual sources.

Systemic Bias

Systemic bias originates outside the model itself, embedded in institutional and societal practices that produced the historical data a model trains on: a hiring model trained on years of biased human hiring decisions inherits that bias structurally, regardless of how carefully the model’s own code is written. No amount of algorithmic tuning fixes a systemic bias problem, because the bias entered the pipeline before the model ever saw the data.

Addressing systemic bias requires intervention at the data and process level rather than the model level; auditing the historical decisions a training dataset encodes, and in some cases excluding or reweighting data reflecting practices the organization no longer wants a model to replicate. This is the category most often missed by teams that treat bias testing as purely a model-output exercise, since the root cause sits upstream of anything a fairness metric on the output would catch.

Computational and Statistical Bias

Computational and statistical bias arises from the technical choices made in building a model; sampling error, unrepresentative training data, or an evaluation metric that inadvertently favors one outcome pattern over another. Unlike systemic bias, this category is directly addressable through standard machine learning practice: rebalancing datasets, adjusting sampling strategy, and selecting evaluation metrics that do not structurally favor the majority class.

This is the bias category most fairness-testing tools like Fairlearn and IBM AI Fairness 360 are built to detect, because it manifests as a measurable statistical disparity in model outputs across groups. Detecting it is comparatively straightforward; the harder problem is deciding which fairness metric to optimize for, since demographic parity and equalized odds can point toward different, sometimes incompatible, model adjustments.

Human-Cognitive Bias

Human-cognitive bias enters through the people building and using the system: the assumptions a data scientist brings to feature selection, or the automation bias a reviewer brings to accepting a model’s output without scrutiny. This category is frequently the hardest to detect through technical testing, because it lives in decisions and judgment calls rather than in code or data that a fairness metric can directly inspect.

Mitigating human-cognitive bias leans more on process design than on tooling; structured review checklists that force explicit justification for feature choices, and calibration exercises that measure how often reviewers defer to model output without independent judgment. Because this category overlaps directly with the automation-bias problem inside human oversight, the two functions of the AI RMF that address it, Measure for bias, Manage for oversight, end up working the same failure mode from different angles.

Mapping bias work to MAP, MEASURE, and MANAGE with named fairness metrics

Algorithmic bias work maps cleanly onto three Core functions: Map 2 categorizes context and the populations a system affects, Measure 2.11 runs fairness and bias evaluation using named metrics such as demographic parity and equalized odds, and Manage 1 prioritizes and responds to whatever bias risk arises from that evaluation. Treating bias as a single Measure-function task, disconnected from Map’s context work, is the most common structural error in bias governance programs.

Fairness Metrics: Demographic Parity and Equalized Odds

Demographic parity measures whether a model produces positive outcomes at similar rates across groups, while equalized odds measures the model’s error rates, both false positives and false negatives, across those same groups. The two metrics conflict: a model can satisfy demographic parity while still producing unequal error rates, and satisfying equalized odds can require accepting some difference in overall positive-outcome rates.

Choosing between them is a decision that belongs in Map, not Measure, because the correct metric depends on the specific harm a use case creates: a lending model where false negatives deny credit unfairly calls for a different metric priority than a content-moderation model where false positives silence legitimate speech. Measure 2.11 evaluates against whichever metric Map’s context work determines is the right one for that system, which is why skipping Map’s context step produces bias testing built on the wrong metric from the start.

Automation bias: when human oversight becomes the bias source

Automation bias, the tendency of reviewers to over-trust model output and rubber-stamp decisions rather than scrutinize them, turns human oversight from a bias control into a bias source, which is exactly the failure mode Measure is positioned to quantify. A human-in-the-loop checkpoint that approves ninety-nine percent of model recommendations without meaningful review is not functioning as oversight; it is functioning as a compliance formality with a human signature attached.

Measuring automation bias requires tracking override rates over time and testing whether reviewers catch seeded errors in a controlled sample, not just establishing that a human technically reviewed each decision. A calibration program that periodically injects known-incorrect model outputs into a reviewer’s queue, and tracks whether they get caught, converts oversight from an assumed safeguard into a measured one; closing the loop between the framework’s human-oversight guidance and its bias-measurement guidance in a single, testable practice.

NIST AI RMF vs Traditional Risk Management Frameworks

ISO 31000, COSO ERM, and ISO/IEC 27001 all predate the AI-specific risk categories the AI RMF was built around; none of them had a fairness, drift, or explainability category to retrofit, because those concepts did not exist as named risk types when each standard was drafted. Gary Marcus has argued that this is not a gap traditional frameworks can close with minor extensions; the underlying risk model itself assumes a predictability AI systems do not offer.

Traditional Risk Management Frameworks

Traditional risk management frameworks, ISO 31000, COSO ERM, and ISO/IEC 27001 among the most widely adopted, share a common design assumption: risk can be identified, assessed, and treated using stable categories that do not shift meaningfully once defined. That assumption holds reasonably well for financial, operational, and information-security risk, which is why these frameworks remain foundational for most enterprises.

The same assumption is what leaves them structurally unequipped for AI-specific risk. None of the three frameworks includes native categories for algorithmic fairness, model drift, or explainability, because none of those concepts existed as named risk categories when the frameworks were designed. Organizations layering AI governance on top of an existing ERM program consistently find that the existing risk register has no natural home for a fairness finding or a drift alert: the taxonomy simply was not built to hold it.

ISO 31000 Risk Management Standard

ISO 31000 is a generic risk management standard intended to apply across any organization and any risk domain, providing a process framework, establishing context, risk assessment, risk treatment, without specifying risk categories tied to any particular industry or technology. Its generality is the source of both its wide adoption and its limitation for AI governance specifically.

Because ISO 31000 deliberately avoids domain-specific content, it says nothing about fairness, explainability, or the stakeholder engagement practices AI risk management increasingly requires. Organizations already running ISO 31000 are not wrong to keep it. The standard’s generic risk process still applies to AI systems at the process level, but what it cannot supply is the AI-specific category vocabulary the AI RMF was purpose-built to provide.

NIST AI RMF vs ISO 31000 (generic risk management)

Where ISO 31000 supplies a domain-agnostic risk process, the AI RMF supplies AI-specific risk categories and a Govern function explicitly focused on building an organizational AI risk culture: the two frameworks operate at different layers rather than competing to answer the same question. An organization does not choose between them; it runs the AI RMF’s categories through ISO 31000’s established process architecture.

In practice, this means an organization with a mature ISO 31000 program can plug AI risk categories directly into its existing risk-assessment and risk-treatment cycle, using the AI RMF’s Map function to identify context-specific AI risks and ISO 31000’s established treatment workflow to act on them. The integration point is the risk register itself; AI risk entries populated using AI RMF categories, treated using the ISO 31000 process the organization already runs for every other risk type.

NIST AI RMF vs COSO ERM (enterprise risk management)

COSO ERM, the dominant enterprise risk management framework, covers strategy, performance, and review across financial and operational risk domains but has no native mechanisms for bias risk, model drift, or AI-specific incident management. COSO’s 2017 update strengthened its treatment of strategy-linked risk, but that update predates the AI-specific risk categories the AI RMF later formalized.

Enterprises running COSO ERM as their board-level risk reporting structure typically treat the AI RMF as a feeder module; AI-specific risk findings roll up into the same enterprise risk reporting COSO already structures, rather than creating a parallel reporting track the board has to learn separately. This integration path keeps AI risk visible at the level where budget and resourcing decisions actually get made, instead of leaving it siloed inside a technical governance function the board rarely reviews directly.

NIST AI RMF vs ISO/IEC 27001 (information security)

ISO/IEC 27001 addresses information security risk, confidentiality, integrity, and availability, which is relevant to AI systems but insufficient for AI governance, because it does not address the algorithmic fairness, transparency, and autonomy concerns the AI RMF makes explicit. A model can be perfectly secure under ISO 27001’s controls and still produce biased or unexplainable outcomes that the security standard has no mechanism to catch.

Organizations pursuing ISO/IEC 42001 certification, the AI-management-system standard explicitly designed to be compatible with the AI RMF, typically run it alongside an existing ISO 27001 certification rather than replacing it, since the two standards protect against different failure categories: 27001 against unauthorized access and data compromise, 42001 and the AI RMF against fairness, transparency, and accountability failures. Article 9 of the EU AI Act requires every high-risk AI system to run a documented risk management system (EU AI Act): a legal benchmark against which each of these three traditional frameworks, evaluated on its own, falls short for AI-specific obligations.

Framework	Primary risk domain	AI-specific coverage	Certifiable standard
NIST AI RMF	AI lifecycle risk	Native, fairness, transparency, drift, accountability	No, voluntary guidance
ISO 31000	Generic organizational risk	None, process only	No, generic standard
COSO ERM	Financial and operational risk	None	No, internal control framework
ISO/IEC 27001	Information security	None	Yes, security certification
ISO/IEC 42001	AI management systems	Native, built to align with AI RMF	Yes, AI-specific certification

NIST AI RMF

The AI RMF’s practical role for an organization running one or more traditional frameworks is an AI risk extension module, add AI-specific risk categories to the existing risk register, add Govern-function practices to the existing governance structure, and add AI-specific measurement to the existing monitoring program, rather than standing up a parallel governance track. This integration framing is what keeps AI governance from becoming a second bureaucracy competing with the first.

Organizations that treat the AI RMF as a replacement for ISO 31000 or COSO ERM typically create duplicate governance overhead; two risk registers, two review cadences, two sets of stakeholders to satisfy. Treating it instead as the AI-specific layer inside an existing risk architecture is both less disruptive to implement and more durable, because it inherits the executive attention and audit legitimacy the existing framework has already earned.

Managing AI Governance Change with the NIST AI RMF

NIST AI RMF programs fail most often from change management breakdowns rather than technical complexity; resistance from AI teams who see governance as bureaucratic overhead, missing executive sponsorship, and system owners who were never trained on what the framework actually requires of them. A technically well-designed Measurement Framework delivers nothing if the teams responsible for using it were never brought along on why it exists.

Executive sponsorship and AI governance culture

AI RMF adoption requires visible senior leadership commitment that survives beyond the executive who first championed it, because a mandate tied to one person’s personal enthusiasm evaporates the moment that person changes roles or moves on. Building sponsorship into the CEO or chief risk officer’s own role, rather than into a specific individual’s personal backing, is what lets the mandate survive a leadership transition intact, instead of forcing the program to re-earn executive attention from whoever holds the seat next.

Sponsorship has to be sustained, not a single kickoff announcement. A program launched with executive fanfare and then left to a mid-level working group without further leadership involvement drifts back toward the bureaucratic-overhead perception the sponsorship was meant to prevent in the first place. Recurring executive review of specific, named systems, not just periodic dashboard updates, is what keeps sponsorship active instead of ceremonial.

AI system owner training and accountability

Every AI system needs a named owner accountable for completing Map and Measure activities, because without named ownership, governance obligations diffuse across a team and go unmet by default rather than by decision. Training for that owner has to cover specific obligations, what Map and Measure require of them for their system, not just an overview of the framework’s four-function structure.

An owner who understands the AI RMF’s structure in the abstract but has never walked through completing a Map exercise for their own system will treat the requirement as someone else’s problem the first time a deadline gets tight. Hands-on training, a facilitated first Map exercise for the owner’s actual system, not a generic workshop, closes that gap and produces an owner who can complete the next cycle independently.

Resistance management from AI development teams

AI engineers often perceive governance as bureaucratic overhead layered on top of their real work, and the durable fix is demonstrating that the AI RMF reduces total rework, catching bias early is measurably cheaper than post-deployment remediation, while automating governance evidence generation through existing MLOps pipelines rather than adding manual paperwork. Timnit Gebru has argued that AI governance culture has to be built from the ground up inside development teams, not imposed from a compliance function sitting outside engineering’s daily workflow.

Automating evidence generation is the single highest-leverage move against resistance, because it removes the perception that governance means extra manual work on top of an already full sprint. A CI/CD pipeline that automatically runs a fairness check and logs the result as part of the existing build process produces Measure-function evidence without asking an engineer to fill out a separate compliance form; governance embedded in the tool an engineer already uses, rather than bolted alongside it.

Phased adoption strategy by risk tier

Phased adoption starts with the highest-risk AI systems, where regulatory pressure is strongest and the case for governance investment is easiest to make, using early gains on those systems to build organizational support for broader rollout. Attempting full-program rollout across every system simultaneously spreads limited implementation capacity too sparse to produce a visible success anywhere.

A single high-risk system taken all the way through inventory, Profile creation, Map, Measure, and Manage produces a concrete, demonstrable result: a governance framework other teams can see working, not just hear described in a policy document. That demonstrated success is what typically unlocks the budget and executive attention needed to extend the program to medium- and lower-risk systems on a realistic timeline.

Embedding NIST AI RMF in existing workflows

Embedding AI RMF requirements into existing CI/CD pipeline gates, model review processes, and sprint ceremonies, rather than creating separate governance processes running parallel to how teams already work, is what determines whether the framework becomes habitual practice or an ignored parallel process. MLOps Governance Integration is the technical mechanism that makes this embedding possible at scale, wiring Measure-function checks directly into the deployment pipeline a system already passes through.

A governance requirement that lives in a separate portal a developer has to remember to visit gets skipped under deadline pressure far more often than one embedded as a required pipeline stage a build cannot pass without. Adapting a structured change model such as the Prosci ADKAR Change Model, awareness, desire, knowledge, ability, reinforcement, to the specific embedding work in each workflow gives teams a repeatable pattern for introducing governance checkpoints without each one feeling like a bespoke negotiation.

Common NIST AI RMF Implementation Challenges

Six failure patterns account for most stalled NIST AI RMF programs: an incomplete system inventory, a Measure-function tooling gap, a disconnect between governance policy and daily operations, uncovered third-party AI risk, stagnant Profiles, and risk registers that grow without matching remediation. Each has a specific diagnostic question that tells a governance lead which failure they are actually looking at.

Incomplete AI system inventory as the foundational failure

Shadow AI, systems deployed without governance review, is the most common inventory gap, diagnosed by comparing the AI system count in the governance registry against the number of active models in the technical model registry; a wide gap between the two numbers is the tell. Remediation runs through an AI discovery exercise across every business unit, cross-referenced against procurement data to catch AI purchased as part of a broader software package rather than flagged explicitly as an AI tool.

Diagnostic: Does the AI System Registry count match the number of active models in the technical model registry?
Remediation: Run a cross-functional discovery exercise with procurement data as a second data source, not engineering self-report alone.

MEASURE function tooling gaps

Organizations frequently complete Govern-function setup, policy, roles, risk tolerance, while never deploying the tooling Measure actually requires, diagnosed by asking whether the organization can produce a quantitative bias measurement for any production AI system today. A governance program that cannot answer that question with a specific number is not yet operating the Measure function, regardless of how complete its Govern documentation looks.

Diagnostic: Can the organization produce a quantitative bias measurement for any production AI system right now, on request?
Remediation: Deploy Fairlearn and Evidently AI as a minimum viable measurement stack rather than waiting for a comprehensive tooling rollout.

Governance-to-operations disconnect

An AI Risk Committee that meets quarterly while Map exercises are not embedded in the model development cycle produces a governance program running on a separate track from actual engineering work, diagnosed by asking whether development teams complete Map before deploying a model. When the answer is no, committee-level oversight is reviewing systems after risk decisions have already been made in practice.

Diagnostic: Do development teams complete a Map exercise before a model reaches production deployment?
Remediation: Add Map completion as a required CI/CD deployment gate, not a parallel review step teams can bypass under deadline pressure.

Third-party AI risk coverage gaps

A stalled third-party coverage gap shows up as a widening spread between two counts that should track together: the vendor contracts procurement has signed with an embedded AI capability, and the number of those systems that actually appear in the AI Risk Register. Tracking that spread quarter over quarter, rather than confirming coverage once at inventory kickoff, is what surfaces a stalled program before an auditor finds the gap first. A governance program with strong internal coverage and no vendor coverage has closed the smaller half of its actual exposure.

Diagnostic: Is every third-party AI system currently in production entered in the AI Risk Register?
Remediation: Reconcile the two counts every quarter; signed vendor-AI contracts vs. Risk Register entries; and treat two consecutive quarters of widening spread as an automatic escalation trigger to the AI Risk Committee, rather than a metric reviewed only when someone remembers to check it.

Profile stagnation and risk without response

Current State and Target State Profiles created once and never revisited stop reflecting reality within a year, and the more useful signal than register size alone is the ratio of open findings to findings with a dated treatment plan: a ratio that should hold roughly steady sprint over sprint, not drift wider. Both failure modes share a root cause; governance activity that produces documentation faster than it produces action.

Diagnostic: When was each system’s Profile last reviewed, and what percentage of identified risks have a documented treatment plan?
Remediation: Institute an annual Profile review cycle and a risk-treatment planning sprint following each Map cycle, sized to actually clear the backlog rather than add to it.

Fairlearn (Microsoft)

Fairlearn’s role inside the broader tooling gap deserves its own note, because it is frequently the single tool organizations deploy to demonstrate Measure-function progress without building the surrounding practice a fairness tool actually requires to be useful. Running Fairlearn against a model produces a set of fairness statistics; it does not by itself define which threshold constitutes a failing result or who owns remediation when one is found.

Framework-level friction compounds this gap: UC Berkeley’s Center for Long-Term Cybersecurity published a formal response to NIST’s Generative AI Profile in June 2024 Generative AI Profile (CLTC), documenting that even NIST’s own framework extensions carry contested areas implementation teams have to track as the guidance itself evolves. A tool like Fairlearn produces numbers against whatever threshold a team defines today; keeping that threshold current with an evolving framework is a governance decision no tool makes on its own.

Measuring NIST AI RMF Effectiveness and Maturity

A five-dimension scorecard, coverage, process maturity, measurement capability, response effectiveness, and outcome quality, turns NIST AI RMF program activity into board-reportable evidence that governance investment is actually reducing AI risk, rather than just counting completed inventories and deployed controls. Effectiveness measurement is a named, first-class element of the framework’s own publication, not an afterthought bolted on by practitioners after the fact.

Five-dimension effectiveness scorecard

Each dimension maps to a specific Core function and a specific measurable target, giving a governance lead a structured way to answer “is the program working” instead of only “is the program active.” Coverage measures whether Govern-function accountability actually reaches every high-risk system; the remaining four dimensions track the other three functions and overall outcomes.

Dimension	Core function	Example metric	Target
Coverage	Govern	% of AI systems in inventory with a named owner	100% for high-risk systems
Process maturity	Map	% of systems with a current risk register entry	100% high-risk, 80% medium-risk
Measurement capability	Measure	% of systems with automated bias/fairness metrics	100% for high-risk systems
Response effectiveness	Manage	Mean time to remediate a high-severity finding	Under 30 days
Outcome quality	Overall	AI incident rate, quarter-over-quarter trend	Declining trend

A weighted average across the five dimensions produces a single overall NIST AI RMF health score suitable for board reporting, giving non-technical stakeholders a single number to track over time without requiring them to read the underlying function-level detail. The weighting itself should shift as a program matures; coverage and process maturity carrying the most weight early, response effectiveness and outcome quality carrying more weight once foundational coverage is already in place.

KPIs for each NIST AI RMF function

Coverage and process-maturity KPIs answer whether the foundation exists; measurement-capability and response-effectiveness KPIs answer whether that foundation is actually producing risk reduction, which is the distinction that separates activity metrics from outcome metrics. A program that reports 100% inventory coverage but cannot report a mean-time-to-remediate figure has strong Govern-function KPIs and an invisible Manage-function performance.

Reporting KPIs by function, rather than as a single blended average, also localizes accountability: a governance lead reviewing the scorecard can immediately see whether a shortfall traces to Map, Measure, or Manage, and route the remediation resourcing accordingly. Blended reporting hides exactly the information a governance lead needs to decide where to invest next quarter’s implementation capacity.

Maturity progression using NIST AI RMF tiers

The framework’s own maturity model runs through four tiers, Partial, Risk Informed, Repeatable, and Adaptive, describing a progression from ad hoc, undocumented practices toward a governance program that adjusts systematically as new risk information arrives. Mapping the five-dimension scorecard against these tiers gives a governance lead a second, independent way to characterize program maturity beyond the raw KPI numbers.

A Partial-tier organization typically scores low on coverage and process maturity simultaneously, since inventory and risk registration are usually the first gaps to close. Reaching Adaptive tier requires the outcome-quality dimension specifically, a declining incident rate over multiple quarters, because that tier is defined by demonstrated systematic improvement, not just the presence of documented process.

Board-level AI governance health reporting

Translating the five-dimension scorecard into board-level language means leading with the single weighted health score and trend direction, then providing function-level detail only where the board specifically asks: a board audience needs the trajectory, not the underlying KPI mechanics. Yoshua Bengio has argued that governance program effectiveness should ultimately be measured by outcomes for affected populations, not process compliance alone, a standard that belongs explicitly in the outcome-quality dimension of board reporting.

A board report built entirely from activity metrics, systems inventoried, policies published, risks conveying progress that has not yet translated into actual risk reduction. Anchoring the report on outcome-quality trend data, alongside the activity metrics that explain why the trend is moving, gives the board the evidence needed to judge whether continued governance investment is earning its return.

Benchmarking against industry peers

Benchmarking AI RMF maturity against industry peers is limited by the absence of a standardized, public disclosure norm; most organizations report scorecard results internally rather than externally, leaving governance leads with process-maturity signals rather than hard peer comparisons. Sector-specific regulatory guidance, where it references the AI RMF, offers a rough proxy for what “adequate” maturity looks like even without a formal benchmark dataset.

In the near term, the more reliable comparison is internal, tracking a single organization’s five-dimension score quarter over quarter, rather than an external peer benchmark that does not yet exist in standardized form. As ISO/IEC 42001 certification adoption grows, certification status itself may become the closest available proxy for external benchmarking, since certification requires demonstrating the same governance maturity the scorecard is designed to measure.

Summary

What separates governance programs that reduce real risk from ones that produce paperwork is not which framework they picked, or even whether that framework carries legal force: it is whether Measure and Manage keep pace with the risk identification work Govern and Map generate. The NIST AI RMF earns its practical authority the same way regardless of its voluntary label, but authority alone does not close the gap between finding risk and treating it.

The Sequencing Discipline That Determines Whether a Program Works

Every implementation failure pattern in this guide traces back to the same root cause in different clothing; inventory outrunning classification, Map outrunning Measure, or risk identification outrunning treatment capacity. The AI System Registry, the Current and Target State Profiles, the Risk Register, and the Measurement Framework are not independent artifacts; they are checkpoints in a single sequence, and skipping ahead on one while the previous checkpoint remains incomplete is what produces a governance program that looks active on paper without reducing risk in practice.

The corrective discipline is resourcing Measure and Manage at the same rate an organization resources Map: a lesson visible in the six common implementation challenges, where four of the six trace directly to Measure or Manage lagging behind risk identification that has already happened. A program that inventories aggressively but measures and remediates slowly accumulates exactly the kind of unaddressed risk register the framework was built to prevent, and the gap compounds every quarter it goes uncorrected.

Where Human Judgment Still Decides the Outcome

No amount of tooling, Fairlearn, Evidently AI, SHAP, or any other named platform in the Measure stack, substitutes for the judgment calls the framework deliberately leaves to the organization: which fairness metric fits a given use case, how much residual risk an executive committee is willing to accept, and when a Profile gap justifies remediation versus documented acceptance. The AI RMF’s voluntary, non-prescriptive design is a feature specifically because those calls cannot be standardized across every organization and every system.

That is also why change management determines outcomes as much as technical implementation does. A program with sophisticated Measure tooling and no executive sponsorship, or comprehensive Map coverage and no trained system owners, stalls at the exact point where a human decision was supposed to close the loop. The organizations that get the most from the NIST AI RMF are the ones that treat the four functions as a discipline for making better AI risk decisions consistently: not as a compliance artifact to file away once the initial implementation phases are complete.

agility at scale