AI Strategy & Transformation

26 MIN READ

AI Integration Layers: Connecting AI to Enterprise Systems

AI integration layers split one job: gateways say no, middleware executes, façades protect legacy systems. Trace one request to find your weakest layer.

AI Integration Layers: Connecting AI to Enterprise Systems sounds like plumbing until a model reaches a system of record and nobody can say who approved the call. Pilots stall at that seam. Each layer between an AI system and the estate carries a different control, and each layer fails in its own way.

Where this article sits

Journey stage 7 of 7: Scale

readiness → use-cases → roi → pilots → kpis → operationalize → scale

this articlelinkedjourney stagepillarrelated (no direct link)

Your trail so far

The articles you visit light up on this map.

What an AI Integration Layer Is: A Governed Seam Between AI and Enterprise Systems

An AI integration layer is the set of APIs, events, gateways, middleware and controlled data access through which an AI system reaches enterprise systems, so that AI runs as a governed workload under the same controls as any other production consumer.

The definition follows the consumer because the consumer is what changes the design. A model is a caller that composes requests no screen ever anticipated, retries when a response looks ambiguous, and holds no accountability of its own. The layer decides what the model may touch.

Enterprise APIs Built for Human Consumers

Enterprise APIs struggle with AI agents because they were built for human-driven, predefined interaction patterns, and agents pursue dynamic, goal-oriented behaviour that those patterns never anticipated Enterprise APIs (arXiv, 2025).

A human works through an interface in a scripted order: search, open a record, edit one field, save. The endpoints behind that interface mirror the order, with permissions granted per role and error messages written for a developer to read. An agent composes calls in sequences nobody scripted, and a permission set sized for a person becomes either too narrow to be useful or too broad to be safe.

The cost of the human-shaped estate shows up in ordinary work. In one AWS Machine Learning Blog scenario, a plant manager needs a single answer before a 10 AM review and spends until 11:15 AM stitching it together from five disconnected systems, while a supervisor and a maintenance technician, each with a different access level, wait hours for exports of the same underlying data (AWS Machine Learning Blog). The data existed, yet it could not speak in one voice.

Reliable tool use is still an open question. API-Bank, an EMNLP 2023 benchmark for tool-augmented LLMs, was built to ask how effective current models are at using tools, how to improve them, and which obstacles remain (API-Bank). A layer designed on that evidence expects imperfect calls: it validates parameters before they reach a system, rejects malformed requests with an error the agent can act on, and makes repeated calls safe.

Ashay Satav’s research on enterprise API and platform strategy in the era of agentic AI puts the dependency plainly: that strategy decides whether AI agents, enterprise data systems and external applications can communicate at all Ashay Satav (Satav, 2025). An enterprise without one discovers the gap a single integration at a time, each with its own credentials and its own failure behaviour.

Enterprise Integration Patterns and an Estate That Is Plural from Day One

Gregor Hohpe’s Enterprise Integration Patterns gives the layer a product-neutral vocabulary for messaging and enterprise application integration, and the AI era extends that pattern language rather than replacing it.

The vocabulary already covers most of what an AI consumer needs. A message router decides where a request goes, a message translator converts formats, and a dead letter channel holds messages that cannot be delivered. AI adds a sender whose messages are less predictable, so the patterns that validate, throttle and quarantine carry more weight than they did with scripted senders.

IBM observes that larger organisations run legacy enterprise application integration for core on-premises systems alongside an Integration Platform as a Service for cloud and SaaS integrations. The layer is therefore plural from the first day: two integration estates, two sets of operational habits, and one AI system expected to reach both.

Every enterprise chooses among three paths: API-based integration, middleware platforms, or fully custom-built AI modules. Each path puts the control points in different layers, which is why the same failure has a different owner depending on the path. A custom-built AI module owns its own identity and audit code, a middleware platform inherits them, and an API-based integration inherits whatever the target API already enforces.

API Gateways and AI Gateways: Where Model Routing, Agent Identity and Action Policy Are Enforced

The API gateway and the AI gateway are both the edge of the AI integration layer, but the API gateway enforces traffic policy on business capabilities while the AI gateway enforces model and agent policy on what a model may ask for.

Treating the two as one box leaves a gap between them: a request can pass every traffic check and still ask for an action no agent should take. The gateway is where the enterprise says no before a model acts.

Dimension	API gateway	AI gateway
Policy it enforces	Traffic policy	Model and agent policy
What it fronts	System, process and experience APIs	Models, agents and tools
Controls it carries	Authentication, throttling, observability, versioning, policy enforcement	Model routing, guardrails, agent identity, action policy, audit
Rate of change	Slow, because business capabilities are stable	Weekly, because models and guardrails move quickly
Question it answers	Who may call this API, and how often?	May this agent take this action now?

The API Gateway Layer: API-Led Connectivity and Policy Enforcement

Under API-led connectivity, the API gateway fronts system, process and experience APIs with authentication, throttling, observability, versioning and policy enforcement, so AI consumes stable business capabilities and never raw databases.

The three tiers divide the work. System APIs expose systems of record, process APIs compose them into business operations, and experience APIs shape the result for one consumer; a study of API-led integration analyses how this three-tier architecture affects system performance and business outcomes System APIs (API-led integration study, 2025). An agent belongs on the process or experience tier, where a capability such as checking order status already carries business rules, and away from the system tier, where a call can reach the table directly.

Enforcement at the ingress can be strict without being slow. A 2025 study of multi-tenant full-stack systems places Envoy Gateway and the Open Policy Agent policy engine at the API ingress, and combines OAuth 2.1 with JWT authentication to give each tenant a separate identity under zero-trust access Open Policy Agent (multi-tenant gateway study). The pattern transfers to agents: each agent, like each tenant, presents an identity, and the policy engine answers whether that identity may call that operation now.

Drift between gateways is the failure to watch. A 2025 arXiv paper on multi-cluster gateway architectures reports that managing security, governance and performance as loosely coupled concerns produces configuration drift (Secure and Governed API Gateway Architectures). A policy updated on one cluster and forgotten on another lets an AI request pass through whichever door is weakest.

The AI Gateway as an Architectural Seam for Agent Identity and Guardrails

The AI gateway is an architectural seam that carries model routing, guardrails, agent identity, action policy and audit in one place, so AI concerns can change weekly while backend systems stay stable.

Agent identity is the first control to get right. An agent needs an identity of its own, distinct from the person it acts for, so the audit trail can show which agent acted, on whose behalf, and under which policy. Action policy sits beside it and sorts operations by consequence: reads pass, writes need a narrower grant, and high-impact actions wait for a human decision. Guardrails screen what goes in and what comes out.

Joe Price and Branimir Đurek’s Evolutionary Architecture Pattern

Joe Price and Branimir Đurek present the AI gateway on InfoQ as an evolutionary architecture pattern: a seam that isolates fast-changing AI concerns from the backend systems behind it. Models, prompts, guardrails and routing rules sit on the AI side of the seam and may change every week, while the enterprise systems on the other side keep their release cadence.

The seam matters because the two sides age at different speeds. Swapping a model provider or tightening an action policy becomes a gateway change with its own test and rollback, and an ERP release stays untouched. Without the seam, the audit trail records that “the AI” changed a record, and nobody can say which model version or which policy allowed it.

Semantic Routing Across Multiple Language Model Backends

Semantic routing sends each request to the most appropriate model by the meaning of the request, where static endpoint configuration and simple cost or latency rules ignore how hard the request is.

As backends integrate several language models with different capabilities and costs, routing becomes a design problem. A static route sends a simple lookup to the most expensive model and a multi-step reasoning request to the cheapest one, because the router never reads the request. The CISAT 2026 paper on Semantic-Aware Intelligent Request Routing in Multi-Model LLM Backend Gateways proposes APIRouter, which embeds a lightweight semantic-aware routing mechanism inside the gateway Multi-Model LLM Backend Gateways (APIRouter).

Model routing inside the gateway also keeps the choice reversible. Because applications call the gateway and never a model endpoint, a new model can take a share of traffic for one class of request while the rest stays put. The consequence is that the router becomes a policy component: a router that decides by meaning needs its own logging and review, since a misrouted request looks like an ordinary answer from an ordinary model.

MCP Gateways and Federated Tool Registries

An MCP gateway puts Model Context Protocol traffic behind one entry point and routes tool calls through federated registries, so agents find approved tools without each team publishing an uncontrolled server.

Thiago Mendes of TM Dev Lab describes a production Model Context Protocol gateway architecture built on federated registries for tool routing. Each domain team registers its tools in a registry, the registries federate, and the gateway resolves which tool serves a request. Without that arrangement, tool servers multiply, and each one carries its own credentials.

Tool access needs its own controls because MCP, introduced by Anthropic as a standard way for AI systems to interact with external data sources and tools in real time, brings new security challenges that a general API gateway was never designed to inspect (Enterprise-Grade Security for MCP, 2025). The gateway can apply OAuth 2.1 scopes per tool, so an agent that may read a catalogue cannot invoke the tool that edits it.

The market is moving the same way. Gartner’s API management analysts, Shameen Pillai, Steve Shwent, John Santoro and Nicholas Carter among them, track API gateways growing toward AI gateway capabilities and agentic traffic. A team that builds the MCP gateway now is building toward the product category the analysts already describe.

Middleware, ESB and iPaaS as the AI Control Plane: Deterministic Execution Behind Probabilistic Reasoning

Middleware is the execution tier of the AI integration layer: the agent requests a governed action, and deterministic middleware carries it out, so business-process orchestration never moves into probabilistic reasoning.

The rule sounds obvious until a team lets a model choose the order of steps in an order-to-cash flow, and a half-finished sequence appears with no code responsible for finishing it. The middleware owns the consequences; the model owns the suggestion.

Separating Business-Process Orchestration from Probabilistic Reasoning

Business-process orchestration stays in deterministic middleware because probabilistic reasoning can propose a step but cannot promise that the same input produces the same sequence of actions tomorrow.

An Enterprise Service Bus and integration-hub patterns still earn their place in legacy estates. They centralise protocol mediation, routing, transformation, orchestration and error handling, and none of that work disappears when an agent arrives. AI adds one rule to the existing job: the model decides what to ask for, and the middleware decides how the request executes, in what order, and what happens when a step fails.

Massimo Pezzini of Gartner has written on iPaaS, ESB, middleware and API management in AI-enabled enterprises, and the long view he takes supports keeping these tiers rather than replacing them. A model that improvises a refund workflow will eventually skip the fraud check on a day the prompt phrasing shifts. A workflow defined in middleware runs the fraud check every time.

Large language models also need a tier of their own. The 2024 arXiv paper Towards a Middleware for Large Language Models argues that as enterprises integrate LLMs into their services, a dedicated middleware layer lets them do so independently of commercial cloud-based offerings Large Language Models (Towards a Middleware for LLMs). The tier hosts the models the enterprise runs itself, on infrastructure the enterprise picks. That choice fixes where prompts and records are processed, so data residency rules can be met without sending regulated data to a commercial cloud endpoint.

iPaaS as the AI Control Plane: Approved Tools, Policy and Compensating Actions

An iPaaS becomes the AI control plane by exposing approved tools, applying identity and policy, converting data formats, and running retries and compensating actions, so every approved request executes the same governed way each time.

Approved tools come first. The iPaaS publishes a short list of operations an agent may request, each bound to a policy, and anything off the list does not exist from the model’s point of view. Format conversion follows, because the model speaks JSON while the ERP speaks something older. Retries handle the transient failures that every long chain of calls produces.

Compensating actions handle the failures that retries cannot fix. A compensating action undoes an earlier completed step, because distributed systems cannot roll back a shipment the way a database rolls back a row. When an agent requests an order cancellation, the control plane cancels the shipment, refunds the payment, and if the refund fails, runs the compensation that reinstates the shipment record for a person to resolve.

The platform market that hosts this work connects cloud, on-premises and legacy environments. Gartner’s iPaaS analysts Andrew Humphreys, Andrew Comes, Keith Guttridge and Allan Wilkins cover it as one market, which suggests that an enterprise already running an iPaaS holds most of the control plane it needs. The remaining work is deciding which operations the agent may request.

Event-Driven Integration for Timely AI Context

Event-driven architecture serves AI integration for two jobs only: it delivers fresh business context to an agent as changes occur, and it leaves an auditable record of those changes.

The choice between events and synchronous calls follows the shape of the question. A synchronous call suits a question whose answer the user waits for, such as the status of one order right now. An event suits a change that other parties care about, such as an order moving from picked to shipped, and the agent hears about it when it matters rather than polling for it.

The auditable record is the second job. Events arrive in order with timestamps, so the sequence of business changes around an agent’s decision can be replayed afterwards: what changed, in which order, and what the agent could have known at the time. Messaging infrastructure already provides this record for human-driven processes, and AI inherits it at no extra cost.

Events never decide and never execute. An event that tells an agent an invoice has been approved does not authorise the agent to pay it. The action still travels through the control plane, and the consequence of blurring the two is an agent that treats every notification as a permission.

Carrying MCP, A2A and AG-UI Traffic Through the Control Plane

The control plane must carry three agent protocols, split as Amazon Bedrock AgentCore splits them: Model Context Protocol connects agents to tools, Agent2Agent connects agents to other agents, and AG-UI connects agents to users Model Context Protocol (AWS Machine Learning Blog).

Each protocol opens a different door. AG-UI is an open protocol for generative UI, shared state and human-in-the-loop interactions, including an agent pausing mid-execution to ask for approval, and it works with frameworks such as Strands Agents, LangGraph and CrewAI. When an agent container is deployed with the AG-UI flag, AgentCore acts as a transparent proxy that handles authentication with Signature Version 4 or OAuth 2.0 through Amazon Cognito, session isolation and scaling.

Agent2Agent matured quickly. Google Cloud’s A2A version 0.3 release added gRPC support and the ability to sign security cards, and reports support from over 150 organizations, with Tyson Foods and Gordon Food Service building collaborative A2A systems for the food supply chain Google Cloud (Google Cloud Blog).

Three protocols mean three identity questions and three policy surfaces. A control plane that authenticates tool calls carefully and lets agent-to-agent messages through unchecked has a door open, and the enterprise learns which door only after the first incident.

Connecting AI to Legacy Systems Without a Rewrite: Façades, Anti-Corruption Layers and the Strangler Fig

The AI integration layer is where legacy systems meet AI without a rewrite, and it works as a fixed sequence: wrap the system in an API façade, translate its semantics, protect it from agent traffic, and replace its implementation later.

Teams under deadline skip steps, and every skip has a price. Wrapping without translating hands a model contradictory meanings. Translating without protecting lets a well-formed request flood a system built for a few hundred human users. Replacing before wrapping breaks every consumer at once.

API wrapping is the fastest path, and it carries a cost: the enterprise stays tied to the old system’s performance limits, data quality issues and release schedule. The choice is therefore the simplest pattern that still gives control over data contracts, security policy, retries and audit trails for AI-initiated reads and writes.

Wrap First: The API Façade as a Safety Boundary

An API façade over a mainframe, ERP, WMS, PeopleSoft or Siebel function is the safety boundary: AI talks to the façade, which exposes only a whitelist of operations, validates every parameter against a declared schema, enforces entitlements and redacts fields the caller may not see.

A mainframe transaction becomes a JSON operation with a declared input schema. A PeopleSoft or Siebel function becomes a named capability such as reading a leave balance, with the update path left out until someone owns it. The façade decides which operations exist for AI at all, so a model never discovers a delete function by guessing.

Contract-first integration comes before the first call. OpenAPI describes synchronous operations, AsyncAPI describes event channels, and JSON Schema validates payloads for both. A technical document on RAML and OpenAPI covers standardisation, documentation generation, client code generation and contract-first design RAML and OpenAPI (API Design and Implementation: RAML and OpenAPI Specification). For AI, tool contracts also declare allowable actions, confidence or uncertainty fields, freshness and provenance, and Kristen Womack’s guidance on API metadata and OpenAPI semantics for agentic automation supplies the reasoning: an agent can act only on what the contract tells it.

The consequence of a thin façade is visible in the first incident. An agent with a broad wrapper and no entitlement check can read fields that a person in the same role would never see, and the leak is a design fault that no prompt can repair.

Normalise Semantics with an Anti-Corruption Layer

An anti-corruption layer translates the legacy data model into an AI-facing model and normalises terminology, units, identifiers and business rules before data reaches the model, because models are sensitive to inconsistent semantics.

A person reading two screens resolves a conflict without noticing: status code 3 means shipped in one system and held in another, weight arrives in pounds from the warehouse and kilograms from the supplier portal, and a customer carries three identifiers across three applications. A model has no colleague to ask. It treats the contradiction as data and produces a fluent answer built on it.

The layer keeps legacy assumptions out of the AI-facing model. Martin Fowler’s writing on legacy-system modernization and evolutionary architecture is the design stance here: the new model stays clean, and the translation code absorbs the mess in one place where it can be tested.

Regulated estates show why the translation matters. A 2025 framework for the architectural transformation of legacy financial systems combines microservices, cloud migration and API-first integration to address outdated infrastructure, regulatory compliance and operational efficiency (Architectural Transformation of Legacy Financial Systems). In such an estate, a unit or identifier error reaches an auditor as well as a model.

Read Models and Circuit Breakers to Protect Production Systems

AI reads should come from CQRS read models fed by event sourcing, while bulkheads, backpressure, rate limits and circuit breakers stand in front of every legacy service an agent can call.

CQRS, command query responsibility segregation, separates the model that answers questions from the model that accepts changes. Event sourcing records each change as an ordered event, and a read model is rebuilt from those events. AI queries hit the replicated read model, and commands keep flowing through validated transactional paths, which prevents expensive or unsafe model-driven reads against production systems.

Agents generate traffic that people rarely do: repeated calls, parallel calls and poorly bounded calls. Rate limits cap how often an agent may call. Backpressure tells the caller to slow down when a service falls behind. Bulkheads give each consumer its own pool of capacity, so one agent’s flood cannot drain the connections that order entry needs. A circuit breaker stops calls to a failing service for a period, which gives it room to recover rather than being hammered by retries.

The consequence of skipping these controls is quiet. The first sign is rarely an outage; it is a batch job that runs slowly at month end because an agent has been re-reading the same ledger tables all afternoon.

Replace Later: The Strangler Fig Behind a Stable Contract

The Strangler Fig pattern lets a team swap the implementation behind the contract later: expose a capability safely first, improve its data quality, then replace the underlying implementation while AI keeps calling the same contract.

The order matters. The façade and the anti-corruption layer come first, because they hold the contract steady. Behind that contract, one capability at a time moves from the old system to a new service, and the AI consumer never learns it happened. AI integration becomes a modernization forcing function: the first agent use case pays for the contract, and every later replacement rides on it.

David R. Longnecker’s API-first AI integration patterns serve as the implementation catalogue: service wrappers, context gateways, response transformation, provider abstraction, versioning, observability and failure handling. Provider abstraction is the pattern that matters most at this step, since it lets the enterprise swap the legacy system for its replacement behind the contract while every consumer keeps the same calls.

Each replaced capability removes a dependency on the old system’s release schedule. The teams that stall are those that treat the wrapper as the destination, and they discover years later that the façade has become the permanent mask over a system nobody dares touch.

Enterprise AI Integration Challenges Traced to the Layer Where They Start

Enterprise AI integration challenges each start in one layer, and naming that layer turns a vague stall after go-live into a diagnosis: undeclared consumers begin at the edge, data dependencies in translation, and hidden feedback loops in execution.

Most post-mortems name the model, since the model is the visible part. The failure usually started earlier in the chain, and adoption counts hide it because a deployment that exists looks the same as a deployment that works.

Sculley’s Machine Learning Debt Risks, Located by Layer

D. Sculley and colleagues found that real-world ML systems commonly incur massive ongoing maintenance costs, and named the risk factors: boundary erosion, entanglement, hidden feedback loops, undeclared consumers, data dependencies, configuration issues and changes in the external world (Hidden Technical Debt in Machine Learning Systems).

Their taxonomy was written about ML systems generally. Placed against the layers of an AI integration stack, four of the risks receive an address.

Risk	Layer where it starts	Address
Undeclared consumers	Edge	Gateway layer
Data dependencies	Translation	Anti-corruption layer
Entanglement	Translation	Anti-corruption layer
Hidden feedback loops	Execution	Middleware control plane

Hidden feedback loops deserve a closer look, because the execution layer is where agent actions write back into data the model later reads. The model then studies the consequences of its own earlier decisions and mistakes them for independent evidence. Boundary erosion appears wherever a contract between layers is missing, and configuration issues share an address with the configuration drift at the gateway.

Debt at the integration layer shrinks when each risk has one owner at its address. A risk with no address is a risk every team assumes another team has handled.

The 70.75% Technology Integration Gap and What It Measures

Mariya Sira’s gap analysis framework puts the technology integration gap at 70.75% and the strategic alignment gap at 80%, against 78% of organisations adopting AI and only 1% reaching mature implementation Mariya Sira (Sira, 2025).

The framework, published in Production Engineering Archives in 2025, assesses implementation across technology adoption, human capital readiness, financial adequacy and strategic alignment. It reports gaps in every dimension: 57.31% for financial optimization and 16.67% for human capital development besides the two above. It also finds that fewer than 20% of organisations track AI performance indicators, which means most enterprises cannot see the gap they are in.

The technology integration gap measures the distance between AI that has been deployed and AI that operates inside enterprise systems. A gap that size means most deployments sit beside the estate instead of inside it, a condition an adoption survey would report as success.

Mariya Sira’s Integration Depth Coefficients

Sira’s integration depth coefficients differentiate reported adoption from functional operational effectiveness, which addresses the limits of binary assessment approaches that only ask whether an organisation has adopted AI.

The coefficients let a deployment count for what it does operationally. A chatbot with read-only access to one knowledge base and an agent executing governed writes across three systems are both “adopted” in a binary survey, and they sit at very different depths.

Applied to a single estate, the idea is practical: list each AI deployment, note which layers its requests traverse, and weight the deployment by the depth it reaches. The list shows where adoption is shallow, and it shows it before the next budget cycle rewards the wrong count.

Integration Uptime as the Per-Layer Operating Metric

Integration uptime, the availability of AI integrations with enterprise systems, should be reported per layer rather than per model, because a model can be healthy while the seam that serves it is down.

A per-model figure hides where the loss happens. Measured at the gateway, uptime shows whether requests are admitted; measured at the middleware, whether actions complete; measured at each façade, whether the legacy system answered within its contract. The share of time each layer met its contract, reported on one page, points at the layer that deserves the next hour of engineering.

The metric answers the finding that fewer than 20% of organisations track AI performance indicators. Integration uptime is one indicator an integration team can start on this week, because the gateway and the middleware already log the events it needs.

The figure creates its own consequence: once uptime is reported per layer, a layer owner can no longer point at the model, and the conversation moves to the layer’s own contract.

Data Mesh and Data Products for Data Dependencies

Data mesh and data products answer data dependencies structurally: domain teams publish trusted data products with documented semantics and access policies, and the model connects to those products instead of to every system.

A model connected directly to twelve systems inherits twelve sets of definitions, twelve refresh schedules and twelve access rules. A data product collapses that into one interface per domain, with a named owner, a stated freshness and a policy that says who may read what. The number of dependencies the AI layer carries falls to the number of products.

The design moves the work to the domains. Each domain team must document what its fields mean and keep the product current, and that cost lands on people who did not ask for it. Data mesh fails where domains have no incentive to maintain the product, and the AI integration layer then reads stale data with full confidence.

Which Integration Path Fits Your Estate?

API-based integration fits systems that already expose stable interfaces, middleware platforms fit work that spans several systems and needs retries and compensation, and custom-built AI modules fit only capabilities that no platform supplies.

Path	Choose it when	What it costs later	Where its risk sits
API-based integration	The target has a stable, documented API and the AI use is narrow or read-heavy	Lowest ownership cost and the cheapest exit, because only the API contract binds the consumer	Gateway and façade
Middleware platform	One request touches several systems and needs retries or compensation	Ties delivery to the integration team and platform roadmap	Control plane
Custom-built AI module	The capability is unique and no platform covers it	Needs a standing on-call team, follows every model and framework upgrade, and is costly to exit once other systems depend on it	Every layer

Three questions sort most cases. Does an interface already exist that a person would trust with this operation? If it does, the first path is open. Does the action need more than one system to succeed together? If it does, the middleware path carries the retries. Who answers the page at 3 a.m. when the custom module misbehaves? If nobody can answer, the third path is not yet ready.

Mixed choices are normal, and the estate in most enterprises will use all three. What matters is that each path has a named owner before the first agent call reaches production.

Who Owns Each Layer of an AI Integration Stack?

Platform and security architects own the gateways, integration leads own the middleware control plane, delivery leads own the legacy façades, and a CIO or platform owner owns the per-layer metric that shows whether the whole stack works.

Ownership fails at the seams. Action policy sits between the security team that enforces it and the business owner who knows which operations carry risk, so each side assumes the other wrote it. A useful rule separates the two duties: the owner of each system of record signs off, and the gateway team enforces. If the two disagree, the owner’s refusal stands for any write, and the platform owner decides the dispute and records the decision in the audit trail.

The people who run each system already know which calls are dangerous. A payroll administrator can name the three updates that must never run unattended in an afternoon of conversation, and that knowledge is a better first draft of an action policy than any generic template. A layer model locates where policy is needed, and the people inside the system supply its content.

Where ownership is unclear, start with the layer that has the largest unexplained failure count and name one person for it, then adjust as the per-layer figures arrive.

What Is the Smallest Experiment That Locates Your Weakest Layer?

Pick one read-only AI use case, tag a single request with one trace ID, and log the same fields at every layer it crosses for two weeks: timestamp, layer, outcome, latency and error type.

At scale, the point is orientation before investment. A small trace shows where a request actually goes, which is often different from the architecture diagram.

Choose a use case that reads data and changes nothing, such as answering an order-status question.
List every layer the request crosses: gateway, control plane, façade, system of record.
Log success, latency and error type at each crossing.
Join the logs on the trace ID at the end of each week, so one request reads as one line per crossing.
Change that layer alone, then repeat the trace and compare.

Only after the read-only trace runs cleanly does a write action join the experiment, and then behind the action policy and the human approval step. Each cycle is small enough to reverse, and each one leaves a measured result that the next decision can use.

Summary

The layers of an AI integration stack divide one job among them: the model suggests, and the estate decides what happens next.

Reasoning Stays Probabilistic and Execution Stays Deterministic at Every Layer

The decision rule that runs through every layer is that the model proposes and the enterprise disposes. The gateway says no before a model acts, the control plane carries out approved requests with retries and compensation, and the façade protects the system of record from the volume and the ambiguity an agent produces. A model never holds credentials for a system and never decides the order of a business process.

The rule also sets the sequence of work. Locate the layer first, then choose the path. A team that picks a platform before it knows which layer fails will buy capability for the layer that already works. A single-request trace gives the location, the choice between the three paths follows from it, and the per-layer uptime figure shows whether the choice worked.

Ignoring the rule has a predictable result. Logic that belongs in middleware drifts into prompts, the model becomes impossible to replace, and each provider change turns into a rewrite of business processes. The cheapest moment to keep the two apart is the first integration, because every later one will copy it.

Every Failure Has an Address, and Every Address Needs an Owner

The distinction that separates good practice from poor practice is whether a failure can be pointed at. The taxonomy of machine learning debt gives each risk a layer, the gap measurement shows how far most estates are from mature operation, and the per-layer metric turns a stall after go-live into a named contract that one layer failed to meet.

Failure appears wherever an address has no owner. Undeclared consumers persist when nobody owns the edge, contradictory meanings persist when nobody owns translation, and stranded half-finished changes persist when nobody owns execution. The array of protocols and gateways adds doors faster than it adds owners, so each new door needs a named person before it opens.

The boundary condition for choosing an approach is the estate itself. Where systems expose stable interfaces, the light path is enough. Where they do not, the wrap-translate-protect-replace sequence gives a way to proceed without waiting for a rewrite. In both cases the reader chooses the next small step, measures it, and adjusts.

Anonymous. Counted, not tracked.

Where is your organisation with this right now?

What is the hardest part where you are?

agility at scale