Agentic AI infrastructure is the set of layers that let AI agents act safely on real systems: a model runtime, a tool layer, a data layer, an identity and policy layer, and observability across all of them. In a private MCP workflow, the tool layer is made of Model Context Protocol servers that run inside your own network, accept only tokens issued for them, and expose narrow tools instead of broad access. The model call is rarely the hard part. Deciding who or what may take which action, and proving afterwards what happened, is.
We make HyperLake, a sovereign AI operating platform. It comes up once, near the end, with a plain statement of where it applies and where it doesn't.

What is agentic AI infrastructure?
Start with what an agent is. In the sense Anthropic uses in its guidance on building agents, an agent is a system where the model chooses its own steps and tool use, as opposed to a workflow, where predefined code paths orchestrate the model and tools (Anthropic, Building effective agents). That's where the infrastructure problem comes from. You can review a fixed workflow in advance. An agent decides at run time, so the controls around it have to work at run time too.
The same Anthropic guidance says agentic systems often trade latency and cost for better task performance, points to the potential for compounding errors, and recommends extensive testing in sandboxed environments plus appropriate guardrails. Sandboxing and guardrails: that's what the rest of the stack is there to provide.
What are the layers of an agentic AI architecture?
The table is our working decomposition, not a standard. Real systems blur the lines, but each layer answers a different question.
| Layer | Question it answers | Typical contents |
|---|---|---|
| Model runtime | Which model reasons, and where does it run? | Hosted API or self-hosted inference |
| Orchestration | What runs next, and who coordinates? | Agent loop, queues, durable state |
| Tool layer | What can the agent do? | MCP servers, APIs, functions |
| Data layer | What can the agent know? | Governed tables, documents, vector and graph stores |
| Identity and policy | Who is acting, and what is allowed? | Identity provider, tokens, policy engine |
| Observability and audit | What happened, and what did it cost? | Traces, logs, spend, evaluations |
Two of these get deeper treatment elsewhere: what an agentic control plane is covers orchestration and governance, and data products for AI agents covers the data layer. For where the model runs, see on-premises AI.
How does MCP fit into the architecture?
MCP is the tool layer's protocol. According to the MCP architecture page, a host application creates and manages multiple clients, and each client talks to exactly one server (MCP architecture). Servers expose resources, tools and prompts. They can be local processes or remote services.
One of the protocol's stated design principles shapes the whole architecture. Servers shouldn't be able to read the whole conversation or "see into" other servers. The full conversation history stays with the host, and the host controls cross-server interactions. In practice, the host is your enforcement point. It decides which servers a given agent may connect to and what each server gets to see.
The MCP tools specification describes tools as model-controlled: the model can discover and invoke them based on context and the user's prompt (MCP tools). The same page says there should always be a human in the loop who can deny tool invocations, and that servers must validate all tool inputs, implement access controls, rate limit invocations and sanitize outputs. We'd treat that as the floor for every server you run.
What makes an MCP workflow private?
"Private" isn't a protocol feature. It's a handful of deployment decisions. Here's our checklist, with the parts the MCP specification itself requires called out.
- Servers run where you control the network. A remote MCP server can still sit behind your own gateway and never be reachable from the public internet.
- Authentication goes through your identity provider. The specification defines an MCP server as an OAuth 2.1 resource server, with a separate authorization server issuing tokens (MCP authorization), and that authorization server can be your own.
- Tokens are audience-bound. The specification requires servers to validate that access tokens were issued specifically for them, and requires clients to send an RFC 8707
resourceparameter naming the server they mean to use. - Egress is controlled. A private server that can call any external URL isn't private. The security best practices page suggests server-side MCP client deployments consider an egress proxy that blocks internal destinations during OAuth discovery, and the same idea applies to tool calls in general.
- The model is yours as well. If prompts and tool results go to a third-party model API, the workflow is only partly private. Self-hosted inference closes that gap, and on-premises AI walks through the trade-offs.
How should you set up MCP transports and authentication?
The MCP specification defines two standard transports, stdio and Streamable HTTP (MCP transports). Each implies a different security model.
| stdio | Streamable HTTP | |
|---|---|---|
| How it runs | Client launches the server as a subprocess | Server is an independent process serving multiple clients |
| Authentication | Credentials come from the environment, not the OAuth flow | OAuth-based authorization applies |
| Network exposure | No network endpoint by default (our reading) | Exposed on an endpoint |
| Fits | Single-user, local tools | Shared, remote, multi-user servers |
The authorization page says authorization is optional, that HTTP-based implementations should conform to it, and that stdio implementations shouldn't follow it and should pull credentials from the environment instead. The transports page adds that HTTP servers must validate the Origin header to prevent DNS rebinding, should bind to localhost when running locally, and should implement authentication for all connections.
One caution. The tools, authorization and transports pages linked above are specification version 2025-06-18, and the protocol keeps changing. The architecture page linked above already describes a newer, stateless version of the protocol. Check which version your clients and servers target before you copy any header or session behavior.
For the full authentication and authorization walk-through, scopes and audit logs included, see MCP authentication and authorization for AI agents.
What are the main risks in an MCP architecture?
The MCP security best practices page lists several attacks. Four matter most for platform design.
- Token passthrough. An MCP server takes a client's token and forwards it to a downstream API. The page forbids this and lists what goes wrong: bypassed rate limits and controls, muddled audit trails, broken trust boundaries. The server should get its own token for the downstream API.
- Confused deputy. An MCP proxy that uses one static OAuth client ID with a third-party provider can be tricked into skipping user consent. The fix is per-client consent stored server-side.
- Local server compromise. A local server runs with the client's privileges. The page recommends sandboxing, minimal default privileges and showing users the exact command before it runs.
- Over-broad scopes. The page's scope minimization section says tokens with broad scopes make a leak more damaging. It recommends a minimal initial scope with incremental elevation, and names wildcard scopes as a common mistake.
OWASP calls the underlying agent problem "excessive agency": damaging actions taken because of unexpected, ambiguous or manipulated model output. Its root causes are excessive functionality, excessive permissions and excessive autonomy (OWASP LLM06:2025). The preventive controls include offering only the tools the agent needs, avoiding open-ended tools such as shell access, executing in the authenticated user's context, requiring approval for high-impact actions, and enforcing authorization in downstream systems instead of trusting the model's judgment. OWASP adds that logging, monitoring and rate limiting don't prevent excessive agency, though they can limit the damage.
How do you scope what an AI agent can do?
Keep authorization outside the model. The model proposes a tool call, and a policy layer decides. Scope by task, not by standing role, and the permission set stays small. Task-based access control for AI agents goes into detail.
The sketch below is a hypothetical, generic task grant, there only to show the shape. It isn't a real product schema or any standard.
{
"task_id": "refund-review-0421",
"agent": "support-agent",
"on_behalf_of": "user:alice",
"allow": [
{ "server": "orders-mcp", "tool": "get_order", "limits": { "calls": 20 } },
{ "server": "orders-mcp", "tool": "propose_refund", "requires_approval": true }
],
"deny_by_default": true,
"expires_in_minutes": 30
}The format matters less than three properties. Access is denied unless allowed. The agent acts on behalf of a named user, so downstream systems see that user's context. And the one consequential action, the refund proposal, waits for a person. That follows the human-in-the-loop guidance in the MCP tools specification and the approval control on the OWASP list.
What does the data layer need?
If an agent can call tools, it can read whatever those tools return. So give agents data through governed interfaces with defined schemas, ownership and access rules, not shared database credentials. Data products for AI agents describes the pattern. Retrieval stores, vector or graph, live in this layer too, and vector vs graph databases vs knowledge graphs covers how to pick.
When is this much architecture unnecessary?
Often. One developer running one agent against local files over stdio doesn't need a control plane, an authorization server or a policy engine. The risks are still there, a malicious server startup command especially, so keep the local sandboxing advice. The full stack earns its keep when several agents, teams or customer environments share tools and data, and when someone has to answer an auditor asking who did what.
Where does HyperLake fit?
HyperLake is a platform for assembling, deploying and governing the data, models, applications and tools behind AI in a customer's own environment or a client's. It isn't a protocol or a model, and this article doesn't claim it implements any specific MCP specification feature.
According to HyperLake's product description, its agentic cloud use case combines model serving, data engines, agent execution, secrets, observability and lifecycle management, with OAuth/OIDC sign-in, JWT identity, network boundaries and OPA policy checks. Its control surface is described as callable through an application, a CLI and MCP-compatible tools. It runs in your own cloud account on AWS and Google Cloud today, with private cloud, on-premises and other environments coming soon. It fits when you want those layers assembled and operated in infrastructure you control. You don't need it for a single local agent.
Frequently asked questions
What is agentic AI infrastructure?
It's the runtime, tool, data, identity, policy and observability layers that together let AI agents take actions on real systems under control. The model is one component. Most of the engineering lives in the layers around it.
What is a private MCP server?
Here, it means an MCP server deployed inside a network you control, authenticating users through your own identity provider and reachable only by approved clients. MCP doesn't define "private". You get there through deployment and network decisions plus the specification's authorization requirements.
Does MCP require OAuth?
Not always. The specification calls authorization optional, and when an HTTP-based server supports it, implementations should follow its OAuth-based flow. For stdio, it says to retrieve credentials from the environment instead.
What is the difference between an MCP server and an API?
An API exposes endpoints for developers to call. An MCP server describes tools with names, descriptions and input schemas so a model can discover and call them. Plenty of MCP servers wrap existing APIs, which is exactly why token handling between the two matters.
Who enforces permissions: the model or the infrastructure?
The infrastructure does. OWASP's guidance is to enforce authorization in downstream systems instead of relying on the model's judgment, since model output can be wrong or manipulated.
Do agents need a human in the loop?
For consequential actions, we think so, and it's the safer design. The MCP tools specification says there should always be a human in the loop who can deny tool invocations, and OWASP lists human approval for high-impact actions as a control.



