Introducing TrueFoundry MCP Gateway: Solving the N×M Integration Problem for AI Agents

If you're building AI agents that need to interact with external tools and APIs, you've likely hit the same wall we did: the N×M integration problem. As the number of agents (N) and tools (M) increases, each agent ends up implementing its own connection, authentication, and error handling for each tool. This creates an N×M matrix of point-to-point integrations that becomes difficult to manage, secure, and observe.Every agent connecting directly to every tool creates a tangled web of point-to-point connections.

Today, we're excited to announce the TrueFoundry MCP Gateway - an enterprise-ready platform that centralizes access to AI development tools using the Model Context Protocol (MCP). Instead of managing hundreds of individual tool configurations across your development teams, you can provide secure, governed access to curated AI tools through a single platform.

The Problem: Direct Agent → Tool Connections Don’t Scale

When agents connect directly to tools, every agent becomes its own miniature integration hub. This works fine for one agent talking to one tool, but it quickly falls apart as your application grows.

Credential Sprawl and Security Risks

In a direct-connect model, each agent stores and manages credentials for every tool it accesses. This creates a massive attack surface. You've got API keys, OAuth tokens, and database connection strings scattered across multiple agent codebases, configuration files, and environment variables. This credential sprawl makes secure rotation nearly impossible and dramatically increases your risk of a leak. If one agent gets compromised, it could expose credentials for dozens of critical internal and external services.

Observability Black Holes

Without a central point of traffic control, you can't get a unified view of agent-tool interactions. Is a specific tool slow? Is an agent making too many calls? Which user action triggered a cascade of five different tool calls? To answer these questions, you'd need to stitch together logs from N different agents, which is often impractical. You end up with an observability black hole where debugging is reactive and performance tuning is based on guesswork, not data.

Inconsistent Error Handling & Retries

Each external tool has its own failure modes, rate limits, and transient error conditions. In a decentralized model, every single agent developer has to implement robust error handling like exponential backoff, retries for idempotent operations, and circuit breakers. This results in inconsistent and often incomplete implementations. One agent might retry a failed call aggressively, accidentally launching a denial-of-service attack on a struggling tool, while another might fail silently and derail a critical business process.

High Maintenance Overhead

Every new tool integration becomes a significant development effort, repeated across multiple agents. Developers waste time writing boilerplate code for authentication, request signing, and response parsing instead of focusing on core agent logic. When a tool's API changes, you have to identify and update every agent that uses it individually. This high maintenance overhead slows down development velocity and makes it difficult to expand an agent's capabilities.

The Solution: MCP Gateway

MCP is an open protocol that standardizes how applications provide context to LLMs. Think of MCP like a USB-C port for AI applications—it provides a standardized way to connect AI models to different data sources and tools.

MCP Servers are programs that expose data and capabilities to LLMs via the MCP protocol. For example:

• A Slack MCP Server might expose tools like "Send a message to a channel" or "Search for messages"
• A GitHub MCP Server might expose tools like "Get the list of repositories" or "Create a pull request"

An MCP Gateway is a specialized reverse proxy that sits between your AI agents (the clients) and your tools (the MCP servers). Instead of agents connecting directly to dozens of different tool endpoints, they all connect to a single, unified gateway endpoint. The gateway then securely routes requests to the appropriate upstream tools. This architecture solves critical problems for developers building agentic systems:

Centralized Security & Governance: The gateway becomes a single chokepoint for all agent-tool interactions. You can enforce authentication, authorization (like role-based access control), and create detailed audit logs in one place.

Unified Observability: An MCP Gateway centralizes logging, metrics, and tracing. You can monitor latency, track error rates, and trace a complex agent task across multiple tool calls from a single dashboard.

Operational Efficiency: A gateway simplifies tool management. It maintains a central registry of available tools, so agents can discover them dynamically. It manages credentials for all upstream tools, securely injecting them into requests as needed.

Cost Management: AI agents can be expensive, making many calls to LLMs and paid tool APIs. A gateway gives you the control to manage these costs through caching, rate limiting, and budget controls.

TrueFoundry MCP Gateway: Architecture and Features

The TrueFoundry AI Gateway is an enterprise-ready platform that centralizes access to AI development tools using MCP. It provides a MCP registry, centralized authentication, and a built-in MCP client that orchestrates the agentic loop between the LLM and the MCP servers.

Architecture Overview

Agents authenticate once to the MCP Gateway, which routes their MCP requests to registered MCP servers (Slack, GitHub, internal tools), with the Control Plane managing tokens, OAuth flows, and access policies.

Key Features of MCP Gateway

1. Centralized MCP Registry

You can add public as well as your self-hosted MCP servers which are registered in the TrueFoundry Control Plane. The Control Plane maintains the centralized registry of all the MCP servers and their authentication mechanisms. It handles user-specific OAuth2 flows, securely storing and refreshing access tokens and ensuring users can only access resources they are authorized for.

This solves the credential sprawl problem: instead of every developer managing their own API keys and OAuth tokens for each tool, the gateway manages them centrally. Users authenticate once with the gateway, and the gateway handles all downstream authentication.

2. Fine-Grained Access Control

While registering an MCP server, you can specify the list of users/teams that have access to it. This allows fine-grained access control at an enterprise level. This is done via MCP Server Groups, wherein you can define managers who can manage and give others access to the MCP servers.

For example, you might have:

• A "Engineering" MCP Server Group with access to GitHub, JIRA, and internal CI/CD tools
• A "Sales" MCP Server Group with access to Salesforce, HubSpot, and email tools
• A "Finance" MCP Server Group with access to accounting and payment processing tools

Each group only sees the tools they're authorized to use, reducing the attack surface and preventing accidental misuse.

3. Unified Authentication

Any user can generate a single Personal Access Token (PAT) using which they can access all the models and MCP servers that they have access to. You can also generate a Virtual Account Token (VAT) to provide access to a specific set of MCP servers to an application.

The gateway supports multiple authentication methods:

• TrueFoundry API Keys: For users with TrueFoundry accounts
• IDP Tokens: For integration with your existing identity provider (Okta, Azure AD, etc). *This allows your end customer tokens to also be validated by the Truefoundry MCP gateway.

The gateway handles the complexity of OAuth2 flows, including:

• Initiating OAuth2 authorization flows
• Storing and securely managing access tokens
• Automatically refreshing expired tokens
• Mapping user tokens to OAuth tokens for different MCP servers

4. Virtual MCP Servers

One of the most powerful features is the ability to create Virtual MCP Servers. These allow you to combine tools from multiple MCP servers into a single, curated MCP server that your application can connect to.

For example, suppose you have integrated MCP servers for GitHub and Slack. A team in your company is working on an Agent that requires access to these two MCP servers, but you don't want to expose dangerous tools like `delete_project`, `delete_pr`, etc.

A Virtual MCP Server allows you to create a new MCP server by taking a subset of safe tools from GitHub and Slack MCP servers. This new Virtual MCP Server can be accessed like any other remote MCP server and does not require a deployment. It's managed entirely by the gateway.

This is particularly useful for:

• Creating safe, curated tool sets for different teams
• Combining tools from multiple sources into logical groupings
• Implementing least-privilege access by exposing only necessary tools

5. Agent Playground

TrueFoundry AI Gateway provides a playground where users can experiment with prompts and different tools of MCP servers to build agents. The gateway comes with commonly used tools like Websearch, WebScraping, document extraction, and code execution.

The gateway comprises an MCP client that orchestrates executing the tools decided by the LLM providers. The Gateway also streams the progress of the request back to the UI so that the user can see the LLM responses, tool calls, and the tool responses in real-time.

This makes it easy for developers to:

• Test different tool combinations
• Debug agent behavior
• Understand how tools are being called
• Iterate quickly on agent prompts

6. Use MCP Servers in Code

The Gateway provides code snippets showing how you can start using the MCP servers in your code. You can use the MCP Gateway API directly or integrate with popular MCP client libraries.

Here's an example of using the gateway from Python:

import asyncio
from fastmcp import Client
from fastmcp.client.transports import StreamableHttpTransport

async def main():
    # Connect to the gateway using your Personal Access Token
    transport = StreamableHttpTransport(
        url="https://{controlPlaneURL}/mcp/{groupName}/{mcpServerName}/server",
        auth="Bearer your-tfy-token"
    )
    
    async with Client(transport=transport) as client:
        # List available tools
        tools = await client.list_tools()
        print(f"Available tools: {tools}")
        
        # Call a tool
        result = await client.call_tool(
            name="github_list_repositories",
            arguments={"org": "my-org"}
        )
        print(f"Result: {result}")

asyncio.run(main())

6. Four-Layer Authentication & Authorization

The gateway implements a comprehensive four-layer authentication and authorization system:

Layer 1: Gateway Authentication

Any user/application requires a token to talk to the gateway - either a TrueFoundry API key or your own IDP token. TrueFoundry AI Gateway can verify your own IDP token and extract the user's email from the token based on the SSO settings.

This means you can integrate with your existing identity provider (Okta, Azure AD, Google Workspace, etc.) without requiring users to create separate TrueFoundry accounts.

Layer 2: Gateway Access Control

You can define at the gateway layer which users have access to which MCP servers. This allows fine-grained access control at an enterprise level. This is done via MCP Server Groups, wherein you can define managers who can manage and give others access to the MCP servers.

Layer 3: External Service Authorization (MCP Server Auth)

This is the authorization implemented by the MCP Server for accessing the external service. TrueFoundry allows MCP servers to be integrated with the following auth mechanisms:

• No Auth: For demo APIs or public APIs (not recommended for production)
• Static Header-Based Auth: For MCP servers that use API keys or static tokens (e.g., Hugging Face)
• OAuth2 and DCR Based Auth: For MCP servers that support OAuth2 (GitHub, Slack, Atlassian, etc.)

The gateway handles the complexity of OAuth2 flows:

• It stores and manages OAuth tokens for different MCP servers for each user
• It keeps a map of user tokens to OAuth tokens for different MCP servers
• It refreshes tokens automatically when they expire
• This allows users to talk to the Gateway with a single token without having to manage multiple tokens

Layer 4: Custom Headers

You can pass any custom headers to MCP servers using the `x-tfy-mcp-headers` header. This is useful for authentication tokens, metadata, or any headers your MCP server needs. Custom headers always override the default authentication configured for the MCP server.

Real-World Use Cases

Use Case 1: Enterprise Development Teams

A large engineering organization wants to give all developers access to GitHub, JIRA, and Slack through AI agents, but with different permission levels:

• Junior Developers: Read-only access to GitHub (can list repos, view PRs, but can't merge)
• Senior Developers: Full access to GitHub, read-only access to JIRA
• Engineering Managers: Full access to all tools

With TrueFoundry MCP Gateway:

1. IT creates MCP Server Groups for each role
2. Each group is configured with appropriate OAuth scopes
3. Developers authenticate once with the gateway using their corporate IDP
4. The gateway handles all OAuth flows and token management
5. Developers use a single Personal Access Token to access all authorized tools

Use Case 2: Multi-Tenant SaaS Platform

A SaaS platform wants to allow their customers to build AI agents that interact with the customer's own tools (e.g., their GitHub, their Slack):

1. Each customer authenticates with their own IDP token
2. The gateway maps customer tokens to their OAuth tokens for their tools
3. Customers can only access their own resources (enforced by OAuth scopes)
4. The platform has full observability into tool usage for billing and support

Use Case 3: Secure Internal Tools

An organization wants to expose internal tools (databases, APIs) to AI agents, but with strict security requirements:

1. Internal MCP servers are registered with header-based authentication
2. Access is restricted to specific teams via MCP Server Groups
3. All tool calls are logged and audited
4. Virtual MCP Servers are used to expose only safe, read-only tools to agents

Getting Started

Getting started with TrueFoundry MCP Gateway is straightforward:

1. Create an MCP Server Group: Organize your MCP servers into logical groups
2. Add MCP Servers: Register public or self-hosted MCP servers with appropriate authentication
3. Configure Access Control: Define which users/teams can access which servers
4. Generate Access Tokens: Users generate Personal Access Tokens to connect to the gateway
5. Start Building: Use the playground to experiment or integrate directly in your code

The gateway supports both user authentication (via OAuth2 flows) and machine-to-machine authentication (via Client Credentials grant), making it suitable for both interactive agents and automated workflows.

Tutorial: Building an OAuth-Protected MCP Server from Scratch and integrating with Truefoundry AI Gateway

Let's walk through building a complete MCP server with OAuth2 authentication and integrating it with TrueFoundry MCP Gateway. We'll create a simple calculator MCP server that demonstrates both user authentication (via the Gateway) and machine-to-machine authentication.

Refer to this documentation for the full tutorial - https://docs.truefoundry.com/docs/ai-gateway/mcp-server-oauth-okta

The complete code for this tutorial is available in our [GitHub repository]

This tutorial demonstrates:

1. Building MCP Servers: Using FastMCP to create tools that LLMs can call
2. OAuth2 Integration: Securing MCP servers with OAuth2 using Okta
3. Machine-to-Machine Auth: Enabling programmatic access without user interaction
4. Gateway Integration: Centralizing MCP server access through TrueFoundry MCP Gateway
5. User Authentication: Allowing end users to authenticate via OAuth flows managed by the gateway

The gateway handles the complexity of OAuth flows, token management, and refresh, allowing users to access all their authorized MCP servers with a single Personal Access Token.

Why This Matters

As AI agents move from prototypes to production, the integration architecture becomes critical. The N×M integration problem isn't just a theoretical concern—it's a real barrier to building reliable, secure, and scalable agentic systems.

An MCP Gateway isn't a niche component anymore. It's critical infrastructure for any team building production-grade AI agents. It provides the essential control plane for managing the security, observability, and operational complexity of agent-tool interactions at scale.

By centralizing tool access through a gateway, you're not just solving today's problems—you're laying a scalable foundation for the future. As your agent ecosystem grows, the gateway grows with it, providing consistent security, observability, and operational patterns across all your agent-tool interactions.

The TrueFoundry MCP Gateway solves the fundamental integration challenges that every team building AI agents faces. By providing a centralized, secure, and observable platform for agent-tool interactions, it enables teams to:

• Move faster: Developers focus on agent logic, not integration boilerplate
• Stay secure: Centralized credential management and fine-grained access control
• Maintain visibility: Unified observability across all agent-tool interactions
• Scale confidently: Architecture that grows with your needs

If you're building AI agents and struggling with the N×M integration problem, we'd love to hear from you. The TrueFoundry MCP Gateway is available now, and we're excited to see what you build with it.

For more information, check out our documentation or reach out to discuss your specific use case.

What are your thoughts on MCP Gateways? Have you faced similar integration challenges? We'd love to hear your experiences in the comments.