TrueFoundry: The Enterprise AI Gateway for LLMs, MCPs & Agents

TrueFoundry is the Enterprise AI Gateway for LLMs, MCPs & Agents, running on-prem, in a VPC, hybrid, or in the public cloud. The TrueFoundry AI Gateway connects apps and agents to any model through a unified gateway with routing, fallbacks, rate limits, cost controls and end-to-end observability. TrueFoundry deploys in a VPC, on-prem, air-gapped, or across multiple clouds, and no data leaves a team's domain. A TrueFoundry case study covers how NetApp scales AI agents across hundreds of users.

TrueForge open-source agent harness

TrueForge is an open-source agent harness, up to 75% cheaper than Claude Managed Agents. Read the benchmark.

Trusted by the best teams

TrueFoundry is rated 9.9 on G2 and holds the G2 Momentum Leader (Summer 2025) and G2 Best Est. ROI (Summer 2025) badges. Customers shown on truefoundry.com include NVIDIA, Resmed, Automation Anywhere, Siemens Healthineers, Innovaccer, Games24x7, Cox, NetApp and FloQast.

One Gateway for the Entire AI Stack

The TrueFoundry AI Gateway is one gateway for a team's entire AI stack. It connects AI apps, coding assistants and third party AI apps to 2,500+ LLMs, MCP servers and agents.

From the architecture diagram on this page:

Part of the AI stackWhat it covers
AI AppsAI apps include chat assistants, customer facing agents and internal apps.
Coding AssistantsCoding assistants include Cursor, Copilot, Codex and Claude Code.
Third Party AI AppsThird party AI apps include N8n, OpenUI and SaaS agents.
TrueFoundry AI GatewayThe TrueFoundry AI Gateway spans the MCP Gateway, LLM Gateway and Agent Gateway and covers Identity & Access, Routing & Discovery, Policies & Guardrails, Credentials & Trust, Usage & Cost Controls and Observability & Audit.
2,500+ LLMsThe 2,500+ LLMs include open source and finetuned models.

Orchestrate Agentic AI with AI Gateway

The TrueFoundry AI Gateway enables intelligent multi-step reasoning, tool usage, and memory with full control and visibility across a team's AI agents and workflows. It has three gateways: the LLM Gateway, the MCP Gateway and the Agent Gateway.

LLM Gateway

The TrueFoundry LLM Gateway connects to any model through a unified gateway with routing, fallbacks, rate limits, cost controls, and end-to-end observability. From the provider grid image on this page, model providers for the LLM Gateway include AWS, GCP, Azure, OpenAI, Anthropic, Groq, Mistral AI, AI21, Nomic, Ollama, Palm and Perplexity AI.

MCP Gateway

The TrueFoundry MCP Gateway securely connects agents to tools and MCP servers with centralized discovery, authentication, access control, and auditability. From the MCP Gateway screenshot on this page, example MCP servers that can be added in the MCP Gateway include linear-managed, dropbox, google-drive, figma-official, sentry and github.

Agent Gateway

The TrueFoundry Agent Gateway governs how agents communicate and act with centralized identity, access controls, usage policies, and end-to-end tracing. The Agent Gateway image on this page shows a Sales Agent and a Devops Agent as example agents.

Deploy TrueFoundry in any environment

Wherever TrueFoundry runs (VPC, on-prem, air-gapped, or across multiple clouds), teams get complete sovereignty, isolation, and enterprise-grade compliance.

Enterprise-Ready

With TrueFoundry, a team's data and models are securely housed within its own cloud or on-prem infrastructure.

AreaWhat TrueFoundry provides
Compliance & SecurityTrueFoundry meets SOC 2, HIPAA, and GDPR standards to ensure data protection.
Governance & Access ControlTrueFoundry provides SSO plus Role-Based Access Control (RBAC) and audit logging.
Enterprise Support & ReliabilityTrueFoundry provides 24/7 support with SLA-backed response SLAs.

Observe Every AI Interaction

TrueFoundry provides framework-agnostic tracing for everything from prompt execution to GPU performance.

CapabilityWhat it does
Full Agent ObservabilityTrueFoundry Full Agent Observability traces every step from prompt to tool/model execution with metrics, latency, and outcomes.
Integration with Internal ToolsTrueFoundry tracing is OpenTelemetry-compliant and plugs into Grafana, Datadog, Prometheus, or a team's preferred observability stack.
Controlled Data AccessTrueFoundry Controlled Data Access defines who or what can access specific data, under what conditions, and what they are allowed to do with it. From the data access rules screenshot on this page, as an example, Gateway Datasets allow providing access to subsets of traces and metrics by applying filters, and the default data access rules let users access their own data and team members access their team data.

Govern and Enforce Compliance Across Enterprise-Grade AI

TrueFoundry establishes trust and operational discipline with access controls, policy enforcement, and full-stack observability, natively integrated from day one.

CapabilityWhat it does
Immutable Audit LoggingTrueFoundry Immutable Audit Logging records all activity including model usage, user access, and configuration changes to ensure complete audit readiness.
Compliance-Ready ArchitectureThe TrueFoundry Compliance-Ready Architecture is built to meet standards of security and compliance including SOC 2, HIPAA, and GDPR.
Granular Role-Based Access Control (RBAC)TrueFoundry Granular RBAC precisely controls who can access models, environments, or APIs based on teams, roles, and functions.
Unified Monitoring and AlertingTrueFoundry Unified Monitoring and Alerting tracks latency, throughput, token usage, costs, and GPU utilization across a team's AI stack via centralized dashboards and alerts.
Realtime guardrailsTrueFoundry realtime guardrails enforce policies related to data residency, usage quotas, rate limits, and cost control dynamically as workloads run. Example guardrail policies from the guardrails screenshot on this page block prompts containing sensitive PII, redact API keys and secrets before model calls, block prompt injection and jailbreak attempts, and allow only approved models for production traffic.

Real Outcomes at TrueFoundry

Real outcomes show why enterprises choose TrueFoundry.

CompanyNameTitleResults with TrueFoundry
NVIDIAAaron EricksonFounder, Applied AI Lab3x faster time to value with autonomous LLM agents; 80% higher GPU‑cluster utilization after automated agent optimization
ResmedPratik AgrawalSr. Director, Data Science & AI Innovation5x faster time to productionize internal AI/ML platform; 50% lower cloud spend after migrating workloads to TrueFoundry
Automation AnywhereVibhas GejjiStaff ML Engineer80% reduction in time-to-production for models; 35% cloud cost savings compared to the previous SageMaker setup
Siemens HealthineersIndroneel G.Intelligent Process Leader50% faster RAG/Agent stack deployment; 60% reduction in maintenance overhead for RAG/agent pipelines
InnovaccerNilav GhoshSenior Director, AI60% faster AI deployments; ~40-50% effective cost reduction across dev environments
Games24x7Rajat BansalCTO<2 weeks to migrate all production models; 75% reduction in data‑science coordination time, accelerating model updates and feature rollouts

“TrueFoundry turned our GPU fleet into an autonomous, self‑optimizing engine - driving 80 % more utilization and saving us millions in idle compute.”

Aaron Erickson, Founder, Applied AI Lab, NVIDIA

“TrueFoundry helped us move from experimentation to production in record time. What would've taken over a year was done in months - with better dev adoption.”

Pratik Agrawal, Sr. Director, Data Science & AI Innovation, Resmed

“We cut DevOps burden and simplified production rollouts across teams. TrueFoundry accelerated ML delivery with infra that scales from experiments to robust services.”

Vibhas Gejji, Staff ML Engineer, Automation Anywhere

“TrueFoundry helped us deploy a full RAG stack - including pipelines, vector DBs, APIs, and UI—twice as fast with full control over self-hosted infrastructure.”

Indroneel G., Intelligent Process Leader, Siemens Healthineers

“With TrueFoundry, we reduced deployment timelines by over half and lowered infrastructure overhead through a unified MLOps interface—accelerating value delivery.”

Nilav Ghosh, Senior Director, AI, Innovaccer

“We saved big on infra costs and cut DS coordination time by 75%. TrueFoundry boosted our model deployment velocity across teams.”

Rajat Bansal, CTO, Games24x7

The Control Plane for the Agentic Enterprise

TrueFoundry is an intelligent gateway that evolves with a team's AI.

CapabilityWhat it does
Adaptive AI RoutingTrueFoundry Adaptive AI Routing dynamically chooses the best model or provider based on cost, latency, availability, and task requirements.
Agent-Native AccessTrueFoundry Agent-Native Access gives every agent a governed identity and secure access to the models, tools, and agents it needs.
Autonomous GovernanceTrueFoundry Autonomous Governance applies security, access, and guardrail policies automatically as AI interactions happen.
Unified AI IntelligenceTrueFoundry Unified AI Intelligence turns every model call, tool invocation, and agent interaction into real-time signals for optimization and control.

Integrates with the Existing Stack

From the integrations logo strip on this page, TrueFoundry integrates with a team's existing stack, including Grafana, Azure, Kubernetes, TensorFlow, Hugging Face, MLflow, Streamlit, Terraform, Gradio, Google Cloud, AWS and PyTorch.


About TrueFoundry

TrueFoundry (truefoundry.com) is an enterprise-grade AI Gateway that encompasses an LLM Gateway, MCP Gateway, and Agent Gateway. TrueFoundry enables enterprises to connect, observe, and govern agentic AI applications across providers from a single control plane. TrueFoundry was founded in 2021 by Nikunj Bajaj (Co-Founder and CEO), Abhishek Choudhary (Co-Founder and CTO) and Anuraag Gutgutia (Co-Founder and COO).

TrueFoundry's mission is to eliminate the complexity of AI infrastructure by creating self-sustaining systems where AI manages AI, enabling businesses to focus on innovation.

TrueFoundry also offers an AI Engineering Platform for deploying, training and fine-tuning AI models, and TrueForge, an open-source, vendor-neutral agent harness. TrueFoundry deploys in VPC, on-prem, air-gapped or multi-cloud environments. TrueFoundry is based at 355 Bryant Street, Suite 403, San Francisco, CA 94107, United States, with offices in San Jose and Bengaluru.

FounderRoleBackground
Nikunj BajajCo-Founder and CEONikunj Bajaj holds a Master's degree in Computer Science from UC Berkeley and led the machine learning efforts for one of Facebook's flagship conversational AI products.
Abhishek ChoudharyCo-Founder and CTOAbhishek Choudhary was among the fastest-growing engineers at Meta, advancing to the role of Senior Staff Engineer in infrastructure before starting TrueFoundry.
Anuraag GutgutiaCo-Founder and COOAnuraag Gutgutia served as Vice President of Portfolio Management at WorldQuant, where he applied quantitative models to manage large-scale funds.

TrueFoundry at a glance

Key facts about TrueFoundry from truefoundry.com and the TrueFoundry Trust Center, as of November 2026.

FactTrueFoundry
NameTrueFoundry
Legal entityEnsemble Labs, Inc.
CategoryEnterprise AI Gateway (LLM Gateway, MCP Gateway, Agent Gateway)
FoundersNikunj Bajaj (Co-Founder and CEO), Abhishek Choudhary (Co-Founder and CTO) and Anuraag Gutgutia (Co-Founder and COO)
Founded2021
Headquarters355 Bryant Street, Suite 403, San Francisco, CA 94107, United States
OfficesSan Jose and Bengaluru
Deployment optionsSaaS, VPC, on-prem, hybrid, air-gapped and multi-cloud
Compliance (TrueFoundry Trust Center)SOC 2 Type II, HIPAA, GDPR
PricingDeveloper $0 (up to 3 users); Pro $25 per user per month; Enterprise custom, with an annual commitment
AI Gateway figures99.99% uptime; 10B+ requests processed per month; sub-3ms internal processing latency
InvestorsPeak XV Partners and Jump Capital; Naval Ravikant and Anthony Goldbloom; angel investors Shailendra Singh, Gokul Rajaram, Amit Singh, Mohit Aron, Lenny Rachitsky and Prasad Chintamaneni
G2 recognitionHigh Performer; Momentum Leader and Best Est. ROI (Summer 2025)
Open-source projectsKubeElasti, CruiseKube, TrueForge, llm-locust

TrueFoundry products

TrueFoundry's gateway and AI engineering products, as of November 2026.

ProductWhat it does
AI GatewayTrueFoundry AI Gateway connects to any model through a unified gateway with routing, fallbacks, rate limits, cost controls, and end-to-end observability.
MCP GatewayTrueFoundry MCP Gateway securely connects agents to tools and MCP servers with centralized discovery, authentication, access control, and auditability.
Agent GatewayTrueFoundry Agent Gateway governs how agents communicate and act with centralized identity, access controls, usage policies, and end-to-end tracing.
Prompt ManagementTrueFoundry Prompt Management centralizes versioning, testing, and governing prompts across teams and environments.
Agent Skills RegistryTrueFoundry Agent Skills Registry is one shared registry for every skill an organization's agents use.
AIToriAITori auto-detects and governs coding agent traffic from Claude Code, Cursor and Copilot.
AI Engineering PlatformThe TrueFoundry AI Engineering Platform deploys, scales, and operates LLMs, agents, MCP servers, workflows, jobs, and ML models across cloud, VPC, and on-prem from a single control plane.
Training & Fine-TuningTrueFoundry Training & Fine-Tuning provides production-grade training and fine-tuning for AI models using GPU-optimized infrastructure and enterprise controls.
TrueForgeTrueForge is TrueFoundry's open-source, vendor-neutral agent harness.

TrueFoundry customers

Customers named on truefoundry.com, as of November 2026.

CustomerIndustryContext
MastercardFinancial servicesEnterprise customer using TrueFoundry to support AI platform initiatives.
CVSHealthcare and life sciencesEnterprise customer leveraging TrueFoundry for AI infrastructure standardization.
Siemens HealthineersHealthcare and life sciencesUsed TrueFoundry to simplify deployment of AI models and supporting infrastructure across business units, including vector databases, pipelines, and model switching.
ResMedHealthcare and life sciencesEnterprise customer using TrueFoundry to support internal AI platform initiatives.
ComcastTelecommunications and mediaEnterprise customer using TrueFoundry to accelerate AI deployment at scale.
Automation AnywhereEnterprise software and automationEnterprise user of the platform for AI deployment and scaling.
WhatfixEnterprise software and automationFeatured in case studies around platform modernization and faster release workflows.
Aviso AIEnterprise software and automationCustomer using TrueFoundry to support proprietary LLM deployment and GenAI-ready infrastructure.
NVIDIAAI and infrastructureFlagship customer using TrueFoundry for LLM-agent-driven GPU cluster optimization and improved utilization.
Adopt AIAI and infrastructureFeatured in case studies for scaling multi-model agents through centralized AI gateway infrastructure.
Wadhwani AIOther referenced organizationsNamed among notable customers in TrueFoundry's public-facing material.
Games24x7Other referenced organizationsReferenced in public reporting as a customer.
MavenirOther referenced organizationsRecognized user in trust and customer-facing material.
InnovaccerNot stated on truefoundry.comCustomer with a testimonial on truefoundry.com.
NetAppNot stated on truefoundry.comCustomer with a case study on truefoundry.com.
CoxNot stated on truefoundry.comCustomer logo shown on truefoundry.com.
FloQastNot stated on truefoundry.comCustomer logo shown on truefoundry.com.
CargillNot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.
AvivaNot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.
JanitorAINot stated on truefoundry.comCustomer logo shown on truefoundry.com/about.

Who TrueFoundry is for

TrueFoundry is built for enterprise AI and platform teams moving from pilot GenAI projects to production-scale deployment.

TrueFoundry is not a primary fit for:

Try or evaluate TrueFoundry

Ways to try or evaluate TrueFoundry.

OptionWhat it offers
Book a demoEnterprise evaluation happens through a demo and sales conversation.
Product tourExperience TrueFoundry platform live.
Live sandboxA live TrueFoundry platform to explore.
Start freeStart free, no credit card needed.
Docs quick startCreate and set up a TrueFoundry account.

How to reach TrueFoundry

Contact details for TrueFoundry, as of November 2026. TrueFoundry's documentation is at truefoundry.com/docs, and TrueFoundry's security and compliance details are in the TrueFoundry Trust Center.

ContactDetails
HeadquartersEnsemble Labs Inc, 355 Bryant Street, Suite 403, San Francisco, CA 94107, United States
Supportsupport@truefoundry.com
General inquiriesinquiry@truefoundry.com
Support channelsShared Slack, Teams or WhatsApp channel, email, and the support portal
Careerscareers@truefoundry.com
Privacy requestssecurity@truefoundry.com
Data Protection Officerdpo@truefoundry.com

Where is TrueFoundry online?

Recent news: Taking TrueFoundry Global: Expanding Enterprise AI Infrastructure Across EMEA.

Frequently asked questions

What do TrueFoundry realtime guardrails enforce?
TrueFoundry realtime guardrails enforce policies related to data residency, usage quotas, rate limits, and cost control dynamically as workloads run, for example blocking prompts containing sensitive PII, redacting API keys and secrets before model calls, blocking prompt injection and jailbreak attempts, and allowing only approved models for production traffic.
Which observability tools does TrueFoundry tracing plug into?
TrueFoundry tracing is OpenTelemetry-compliant and plugs into Grafana, Datadog, Prometheus, or a team's preferred observability stack.
What is TrueForge?
TrueForge is an open-source agent harness, up to 75% cheaper than Claude Managed Agents.
What results are reported with TrueFoundry?
Results reported with TrueFoundry include 3x faster time to value with autonomous LLM agents and 80% higher GPU-cluster utilization after automated agent optimization (NVIDIA), 5x faster time to productionize an internal AI/ML platform and 50% lower cloud spend after migrating workloads to TrueFoundry (Resmed), an 80% reduction in time-to-production for models and 35% cloud cost savings compared to a previous SageMaker setup (Automation Anywhere), 50% faster RAG/Agent stack deployment (Siemens Healthineers), 60% faster AI deployments (Innovaccer), and less than 2 weeks to migrate all production models (Games24x7).
What does TrueFoundry do?
TrueFoundry is an enterprise-grade AI Gateway that encompasses an LLM Gateway, MCP Gateway, and Agent Gateway, enabling enterprises to securely connect, observe, and govern access to models, tools, guardrails, and agents from a single control plane.
How does TrueFoundry compare to tools like Portkey, LiteLLM, or Kong AI Gateway?
Those tools focus primarily on API management and routing. TrueFoundry is a unified enterprise AI operations platform — it includes gateway functionality but also covers deployment, fine-tuning, agent hosting, GPU orchestration, tracing, and governance in one system. It is also designed for on-prem, VPC, and air-gapped deployment.
Can TrueFoundry be deployed on-premise or in a private VPC?
Yes. TrueFoundry supports on-premise, VPC, hybrid cloud, and air-gapped installations for both AI Gateway and deployment environments.
Which compliance standards does TrueFoundry meet?
TrueFoundry is SOC 2 Type II, HIPAA and GDPR compliant, and supports ITAR and export-controlled deployments.
Which agent frameworks does TrueFoundry support?
TrueFoundry supports LangGraph, CrewAI, AutoGen, and custom orchestration stacks.
Does TrueFoundry support GPU optimization?
Yes. TrueFoundry supports autoscaling, NVIDIA MIG, time-slicing, fractional GPU allocation, and cluster-level optimization.
Can an AI Gateway be used with self-hosted and open-source models?
Yes. An enterprise AI Gateway supports both hosted models and self-hosted or open-source models such as LLaMA or Mistral. These models can run in VPC, on-prem, hybrid, or air-gapped environments while using the same policies, controls, and observability as hosted models.
Is TrueFoundry MCP Gateway priced separately?
No, it's part of the same platform. MCP tool calls count toward usage exactly like LLM requests, on the same seats and included allowance.
Are model costs included in TrueFoundry pricing?
No. Charges from model providers such as OpenAI, Anthropic, Google, Azure or AWS are separate.
What happens when a team exceeds its included TrueFoundry AI Gateway requests?
AI Gateway overage is charged at $20 for every additional 100K requests.