What’s the key difference between TrueFoundry and Portkey?

The difference between Portkey and TrueFoundry is that Portkey is an AI Gateway. It routes and monitors your API calls to external model providers. TrueFoundry is a complete AI infrastructure platform. Yes, our Gateway handles routing just like Portkey does, but we also manage the actual compute underneath. That means you can train models, fine-tune them, and deploy them on your own infrastructure, not just route traffic to someone else's API.

How does Portkey being free and open source compare to TrueFoundry as a paid platform?

If you’re comparing free Portkey vs TrueFoundry as a paid platform, Portkey's open-source gateway works well for prototypes and smaller projects. But in production, "free" usually means you're building your own reliability and support system. TrueFoundry is a commercial platform with guaranteed SLAs, 24/7 enterprise support, and managed infrastructure security. If your AI applications are business-critical, you need that safety net.

Which solution provides more advanced debugging tools?

Between TrueFoundry vs Portkey, TrueFoundry gives you full-stack visibility. Portkey logs your API requests: inputs, outputs, that kind of thing. Useful for debugging prompts. TrueFoundry connects those logs with your infrastructure metrics like GPU memory, pod health, and container logs. So when something breaks, you can see whether it's a model issue or an infrastructure problem like an OOM error. Portkey can't do that because it doesn't touch your infrastructure.

How does model deployment differ in TrueFoundry vs Portkey?

There is a critical distinction in model deployment in Portkey vs TrueFoundry. Portkey does not deploy or host models; it routes traffic to models already hosted elsewhere (like OpenAI or Anyscale). TrueFoundry acts as an orchestration engine. We allow you to take an open-source model (like Llama 3), containerize it, and deploy it directly onto your own cloud or on-premise infrastructure. We handle the autoscaling, GPU provisioning, and health checks, giving you ownership of both the model and the compute it runs on.

Which platform offers broader platform coverage?

If we compare broader platform coverage of Portkey and TroueFoundry, Portkey focuses on one stage: inference routing and monitoring. TrueFoundry covers your entire AI workflow: data prep, training, fine-tuning, evaluation, and deployment. Instead of juggling Portkey for routing, another tool for training, another for serving, you get one platform that handles everything.

Is TrueFoundry better than Portkey for production workloads?

If you are comparing TrueFoundry vs Portkey for strict data sovereignty requirements, TrueFoundry is usually the better fit. We run everything (compute, gateway, storage) inside your VPC or air-gapped environment. Native integration with your Kubernetes clusters, IAM, RBAC, and secrets management. Your model weights, training data, and everything stay inside your controlled infrastructure. Both platforms offer private deployments, but TrueFoundry gives you complete control from day one.

When evaluating TrueFoundry vs Portkey, which option fits long-term scaling needs?

When evaluating TrueFoundry vs Portkey, TrueFoundry is built for long-term scalability. Most teams start by consuming external APIs but eventually need to fine-tune or self-host models to reduce costs and latency. Portkey manages the API phase well, but forces you to find new tools when you shift to self-hosting. TrueFoundry manages both external APIs and internal self-hosted models seamlessly in one interface. This allows you to migrate from OpenAI to a private Llama model without changing your platform or rewriting your application logic.

We’re already using Portkey’s open-source gateway for LLMs - does it work fine for most use cases?

That’s great for the LLM API part – but consider the broader picture. TrueFoundry actually incorporates similar gateway capabilities and manages the surrounding infrastructure. You won’t need to build custom deployment pipelines or monitoring for your own models – it’s all provided out of the box. Plus, you continue to enjoy a unified API for external models while gaining enterprise reliability and support.

Do teams prefer open-source tools to avoid vendor lock-in?

TrueFoundry is deployed in your cloud account and built on open standards (containers, Kubernetes). Your data never leaves your environment. While the platform itself isn’t open-source, it doesn’t lock in your models – if needed, you could remove TrueFoundry and your apps would still run on standard infrastructure. We embrace open APIs and integration with OSS tools, so you get flexibility without having to maintain everything yourself.

If the use case is mostly routing to OpenAI or Anthropic, is a full platform overkill?

TrueFoundry can operate in a lightweight mode for just inference routing if that’s all you need today. However, many teams find that needs evolve: tomorrow you may want to deploy a custom model (for cost, latency, or privacy reasons) or add streaming data pipelines. With TrueFoundry, you’re already prepared. It’s not overkill – it’s future-proofing. In the meantime, the overhead is minimal, and you gain extras like unified monitoring across all your LLM providers and any custom models.

If a team has strong DevOps capabilities, can they manage ML infrastructure with existing tools?

Certainly, a skilled team can stitch together solutions (K8s, Portkey, custom scripts, etc.). But consider the opportunity cost: every hour spent on building and fixing infrastructure is an hour not spent on delivering ML value. TrueFoundry accelerates your DevOps efforts – it provides battle-tested automation (for scaling, logging, CI/CD) so your engineers can focus on higher-level innovation. Even the best teams leverage platforms to move faster and avoid re-inventing the wheel.

How does Portkey being free and open source compare to TrueFoundry as a paid platform?

If you’re comparing free Portkey vs TrueFoundry as a paid platform, TrueFoundry’s value is in the savings and efficiency gains it delivers. In practice, our customers report substantial cost savings (e.g. 40%+ cloud cost reduction) that often outweigh the platform fees. Also, the time saved in engineering (deployment automation, troubleshooting) translates to saved $$$ in manpower. Portkey being free addresses only one slice of the problem – you might still incur higher cloud bills and dev costs. TrueFoundry optimizes the whole pipeline, typically leading to a lower total cost of ownership.

Is TrueFoundry as up to date and innovative as newer LLM tools like Portkey?

TrueFoundry is at the cutting edge of GenAI deployment. In fact, it offers an AI Gateway comparable to Portkey’s (supporting 250+ models, guardrails, etc.), plus a comprehensive platform around it. We actively integrate the latest open-source tech (and we even partner with communities like LangChain, HuggingFace). With frequent updates, we ensure you have the newest capabilities – from supporting the latest LLMs to advanced features like RAG (Retrieval Augmented Generation) and more.

Compare TrueFoundry vs Kong

When TrueFoundry Makes Sense?

Choose TrueFoundry if you're deploying and scaling agentic AI that takes real actions, and doesn't just route model calls. Kong is fundamentally an API gateway, engineered for cheap, instant, stateless API calls, not expensive, chained AI requests. The result is that cost is not a first-class concept and nothing holds state long enough to price a chain or pause an action. TrueFoundry was built AI-native, with AI Gateway, MCP Gateway, and Agent Gateway deployed inside your VPC.

Architecture

Built AI-native. Models, MCP servers, tools, agents, and approval policies are first-class objects, and the control plane runs inside your VPC, on-prem, or air-gapped. Add a provider once, then pick models from a catalog with pricing pre-filled.

The AI Gateway is delivered as plugins attached to Services and Routes. Every new model is rebuilt from scratch: a Service, one Route per capability, and an AI Proxy Advanced plugin, roughly 4x to 5x the configuration burden, every single time.

MCP Auth

MCP auth runs both directions. Inbound SSO and dynamic client registration for IDEs and agents; outbound per-user OAuth to each provider, with consent, token storage, and automatic refresh held in the gateway. Managed MCP catalog, GA and self-hostable

MCP auth is inbound only. There is no outbound per-user consent, storage, or refresh. The MCP Registry is a Tech Preview that catalogs servers in Konnect, so it is unavailable on-prem.

Agentic Governance

Central agent registry to enforce RBAC and guardrails on agents. Destructive tool calls route to a named human for approval before executing. The gateway governs traffic; async services handle long-running execution.

No way to pause a tool call for human approval, and no native async execution. Long-running agent loops require your team to build and maintain the orchestration layer.

Guardrails

Subject-scoped rules, MCP per-invocation hooks firing before and after every tool call, and built-in PII/PHI detection, all in-process with zero external dependencies. HIPAA, GDPR, and air-gap ready.

Guardrails cover LLM prompts and responses only, so tool calls and the data they return go uninspected. Since a guardrail attaches to a model or a user group, never a single tool, the same rule has to be rebuilt for each group it should cover.

Data Residency

Auth, rate limits, guardrails, traces run inside your Kubernetes cluster. Built-in PII/PHI and secrets detection with no external services required. OTEL traces export to your own backends. Nothing leaves your environment by default.

PII sanitization is version and license-gated, and in the hybrid model most teams treat as on-prem the control plane still runs in Kong's cloud.

Gateway Performance

Purpose-built for AI traffic: ~3ms latency at 250 RPS per pod, scaling linearly. Auth, rate limiting, and guardrails all run in-memory on the hot path, with no plugin chain overhead and no licensing surprises.

A general-purpose API gateway extended with AI plugins. Multi-target AI routing requires an Enterprise license.

Routing & Load Balancing

Native latency-based routing using inter-token latency / TPOT, adaptive priority with SLA cutoffs, typed YAML policies, and OTEL export. Routing is configurable at team, model, and application level.

AI routing is a plugin layer on top of a general API gateway. Feature availability depends on your plugin version and license tier.

Observability

LLM request traces connected to GPU memory, pod health, and container logs in a single UI. Infrastructure failures and model failures are diagnosed in the same place, with no additional tooling required.

Integrates into existing Kong OTel/Prometheus/Grafana pipelines. AI metrics require explicit plugin configuration. Prompt body capture needs a deliberate redaction strategy.

Prompt Management

Version history, compare/diff, CI-gated deployments, and dry-run previews, so prompt changes and infrastructure changes move through one pipeline.

Gateway-level prompt injection via AI Prompt Decorator. No versioning registry, no playground, no per-model prompt overrides. Falls short for teams doing active prompt iteration.

Org & Team Management

Tenant isolation backed by Kubernetes namespace boundaries. Scales from 5 to 500 teams from a single static config.

Kong's consumer and consumer-group model is mature, but it lives entirely at the API gateway layer. The AI gateway has no identity of its own. No SCIM provisioning, and budget controls lack cumulative reset and alert thresholds.

Support

24×7 via Slack and on-call engineers, dedicated AM. G2 rating 9.9/10.

Community support for OSS tier. Enterprise SLAs available.

Key Evaluation Questions

We're already running Kong for our APIs. Should we use it for AI too?

API gateways assume requests are cheap, fast, and independent. AI requests are none of these: one user action becomes a chain of model and tool calls that is expensive and consequential. TrueFoundry is built around that chain, so cost, identity, and approval apply to the whole sequence.

Kong evaluates each call independently and discards it, so a chain of calls is never governed as a whole. Cost across the chain, the user an agent acted for, and approval on risky calls become your team's to build. This is a core architectural gap that an API Gateway cannot resolve.

Are we planning to scale agentic workflows on production workloads?

TrueFoundry MCP Gateway holds destructive tool calls for human approval, fires guardrails before and after every tool call, and brokers per-user credentials so agents act as the logged-in user, all inside your VPC and production-ready today.

Kong's AI MCP Proxy and per-tool ACLs give existing Kong users an MCP control surface. However, no guardrail reaches the tool-call path, and there's no way to pause a call for approval. This means agents can take destructive actions with no checkpoints.

How do we control AI costs across teams and self-hosted models?

Pricing is maintained for you in an open source, auto-updated catalog with regional and tiered rates, and private contract rates are supported. Budgets enforce on the hot path and block requests at the limit. Cost attribution runs across team, user, model, and application, self-hosted or air-gapped.

Kong governs token counts rather than dollars. Dollar figures are computed from prices your team types in per model and maintains by hand. Providers reprice often, nothing flags it, so budgets enforce stale numbers. Self-hosted cost attribution is not supported.

We need full data sovereignty, no payload or metadata egress.

TrueFoundry runs the entire hot path inside your K8s cluster with no external dependencies. Built-in PII/PHI and secrets detection requires no external services. OTEL traces export to your own backends. Full sovereignty is the default, not an add-on.

PII sanitization and advanced compliance features are enterprise license-gated and version-dependent. And in the hybrid model, the control plane still runs in Kong's cloud.

Do we need full-stack observability or just LLM-level metrics?

TrueFoundry connects LLM request traces with GPU memory, pod health, and container logs in one UI, infrastructure failures and model failures are diagnosed in the same place, without additional tooling.

Kong’s observability integrates cleanly into existing OTel/Prometheus/Grafana pipelines. But AI metrics require explicit plugin configuration and there’s no infrastructure-level visibility since Kong doesn’t host models.

We want to move from external APIs to self-hosted models without re-architecting.

TrueFoundry manages both external API routing and self-hosted model deployment from one platform. Moving from OpenAI to a private Llama deployment is a configuration change, not a migration. Training, fine-tuning, serving, and gateway are unified.

Kong routes AI traffic to wherever you point it, external or self-hosted. But model deployment, training, and fine-tuning are entirely outside its scope. As your AI stack matures, you’ll need additional platforms to cover what Kong doesn’t.

How TrueFoundry acts as a Painkiller

Limited MCP & Agent Governance

Human approval gates on destructive tool calls, dedicated MCP pre/post-tool guardrail hooks, Virtual MCP Servers, Cedar-based policy, and async execution lifecycle, all documented and production-ready in one platform.

Teams ship agents that write, pay, and deploy instead of stalling at read-only pilots. Every risky action has a named approver and a recorded decision, so the control satisfies audit requirements and agents can move from pilots to production.

No Dollar-Based Cost Controls

Native per-token cost tracking everywhere including on-prem, budgets enforced on the hot path so overspend is blocked before it happens, and attribution by team, user, model, and application.

Finance sees AI spend by team, customer, and application. Budgets stop overspending at the limit rather than after the invoice arrives. Nobody is reconciling model prices by hand.

Plugin Complexity That Grows With Your Stack

TrueFoundry is purpose-built for AI. One virtual model object per model means adding or swapping a model is a single declarative change, so your configuration surface stays flat as your model count grows.

Adding or swapping a model takes minutes instead of a coordinated change across several objects. Engineering time shifts from gateway plumbing to AI products, and the configuration surface stays flat as the model count grows.

Incomplete Data Sovereignty

Every enforcement layer (auth, rate limits, guardrails, PII/PHI detection) runs in-process inside your K8s cluster. No external service calls on the hot path.

Security and compliance sign off without exceptions, because nothing leaves the cluster. Regulated, air-gapped, and government workloads run on the same architecture as everything else, so there is no separate compliance track to maintain.

No Native Support for Self-Hosted Models

TrueFoundry manages both external API routing and internal self-hosted model deployment from one interface. Switching from a managed API to a private model is a config change, not a platform migration.

When workloads need to shift to private models, the move is a configuration change. Teams avoid a second platform, a second integration to maintain, and a migration project nobody budgeted for.

Slow Time-to-Production for AI Teams

Platform teams set policy once at the infrastructure layer: access rules, guardrails, fallback and load-balancing policies, rate limits, and budgets. Application teams then self-serve models, MCP servers, and agents within those bounds, inheriting every control automatically, with no gateway configuration per application.

AI teams self-serve deployments in hours rather than waiting on platform tickets. Shipping an AI application no longer requires gateway expertise, so the platform team stops being a bottleneck for every new model, route, and policy.

Competitive Differentiators Evaluation Questions Key Painpoints

FAQs/Common Objections

What's the key difference between TrueFoundry and Kong AI Gateway?

Kong is a general-purpose API gateway with AI added through plugins. Architecturally it is a phase-based proxy: one independent decision per request, then discarded. TrueFoundry is a vendor-neutral AI infrastructure platform built for AI workloads, with a stateful control plane holding paused approvals, sessions, and per-user credentials as durable objects. That is why Kong cannot pause a tool call for approval, its guardrails never reach the MCP tool-call path, and it prices spend from figures your team maintains by hand. AI Gateway, MCP Gateway, Agent Gateway, and model deployment run in one Kubernetes-native system inside your VPC.

We're already running Kong for our APIs. Should we use it for AI too?

Only while AI remains model routing. Kong configures AI separately from your API gateway, so an AI layer is new work either way. An API gateway evaluates one call at a time, but a single agent instruction becomes dozens of model and tool calls. TrueFoundry governs those as one job, so cost, identity, and approval apply across the sequence. It deploys alongside Kong, so this is an addition, not a migration.

Kong has an MCP OAuth plugin. Doesn't that cover auth?

It covers who is calling your gateway, not who your agents act as in downstream tools. Kong's MCP OAuth2 plugin validates inbound tokens and supports token exchange where your identity provider offers it, but there is no authorization-code consent flow to third-party providers. When an agent needs to act as a specific person in Slack or GitHub, with that person's permissions, your engineers build and operate consent, token storage, and refresh. TrueFoundry runs that lifecycle natively, and for common tools the OAuth client is operated for you.

How does MCP governance compare between the two platforms?

TrueFoundry provides a purpose-built MCP surface: approval gates that hold destructive tool calls until a named person decides, pre and post-tool guardrail hooks, Virtual MCP Servers, Cedar policy, and gateway-brokered per-user credentials, all production-ready in your cluster today. Kong's AI MCP Proxy and per-tool ACLs are a real control surface, and MCP OAuth2 validates inbound tokens and supports token exchange where your identity provider offers it. The gaps: no guardrail sees a tool call, and nothing holds one for approval.

How does cost control work across teams and self-hosted models?

TrueFoundry enforces budgets on the hot path, so overspend is blocked rather than flagged after. Pricing is maintained for you in an open source, auto-updated catalog with regional, tiered, and private contract rates, and attribution runs across teams, users, models, and applications, including self-hosted fleets. Kong can show a dollar figure, but it is computed from prices your team types in per model and maintains by hand. Providers reprice often and nothing flags it, so budgets quietly enforce stale numbers.

Which platform is better for production agent workloads?

TrueFoundry documents both gateway governance and execution lifecycle in one architecture. Guardrails fire on LLM input, LLM output, before a tool call, and after it returns. Destructive calls route to a named human before executing, and the split-plane design lets the gateway govern traffic while async services handle long-running loops. Kong brings tool traffic into the plugin ecosystem, which helps existing users, but there is no approval gate and no async execution substrate.

How does data residency differ?

TrueFoundry runs the entire hot path, including auth, rate limits, guardrails, PII and PHI detection, and traces, inside your Kubernetes cluster with no external dependencies. Sovereignty is the default architecture, not a configuration option. Kong's data governance catalog is strong, but the capabilities regulated industries need most, such as bidirectional PII sanitization with restoration, are license-gated and version-dependent. Worth pressure-testing early.

How does observability compare?

TrueFoundry gives full-stack visibility out of the box: LLM traces connected to GPU memory, pod health, and container logs in one UI, with no configuration needed for meaningful signal. Kong's observability is powerful for teams already running OTel, Prometheus, and Grafana, where LLM traffic joins the same pipeline. The tradeoff is setup: AI metrics need explicit plugin configuration, and prompt capture needs a redaction strategy first.

Can we wait? Kong ships quickly.

Some gaps are roadmap items. Two are design choices: approval enforcement is delegated to the agent's client deliberately, and per-user tool authentication assumes your identity provider handles it. Closing either requires what a phase-based proxy is built to avoid, state that survives the request. Holding a call for hours, or refreshing a token per user, is a new stateful subsystem with its own storage, failover, and tenancy, not a plugin. Waiting bets your compliance posture on an unpublished timeline.

Do we really need TrueFoundry if we only do AI gateway routing today?

TrueFoundry can run as a lightweight routing layer with monitoring, guardrails, and cost controls. AI workloads rarely stay still, though. As your AI workflows expand, the architecture has to scale alongside them. Routing is easy to move later; approval gates, per-user credentials, and chain-level cost are what force a re-platform, and choosing a gateway without them simply delays an inevitable restructuring.