Learn how to use TrueFoundry’s unified Chat Completions API to interact with models from multiple providers through a consistent interface.
TrueFoundry AI Gateway provides a universal API for all supported models via the standard OpenAI /chat/completions endpoint. This unified interface allows you to seamlessly work with models from different providers through a consistent API.API Reference:POST /chat/completions
Provider capabilities
The AI Gateway maps the OpenAI Chat Completions contract to each provider. The table below summarizes feature support by provider for this endpoint.
You can use the standard OpenAI client to send requests to the AI Gateway:
from openai import OpenAIclient = OpenAI( api_key="your_truefoundry_api_key", base_url="{GATEWAY_BASE_URL}")response = client.chat.completions.create( model="openai-main/gpt-4o-mini", # this is the truefoundry model id messages=[{"role": "user", "content": "Hello, how are you?"}])print(response.choices[0].message.content)
Configuration
You will need to configure the following:
base_url: The base URL of the TrueFoundry AI Gateway
model: TrueFoundry model ID in the format provider_account/model_name (available in the LLM playground UI)
See Integrate with code for instructions on obtaining these values.For using native provider SDKs (OpenAI, Google Gen AI, Anthropic, boto3), see Native SDK Support.
Input Controls
System Prompts
System prompts set the behavior and context for the model by defining the assistant’s role, tone, and constraints:
response = client.chat.completions.create( model="openai-main/gpt-4o-mini", messages=[ {"role": "system", "content": "You are a helpful assistant that specializes in Python programming."}, {"role": "user", "content": "How do I write a function to calculate factorial?"} ])
Request Parameters
Fine-tune model behavior with these common parameters:
response = client.chat.completions.create( model="openai-main/gpt-4o-mini", messages=[{"role": "user", "content": "Hello, how are you?"}], temperature=0.7, # Controls randomness (0.0 to 1.0) max_tokens=100, # Maximum tokens to generate verbosity="high", # Constrains verbosity: low, medium, high top_p=0.9, # Nucleus sampling parameter frequency_penalty=0.0, # Reduces repetition presence_penalty=0.0, # Encourages new topics stop=["\n", "Human:"] # Stop sequences)
Some models don’t support all parameters. For example, temperature is not supported by o series models like o3-mini.
max_tokens defaults per provider
max_tokens is optional in the OpenAI-compatible API, but what happens when you omit it depends on the provider integration. Anthropic’s Messages API requiresmax_tokens, so for integrations that speak that format the AI Gateway supplies a default rather than letting the request fail.
Provider integration
If you omit max_tokens
Effective limit
Anthropic
Gateway sends a default
2048 tokens
AWS Bedrock Mantle (Claude models)
Gateway sends a default
2048 tokens
AWS Bedrock (Converse)
Gateway sends nothing; Bedrock applies its own per-model default
Model-dependent — 4096 for Claude Opus and Sonnet
All other providers
Gateway sends nothing; the provider’s own default applies
Provider-dependent
Always set max_tokens explicitly for long-form generations. If you rely on the default, a response can be truncated mid-sentence with finish_reason: "length" and no error — most visibly on Claude Opus and Sonnet via AWS Bedrock, where Bedrock caps output at 4096 tokens despite the models supporting far more.
Service tier
Set service_tier to flex, priority, fast, or ultrafast when the provider and model support it. The AI Gateway bills the served tier from the public catalog overlay, not the value you requested. See Service tiers.
{"docsConfig":{"theme":"mint","$schema":"https://mintlify.com/docs.json","name":"TrueFoundry Docs","description":"TrueFoundry documentation — enterprise-grade AI Gateway combining LLM, MCP, and Agent Gateways to connect, monitor, and govern agentic AI applications across providers from a unified control plane.","colors":{"primary":"#4f46e5","light":"#4f46e5","dark":"#4f46e5"},"logo":{"light":"https://mintcdn.com/truefoundry/e2uUimEwX30qkWxc/logo/light.svg?fit=max\u0026auto=format\u0026n=e2uUimEwX30qkWxc\u0026q=85\u0026s=6068d83664aeda0239bec2422657dd20","dark":"https://mintcdn.com/truefoundry/e2uUimEwX30qkWxc/logo/dark.svg?fit=max\u0026auto=format\u0026n=e2uUimEwX30qkWxc\u0026q=85\u0026s=c1cb26554a546d84500b40f7057bece8"},"favicon":"https://mintcdn.com/truefoundry/6bbtTt95c1iCcyph/favicon.svg?fit=max\u0026auto=format\u0026n=6bbtTt95c1iCcyph\u0026q=85\u0026s=f55bde34214b38034126d05b08ea16c7","appearance":{"default":"light"},"background":{"decoration":"grid"},"navbar":{"links":[{"href":"https://www.truefoundry.com/book-demo","label":"Book Demo"},{"href":"https://www.truefoundry.com/blog","label":"Blog"}],"primary":{"type":"button","label":"Sign Up","href":"https://www.truefoundry.com/register"}},"footer":{"socials":{"x":"https://twitter.com/truefoundry","linkedin":"https://in.linkedin.com/company/truefoundry"}},"seo":{"metatags":{"og:type":"website","og:site_name":"TrueFoundry Docs","og:image":"https://assets.production.truefoundry.com/logo.svg","og:description":"TrueFoundry offers an enterprise-grade AI Gateway combining LLM, MCP, and Agent Gateways—empowering businesses to connect, monitor, and govern agentic AI applications across providers from a unified control plane.","description":"TrueFoundry offers an enterprise-grade AI Gateway combining LLM, MCP, and Agent Gateways—empowering businesses to connect, monitor, and govern agentic AI applications across providers from a unified control plane."}},"icons":{"library":"fontawesome"},"styling":{"codeblocks":"dark","latex":true},"contextual":{"options":["copy","view","chatgpt","claude","mcp","cursor","vscode"],"display":"header"},"navigation":{"pages":[]}},"deployment":{"deploymentMetadataValue":{"subdomain":"truefoundry","hasAudioNarration":false,"actualSubdomain":"truefoundry","customDomains":["www.truefoundry.com"],"gitSource":{"type":"github","owner":"truefoundry","repo":"docs-mintlify","deployBranch":"main","contentDirectory":"","isPrivate":true,"mountPath":"","label":""},"feedback":{"thumbs":true},"entitlements":{"AI_CHAT":{"status":"ENABLED"},"WEB_MCP":{"status":"ENABLED"},"NARRATION":{"status":"ENABLED"}},"buildId":"6b00f4105dafc505a90bf8d7","clientVersion":"0.0.4270","searchFilterCounts":[{"language":"default","tag":"AI Gateway","count":253},{"language":"default","tag":"API Reference","count":202},{"language":"default","tag":"Platform","count":196},{"language":"default","tag":"Integrations","count":138},{"language":"default","tag":"LLM Gateway","count":118},{"language":"default","tag":"AI Deployment","count":109},{"language":"default","tag":"Integrate Models","count":51},{"language":"default","tag":"Guardrails and Security","count":48},{"language":"default","tag":"Observability","count":47},{"language":"default","tag":"MCP Servers","count":44}]},"assistantConfig":{"enableAskAiButton":true,"enableAskAiSearchBar":true,"deflection":{"enabled":true,"email":"[email protected]","showHelpButton":true}},"basePath":"","hasChatPermissions":true,"hasWebMcpPermissions":true,"toggles":[{"name":"kill-isr","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-rbac-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-mintlify-slack","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"agent-integrations-settings","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"rbac-expansion","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-pr-grouping","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"custom-prefix","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"analytics-v7","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"user-flows-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"no-publish-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-onboarding","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving-live","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"stripTemperatureMiddleware","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"support-platform-pylon-zendesk","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"snapshot-serving","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"instant-rollback","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications-server","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"granular-notifications","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automations-fable","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fixed-ai-action-pricing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"site-roles","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"automation-resolve-merge-conflict","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"scoped-publishing","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"mintql","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"page-level-analytics","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-version-history-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"billing-page-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"full-page-search","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workspace-projects","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"assistant-v5","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"fork-pr-previews","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"revalidation-jwt","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-sso-scim","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"hobby-abuse-scan","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"dashboard-inbox","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-signup","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-dual-write","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"workos-enabled","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"approve-review-flow","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"editor-metadata","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"sso-setup-deadline-banner","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"turbopuffer-serve","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"live-agents","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"starter-pricing","enabled":true,"variant":{"name":"control","enabled":true,"payload":{"type":"string","value":"control"},"feature_enabled":true},"impressionData":false},{"name":"llms-txt-v2","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false},{"name":"wildcard-email-auth","enabled":true,"variant":{"name":"disabled","enabled":false,"feature_enabled":true},"impressionData":false}],"googleFontsInlined":false},"customCss":[{"filePath":"style.css","content":".line-highlight{background:rgb(var(--primary-light)/.7)!important}.tfy-cat-anchor{scroll-margin-top:6rem;margin-block:1.75rem}.line-highlight:before{background:rgb(var(--primary-light)/.7)!important}.line-highlight:after{background:rgb(var(--primary-light)/.7)!important}"}]}
{"page":{"pagePath":"docs/ai-gateway/chat-completions-overview","slug":"/docs/ai-gateway/chat-completions-overview","pageMetadata":{"title":"Chat Completions API (/chat/completions)","sidebarTitle":"Overview","description":"Learn how to use TrueFoundry's unified Chat Completions API to interact with models from multiple providers through a consistent interface.","href":"/docs/ai-gateway/chat-completions-overview"},"description":{"compiledSource":"\"use strict\";\nconst {jsx: _jsx} = arguments[0];\nconst {useMDXComponents: _provideComponents} = arguments[0];\nfunction _createMdxContent(props) {\n const _components = {\n p: \"p\",\n ..._provideComponents(),\n ...props.components\n };\n return _jsx(_components.p, {\n children: \"Learn how to use TrueFoundry’s unified Chat Completions API to interact with models from multiple providers through a consistent interface.\"\n });\n}\nfunction MDXContent(props = {}) {\n const {wrapper: MDXLayout} = {\n ..._provideComponents(),\n ...props.components\n };\n return MDXLayout ? _jsx(MDXLayout, {\n ...props,\n children: _jsx(_createMdxContent, {\n ...props\n })\n }) : _createMdxContent(props);\n}\nreturn {\n default: MDXContent\n};\n","frontmatter":{},"scope":{}},"pageType":"default","mdx":{"mdxSource":{"compiledSource":"\"use strict\";const{Fragment:_Fragment,jsx:_jsx,jsxs:_jsxs}=arguments[0];const{useMDXComponents:_provideComponents}=arguments[0];function _createMdxContent(props){const _components={a:`a`,code:`code`,li:`li`,ol:`ol`,p:`p`,pre:`pre`,span:`span`,strong:`strong`,tbody:`tbody`,td:`td`,th:`th`,thead:`thead`,tr:`tr`,ul:`ul`,..._provideComponents(),...props.components},{CodeBlock,Heading,Icon,Info,Note,Table,Warning}=_components;if(!CodeBlock)_missingMdxReference(`CodeBlock`,true);if(!Heading)_missingMdxReference(`Heading`,true);if(!Icon)_missingMdxReference(`Icon`,true);if(!Info)_missingMdxReference(`Info`,true);if(!Note)_missingMdxReference(`Note`,true);if(!Table)_missingMdxReference(`Table`,true);if(!Warning)_missingMdxReference(`Warning`,true);return _jsxs(_Fragment,{children:[_jsxs(_components.p,{children:[`TrueFoundry AI Gateway provides a universal API for all supported models via the standard OpenAI `,_jsx(_components.code,{children:`/chat/completions`}),` endpoint. This unified interface allows you to seamlessly work with models from different providers through a consistent API.`]}),`\n`,_jsxs(_components.p,{children:[_jsx(_components.strong,{children:`API Reference:`}),` `,_jsx(_components.a,{href:`/docs/api-reference/chat/chat-completions`,children:_jsx(_components.code,{children:`POST /chat/completions`})})]}),`\n`,_jsx(Heading,{level:`2`,id:`provider-capabilities`,children:`Provider capabilities`}),`\n`,_jsx(_components.p,{children:`The AI Gateway maps the OpenAI Chat Completions contract to each provider. The table below summarizes feature support by provider for this endpoint.`}),`\n`,_jsxs(Info,{children:[_jsx(_components.p,{children:`Legend:`}),_jsxs(_components.ul,{children:[`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`✅`}),` Supported by provider and TrueFoundry`]}),`\n`,_jsxs(_components.li,{children:[_jsx(Icon,{icon:`circle-xmark`,iconType:`regular`,color:`red`}),` Provided by provider, but not by TrueFoundry`]}),`\n`,_jsxs(_components.li,{children:[_jsx(Icon,{icon:`circle-minus`,iconType:`regular`}),` Provider does not support this feature`]}),`\n`]})]}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Provider`}),_jsx(_components.th,{children:`Stream`}),_jsx(_components.th,{children:`Non Stream`}),_jsx(_components.th,{children:`Tools`}),_jsx(_components.th,{children:`JSON Mode`}),_jsx(_components.th,{children:`Schema Mode`}),_jsx(_components.th,{children:`Prompt Caching`}),_jsx(_components.th,{children:`Reasoning`}),_jsx(_components.th,{children:`Structured Output`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`OpenAI`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Azure OpenAI`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Anthropic`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Bedrock`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Vertex`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Cohere`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Gemini`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Groq`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`AI21`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Cerebras`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Wafer`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Fireworks AI`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`SambaNova`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Perplexity-AI`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Together-AI`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`xAI`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`DeepInfra`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:`✅`}),_jsx(_components.td,{children:_jsx(Icon,{icon:`circle-minus`,iconType:`regular`})})]})]})]}),`\n`,_jsx(Heading,{level:`2`,id:`contents`,children:`Contents`}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Section`}),_jsx(_components.th,{children:`Description`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`/docs/ai-gateway/chat-completions-overview`,children:`Overview`})}),_jsx(_components.td,{children:`Provider capabilities, getting started, and input controls`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`/docs/ai-gateway/chat-completions-multimodal`,children:`Multimodal`})}),_jsx(_components.td,{children:`Images, audio, video, PDFs, and vision models`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`/docs/ai-gateway/chat-completions-tools`,children:`Tools \u0026 signatures`})}),_jsx(_components.td,{children:`Function calling and thought signatures`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`/docs/ai-gateway/chat-completions-structured-outputs`,children:`Structured outputs`})}),_jsx(_components.td,{children:`JSON mode, JSON schema, and Pydantic`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`/docs/ai-gateway/chat-completions-advanced`,children:`Caching \u0026 reasoning`})}),_jsx(_components.td,{children:`Prompt caching and reasoning models`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:_jsx(_components.a,{href:`/docs/ai-gateway/chat-completions-extended-thinking`,children:`Extended thinking`})}),_jsx(_components.td,{children:`Thinking blocks, multi-turn reasoning, grounding`})]})]})]}),`\n`,_jsx(Heading,{level:`2`,id:`getting-started`,children:`Getting Started`}),`\n`,_jsx(_components.p,{children:`You can use the standard OpenAI client to send requests to the AI Gateway:`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`13`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`13`,children:[_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#C586C0`,\"--shiki-dark\":`#C586C0`},children:`from`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` openai `}),_jsx(_components.span,{style:{color:`#C586C0`,\"--shiki-dark\":`#C586C0`},children:`import`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` OpenAI`})]}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`client = OpenAI(`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` api_key`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"your_truefoundry_api_key\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` base_url`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#569CD6`,\"--shiki-dark\":`#569CD6`},children:`{GATEWAY_BASE_URL}`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`)`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`response = client.chat.completions.create(`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` model`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"openai-main/gpt-4o-mini\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#6A9955`,\"--shiki-dark\":`#6A9955`},children:`# this is the truefoundry model id`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` messages`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=[{`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"user\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"Hello, how are you?\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`}]`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`)`})}),`\n`,_jsx(_components.span,{className:`line`}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#DCDCAA`,\"--shiki-dark\":`#DCDCAA`},children:`print`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`(response.choices[`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`0`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`].message.content)`})]}),`\n`]})})}),`\n`,_jsx(Heading,{level:`3`,id:`configuration`,children:`Configuration`}),`\n`,_jsx(_components.p,{children:`You will need to configure the following:`}),`\n`,_jsxs(_components.ol,{children:[`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`base_url`}),`: The base URL of the TrueFoundry AI Gateway`]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`api_key`}),`: API key generated from `,_jsx(_components.a,{href:`/docs/ai-gateway/authentication`,children:`Personal Access Tokens`})]}),`\n`,_jsxs(_components.li,{children:[_jsx(_components.strong,{children:`model`}),`: TrueFoundry model ID in the format `,_jsx(_components.code,{children:`provider_account/model_name`}),` (available in the LLM playground UI)`]}),`\n`]}),`\n`,_jsxs(_components.p,{children:[`See `,_jsx(_components.a,{href:`/docs/ai-gateway/quick-start#gateway-base-url`,children:`Integrate with code`}),` for instructions on obtaining these values.`]}),`\n`,_jsxs(_components.p,{children:[`For using native provider SDKs (OpenAI, Google Gen AI, Anthropic, boto3), see `,_jsx(_components.a,{href:`/docs/ai-gateway/native-sdk-support`,children:`Native SDK Support`}),`.`]}),`\n`,_jsx(Heading,{level:`2`,id:`input-controls`,children:`Input Controls`}),`\n`,_jsx(Heading,{level:`3`,id:`system-prompts`,children:`System Prompts`}),`\n`,_jsx(_components.p,{children:`System prompts set the behavior and context for the model by defining the assistant’s role, tone, and constraints:`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`7`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`7`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`response = client.chat.completions.create(`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` model`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"openai-main/gpt-4o-mini\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` messages`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=[`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` {`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"system\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"You are a helpful assistant that specializes in Python programming.\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`},`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` {`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"user\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"How do I write a function to calculate factorial?\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`}`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:` ]`})}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`)`})}),`\n`]})})}),`\n`,_jsx(Heading,{level:`3`,id:`request-parameters`,children:`Request Parameters`}),`\n`,_jsx(_components.p,{children:`Fine-tune model behavior with these common parameters:`}),`\n`,_jsx(CodeBlock,{filename:``,numberOfLines:`11`,language:`python`,children:_jsx(_components.pre,{className:`shiki shiki-themes dark-plus dark-plus`,style:{backgroundColor:`#0B0C0E`,\"--shiki-dark-bg\":`#0B0C0E`,color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},language:`python`,children:_jsxs(_components.code,{language:`python`,numberOfLines:`11`,children:[_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`response = client.chat.completions.create(`})}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` model`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"openai-main/gpt-4o-mini\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`,`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` messages`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=[{`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"role\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"user\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"content\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`: `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"Hello, how are you?\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`}],`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` temperature`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`0.7`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#6A9955`,\"--shiki-dark\":`#6A9955`},children:`# Controls randomness (0.0 to 1.0)`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` max_tokens`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`100`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#6A9955`,\"--shiki-dark\":`#6A9955`},children:`# Maximum tokens to generate`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` verbosity`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"high\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#6A9955`,\"--shiki-dark\":`#6A9955`},children:`# Constrains verbosity: low, medium, high`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` top_p`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`0.9`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#6A9955`,\"--shiki-dark\":`#6A9955`},children:`# Nucleus sampling parameter`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` frequency_penalty`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`0.0`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#6A9955`,\"--shiki-dark\":`#6A9955`},children:`# Reduces repetition`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` presence_penalty`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=`}),_jsx(_components.span,{style:{color:`#B5CEA8`,\"--shiki-dark\":`#B5CEA8`},children:`0.0`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#6A9955`,\"--shiki-dark\":`#6A9955`},children:`# Encourages new topics`})]}),`\n`,_jsxs(_components.span,{className:`line`,children:[_jsx(_components.span,{style:{color:`#9CDCFE`,\"--shiki-dark\":`#9CDCFE`},children:` stop`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`=[`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#D7BA7D`,\"--shiki-dark\":`#D7BA7D`},children:`\\\\n`}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`, `}),_jsx(_components.span,{style:{color:`#CE9178`,\"--shiki-dark\":`#CE9178`},children:`\"Human:\"`}),_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`] `}),_jsx(_components.span,{style:{color:`#6A9955`,\"--shiki-dark\":`#6A9955`},children:`# Stop sequences`})]}),`\n`,_jsx(_components.span,{className:`line`,children:_jsx(_components.span,{style:{color:`#D4D4D4`,\"--shiki-dark\":`#D4D4D4`},children:`)`})}),`\n`]})})}),`\n`,_jsx(Note,{children:_jsxs(_components.p,{children:[`Some models don’t support all parameters. For example, temperature is not supported by `,_jsx(_components.code,{children:`o`}),` series models like o3-mini.`]})}),`\n`,_jsxs(Heading,{level:`4`,id:`max_tokens-defaults-per-provider`,children:[_jsx(_components.code,{children:`max_tokens`}),` defaults per provider`]}),`\n`,_jsxs(_components.p,{children:[_jsx(_components.code,{children:`max_tokens`}),` is optional in the OpenAI-compatible API, but what happens when you omit it depends on the provider integration. Anthropic’s Messages API `,_jsx(_components.strong,{children:`requires`}),` `,_jsx(_components.code,{children:`max_tokens`}),`, so for integrations that speak that format the AI Gateway supplies a default rather than letting the request fail.`]}),`\n`,_jsxs(Table,{children:[_jsx(_components.thead,{children:_jsxs(_components.tr,{children:[_jsx(_components.th,{children:`Provider integration`}),_jsxs(_components.th,{children:[`If you omit `,_jsx(_components.code,{children:`max_tokens`})]}),_jsx(_components.th,{children:`Effective limit`})]})}),_jsxs(_components.tbody,{children:[_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`Anthropic`}),_jsx(_components.td,{children:`Gateway sends a default`}),_jsx(_components.td,{children:`2048 tokens`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`AWS Bedrock Mantle (Claude models)`}),_jsx(_components.td,{children:`Gateway sends a default`}),_jsx(_components.td,{children:`2048 tokens`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`AWS Bedrock (Converse)`}),_jsx(_components.td,{children:`Gateway sends nothing; Bedrock applies its own per-model default`}),_jsx(_components.td,{children:`Model-dependent — 4096 for Claude Opus and Sonnet`})]}),_jsxs(_components.tr,{children:[_jsx(_components.td,{children:`All other providers`}),_jsx(_components.td,{children:`Gateway sends nothing; the provider’s own default applies`}),_jsx(_components.td,{children:`Provider-dependent`})]})]})]}),`\n`,_jsx(Warning,{children:_jsxs(_components.p,{children:[`Always set `,_jsx(_components.code,{children:`max_tokens`}),` explicitly for long-form generations. If you rely on the default, a response can be truncated mid-sentence with `,_jsx(_components.code,{children:`finish_reason: \"length\"`}),` and no error — most visibly on Claude Opus and Sonnet via AWS Bedrock, where Bedrock caps output at 4096 tokens despite the models supporting far more.`]})}),`\n`,_jsx(Heading,{level:`4`,id:`service-tier`,children:`Service tier`}),`\n`,_jsxs(_components.p,{children:[`Set `,_jsx(_components.code,{children:`service_tier`}),` to `,_jsx(_components.code,{children:`flex`}),`, `,_jsx(_components.code,{children:`priority`}),`, `,_jsx(_components.code,{children:`fast`}),`, or `,_jsx(_components.code,{children:`ultrafast`}),` when the provider and model support it. The AI Gateway bills the `,_jsx(_components.strong,{children:`served`}),` tier from the public catalog overlay, not the value you requested. See `,_jsx(_components.a,{href:`/docs/ai-gateway/cost-tracking#service-tiers`,children:`Service tiers`}),`.`]})]})}function MDXContent(props={}){const{wrapper:MDXLayout}={..._provideComponents(),...props.components};return MDXLayout?_jsx(MDXLayout,{...props,children:_jsx(_createMdxContent,{...props})}):_createMdxContent(props)}return{default:MDXContent};function _missingMdxReference(id,component){throw new Error(`Expected `+(component?`component`:`object`)+\" `\"+id+\"` to be defined: you likely forgot to import, pass, or provide it.\")}","frontmatter":{},"scope":{"codeStyling":"dark","config":{},"pageMetadata":{"title":"Chat Completions API (/chat/completions)","sidebarTitle":"Overview","description":"Learn how to use TrueFoundry's unified Chat Completions API to interact with models from multiple providers through a consistent interface.","href":"/docs/ai-gateway/chat-completions-overview"}}},"mdxSourceWithNoJs":null},"mdxExtracts":{"tableOfContents":[{"title":"Provider capabilities","slug":"provider-capabilities","depth":2,"children":[]},{"title":"Contents","slug":"contents","depth":2,"children":[]},{"title":"Getting Started","slug":"getting-started","depth":2,"children":[{"title":"Configuration","slug":"configuration","depth":3,"children":[]}]},{"title":"Input Controls","slug":"input-controls","depth":2,"children":[{"title":"System Prompts","slug":"system-prompts","depth":3,"children":[]},{"title":"Request Parameters","slug":"request-parameters","depth":3,"children":[]},{"title":"max_tokens defaults per provider","slug":"max_tokens-defaults-per-provider","depth":4,"children":[]},{"title":"Service tier","slug":"service-tier","depth":4,"children":[]}]}],"codeExamples":{}},"jsLinks":["/_mintlify/custom/truefoundry/9e0085f7ed271180/scripts.js"],"subdomain":"truefoundry","siteName":"TrueFoundry Docs","title":"Chat Completions API (/chat/completions) - TrueFoundry Docs","navMaps":{"firstHrefInVersion":{},"firstHrefInLanguage":{}},"lastModified":"2026-11-13T07:27:51.532Z","originalFileLocation":"docs/ai-gateway/chat-completions-overview.mdx"},"headHtml":"\u003cscript data-mint-head>(function(){var d=document.documentElement,p=null;if(!p){try{p=localStorage.getItem(\"isDarkMode\")}catch(e){}if(p==='true')p='dark';else if(p==='false')p='light';if(p!=='light'\u0026\u0026p!=='dark'\u0026\u0026p!=='system')p=\"light\"}var r=p==='system'?(matchMedia(\"(prefers-color-scheme: dark)\").matches?'dark':'light'):p;d.classList.remove(r==='dark'?'light':'dark');d.classList.add(r);d.setAttribute(\"data-theme-preference\",p)})();\u003c/script>\u003cmeta data-mint-head name=\"description\" content=\"Learn how to use TrueFoundry\u0026#x27;s unified Chat Completions API to interact with models from multiple providers through a consistent interface.\">\u003cmeta data-mint-head name=\"application-name\" content=\"TrueFoundry Docs\">\u003cmeta data-mint-head name=\"generator\" content=\"Mintlify\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon/android-chrome-192x192.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"apple-touch-icon\" type=\"image/png\" sizes=\"180x180\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon/apple-touch-icon.png\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon/favicon-16x16.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon/favicon-32x32.png\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon/favicon.ico\" media=\"(prefers-color-scheme: light)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"192x192\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon-dark/android-chrome-192x192.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"16x16\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon-dark/favicon-16x16.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"icon\" type=\"image/png\" sizes=\"32x32\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon-dark/favicon-32x32.png\" media=\"(prefers-color-scheme: dark)\">\u003clink data-mint-head rel=\"shortcut icon\" type=\"image/x-icon\" href=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon-dark/favicon.ico\" media=\"(prefers-color-scheme: dark)\">\u003cmeta data-mint-head property=\"og:title\" content=\"Chat Completions API (/chat/completions) - TrueFoundry Docs\">\u003cmeta data-mint-head property=\"og:description\" content=\"TrueFoundry offers an enterprise-grade AI Gateway combining LLM, MCP, and Agent Gateways—empowering businesses to connect, monitor, and govern agentic AI applications across providers from a unified control plane.\">\u003cmeta data-mint-head property=\"og:site_name\" content=\"TrueFoundry Docs\">\u003cmeta data-mint-head property=\"og:url\" content=\"https://www.truefoundry.com/docs/ai-gateway/chat-completions-overview\">\u003cmeta data-mint-head property=\"og:type\" content=\"website\">\u003cmeta data-mint-head property=\"og:image\" content=\"https://assets.production.truefoundry.com/logo.svg\">\u003cmeta data-mint-head property=\"og:image:width\" content=\"1200\">\u003cmeta data-mint-head property=\"og:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"twitter:card\" content=\"summary_large_image\">\u003cmeta data-mint-head name=\"twitter:title\" content=\"Chat Completions API (/chat/completions) - TrueFoundry Docs\">\u003cmeta data-mint-head name=\"twitter:description\" content=\"TrueFoundry offers an enterprise-grade AI Gateway combining LLM, MCP, and Agent Gateways—empowering businesses to connect, monitor, and govern agentic AI applications across providers from a unified control plane.\">\u003cmeta data-mint-head name=\"twitter:image\" content=\"https://assets.production.truefoundry.com/logo.svg\">\u003cmeta data-mint-head name=\"twitter:image:width\" content=\"1200\">\u003cmeta data-mint-head name=\"twitter:image:height\" content=\"630\">\u003cmeta data-mint-head name=\"msapplication-config\" content=\"/mintlify-assets/_mintlify/favicons/truefoundry/DMuq4kzeTciVEGUc/_generated/favicon/browserconfig.xml\">\u003cmeta data-mint-head name=\"apple-mobile-web-app-title\" content=\"TrueFoundry Docs\">\u003cmeta data-mint-head name=\"msapplication-TileColor\" content=\"#4f46e5\">\u003clink data-mint-head rel=\"canonical\" href=\"https://www.truefoundry.com/docs/ai-gateway/chat-completions-overview\">\u003clink data-mint-head rel=\"alternate\" type=\"application/xml\" href=\"/sitemap.xml\">\u003clink data-mint-head rel=\"alternate\" type=\"text/markdown\" href=\"/docs/ai-gateway/chat-completions-overview.md\">\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@graph\":[{\"@type\":\"Organization\",\"@id\":\"https://www.truefoundry.com/#organization\",\"name\":\"TrueFoundry Docs\",\"url\":\"https://www.truefoundry.com\",\"logo\":{\"@type\":\"ImageObject\",\"url\":\"https://mintcdn.com/truefoundry/e2uUimEwX30qkWxc/logo/light.svg?fit=max\u0026auto=format\u0026n=e2uUimEwX30qkWxc\u0026q=85\u0026s=6068d83664aeda0239bec2422657dd20\"}},{\"@type\":\"WebSite\",\"@id\":\"https://www.truefoundry.com#website\",\"name\":\"TrueFoundry Docs\",\"url\":\"https://www.truefoundry.com\",\"publisher\":{\"@id\":\"https://www.truefoundry.com/#organization\"}},{\"@type\":\"WebPage\",\"@id\":\"https://www.truefoundry.com/docs/ai-gateway/chat-completions-overview#webpage\",\"url\":\"https://www.truefoundry.com/docs/ai-gateway/chat-completions-overview\",\"name\":\"Chat Completions API (/chat/completions)\",\"description\":\"Learn how to use TrueFoundry's unified Chat Completions API to interact with models from multiple providers through a consistent interface.\",\"dateModified\":\"2026-11-13T07:27:51.532Z\",\"isPartOf\":{\"@id\":\"https://www.truefoundry.com#website\"},\"breadcrumb\":{\"@id\":\"https://www.truefoundry.com/docs/ai-gateway/chat-completions-overview#breadcrumb\"}},{\"@type\":\"BreadcrumbList\",\"@id\":\"https://www.truefoundry.com/docs/ai-gateway/chat-completions-overview#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"LLM Gateway\",\"item\":\"https://www.truefoundry.com/docs/ai-gateway/truefoundry-managed\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Chat Completions API (/chat/completions)\",\"item\":\"https://www.truefoundry.com/docs/ai-gateway/chat-completions-overview\"}]},{\"@type\":[\"Article\",\"TechArticle\"],\"@id\":\"https://www.truefoundry.com/docs/ai-gateway/chat-completions-overview#article\",\"headline\":\"Chat Completions API (/chat/completions)\",\"name\":\"Chat Completions API (/chat/completions)\",\"description\":\"Learn how to use TrueFoundry's unified Chat Completions API to interact with models from multiple providers through a consistent interface.\",\"url\":\"https://www.truefoundry.com/docs/ai-gateway/chat-completions-overview\",\"mainEntityOfPage\":{\"@id\":\"https://www.truefoundry.com/docs/ai-gateway/chat-completions-overview#webpage\"},\"image\":\"https://assets.production.truefoundry.com/logo.svg\",\"dateModified\":\"2026-11-13T07:27:51.532Z\",\"publisher\":{\"@id\":\"https://www.truefoundry.com/#organization\"},\"isPartOf\":{\"@id\":\"https://www.truefoundry.com#website\"}}]}\u003c/script>\u003cscript data-mint-head type=\"application/ld+json\">{\"@context\":\"https://schema.org\",\"@type\":\"WebSite\",\"name\":\"TrueFoundry Docs\",\"publisher\":{\"@type\":\"Organization\",\"name\":\"TrueFoundry Docs\"},\"provider\":{\"@type\":\"Organization\",\"name\":\"Mintlify\",\"url\":\"https://mintlify.com\"}}\u003c/script>","agentPointerHref":"/docs/ai-gateway/chat-completions-overview.md","siteName":"TrueFoundry Docs","nav":{"global":null,"tabs":[{"tab":"AI Gateway","groups":[{"group":"Get Started","pages":[{"title":"Introduction to AI Gateway","sidebarTitle":"AI Gateway Overview","href":"/docs/ai-gateway/intro-to-llm-gateway"},{"title":"Quick Start Guide: Setup \u0026 Integration","sidebarTitle":"Quick Start","href":"/docs/ai-gateway/quick-start"}]},{"group":"LLM Gateway","pages":[{"group":"Integrate Models","pages":[{"title":"TrueFoundry Managed Models","sidebarTitle":"TrueFoundry Managed Models","href":"/docs/ai-gateway/truefoundry-managed"},{"title":"Providers","sidebarTitle":"Supported Providers","href":"/docs/ai-gateway/supported-providers"},{"title":"OpenAI","sidebarTitle":"OpenAI Models","href":"/docs/ai-gateway/openai"},{"title":"Anthropic","sidebarTitle":"Anthropic","href":"/docs/ai-gateway/anthropic"},{"title":"AWS Claude Platform","sidebarTitle":"AWS Claude Platform","href":"/docs/ai-gateway/aws-claude-platform"},{"title":"AWS Bedrock","sidebarTitle":"AWS Bedrock","href":"/docs/ai-gateway/aws-bedrock"},{"title":"AWS Bedrock Mantle","sidebarTitle":"AWS Bedrock Mantle","href":"/docs/ai-gateway/aws-bedrock-mantle"},{"title":"AWS Sagemaker","sidebarTitle":"AWS Sagemaker","href":"/docs/ai-gateway/aws-sagemaker"},{"title":"Google Vertex","sidebarTitle":"Google Vertex AI","href":"/docs/ai-gateway/google-vertex"},{"title":"Google Gemini","sidebarTitle":"Google Gemini","href":"/docs/ai-gateway/google-gemini"},{"group":"Azure Models","pages":[{"title":"Choosing an Azure model integration","sidebarTitle":"Overview","href":"/docs/ai-gateway/azure-models-guide"},{"title":"Microsoft Foundry","sidebarTitle":"Microsoft Foundry","tag":"NEW","href":"/docs/ai-gateway/microsoft-foundry"},{"title":"Migrate to Microsoft Foundry","sidebarTitle":"Migrate to Microsoft Foundry","href":"/docs/ai-gateway/migrate-to-microsoft-foundry"},{"title":"Azure OpenAI","sidebarTitle":"Azure OpenAI","tag":"LEGACY","href":"/docs/ai-gateway/azure-openai"},{"title":"Azure Foundry","sidebarTitle":"Azure AI Foundry","tag":"LEGACY","href":"/docs/ai-gateway/azure-ai-foundry"},{"group":"Authentication","pages":[{"title":"Azure Entra ID Certificate Based Authentication","sidebarTitle":"Azure Certificate Auth","href":"/docs/ai-gateway/azure-entra-certificate-auth"},{"title":"Azure Entra ID Client Secret Based Authentication","sidebarTitle":"Azure Client Secret Auth","href":"/docs/ai-gateway/azure-entra-client-secret-auth"},{"title":"Azure Workload Identity Federation Authentication","sidebarTitle":"Azure Workload Identity","href":"/docs/ai-gateway/azure-entra-wif-auth"}]}]},{"title":"Databricks","sidebarTitle":"Databricks Models","href":"/docs/ai-gateway/databricks-models"},{"title":"Cohere","sidebarTitle":"Cohere Models","href":"/docs/ai-gateway/cohere"},{"title":"AI21","sidebarTitle":"AI21","href":"/docs/ai-gateway/ai21"},{"title":"Deepinfra","sidebarTitle":"Deepinfra Models","href":"/docs/ai-gateway/deepinfra"},{"title":"Groq","sidebarTitle":"Groq Models","href":"/docs/ai-gateway/groq"},{"title":"ElevenLabs","sidebarTitle":"ElevenLabs","href":"/docs/ai-gateway/elevenlabs"},{"title":"Deepgram","sidebarTitle":"Deepgram","href":"/docs/ai-gateway/deepgram"},{"title":"Cartesia","sidebarTitle":"Cartesia","href":"/docs/ai-gateway/cartesia"},{"title":"Smallest AI","sidebarTitle":"Smallest AI","href":"/docs/ai-gateway/smallest-ai"},{"title":"Mistral AI","sidebarTitle":"Mistral AI Models","href":"/docs/ai-gateway/mistral"},{"title":"Cloudera","sidebarTitle":"Cloudera Models","href":"/docs/ai-gateway/cloudera"},{"title":"OpenRouter","sidebarTitle":"OpenRouter","href":"/docs/ai-gateway/openrouter"},{"title":"Perplexity AI","sidebarTitle":"Perplexity Models","href":"/docs/ai-gateway/perplexity-ai"},{"title":"Together AI","sidebarTitle":"Together AI","href":"/docs/ai-gateway/together-ai"},{"title":"xAI","sidebarTitle":"xAI Models","href":"/docs/ai-gateway/xai"},{"title":"SambaNova","sidebarTitle":"SambaNova","href":"/docs/ai-gateway/sambanova"},{"title":"Cerebras","sidebarTitle":"Cerebras","href":"/docs/ai-gateway/cerebras"},{"title":"Wafer","sidebarTitle":"Wafer","href":"/docs/ai-gateway/wafer"},{"title":"Fireworks AI Model Provider for TrueFoundry AI Gateway","sidebarTitle":"Fireworks AI","href":"/docs/ai-gateway/fireworks-ai"},{"title":"Snowflake Cortex","sidebarTitle":"Snowflake Cortex","href":"/docs/ai-gateway/snowflake-cortex"},{"title":"Baseten","sidebarTitle":"Baseten","href":"/docs/ai-gateway/baseten"},{"title":"TypeSafe","sidebarTitle":"TypeSafe","href":"/docs/ai-gateway/typesafe"},{"group":"Custom Endpoints","root":{"title":"Custom Endpoints","sidebarTitle":"Custom Endpoints","href":"/docs/ai-gateway/custom-endpoints"},"expanded":false,"pages":[{"group":"Examples","pages":[{"title":"CoreWeave","sidebarTitle":"CoreWeave Models","href":"/docs/ai-gateway/coreweave-models"},{"title":"PipeShift","sidebarTitle":"PipeShift","href":"/docs/ai-gateway/pipeshift"},{"title":"SiliconFlow","sidebarTitle":"SiliconFlow","href":"/docs/ai-gateway/siliconflow"},{"title":"io.net","sidebarTitle":"io.net","href":"/docs/ai-gateway/io-net"},{"title":"Resemble AI","sidebarTitle":"Resemble AI","href":"/docs/ai-gateway/resemble-ai"},{"title":"Ringg.AI","sidebarTitle":"Ringg.AI","href":"/docs/ai-gateway/ringg-ai"},{"title":"Crusoe","sidebarTitle":"Crusoe","href":"/docs/ai-gateway/crusoe"},{"title":"Jev (TypeSafe)","sidebarTitle":"Jev","href":"/docs/ai-gateway/jev"},{"title":"SCX.ai","sidebarTitle":"SCX.ai","href":"/docs/ai-gateway/scx-ai"}]}]},{"title":"Self-Hosted Models","sidebarTitle":"Self-Hosted Models","description":"Learn how to add and configure self-hosted or external AI models to TrueFoundry AI Gateway for centralized management and inference.","href":"/docs/ai-gateway/self-hosted-models"}]},{"group":"Supported APIs","pages":[{"group":"Chat Completions","root":{"title":"Chat Completions API (/chat/completions)","sidebarTitle":"Overview","description":"Learn how to use TrueFoundry's unified Chat Completions API to interact with models from multiple providers through a consistent interface.","href":"/docs/ai-gateway/chat-completions-overview"},"expanded":false,"directory":"accordion","pages":[{"title":"Chat Completions API (/chat/completions)","sidebarTitle":"Overview","description":"Learn how to use TrueFoundry's unified Chat Completions API to interact with models from multiple providers through a consistent interface.","href":"/docs/ai-gateway/chat-completions-overview"},{"title":"Chat Completions: Multimodal","sidebarTitle":"Multimodal","description":"Send images, audio, video, and PDFs with the Chat Completions API, plus vision model support.","href":"/docs/ai-gateway/chat-completions-multimodal"},{"title":"Chat Completions: Tools \u0026 Signatures","sidebarTitle":"Tools","description":"Function calling, tool workflows, and thought signatures for Chat Completions.","href":"/docs/ai-gateway/chat-completions-tools"},{"title":"Chat Completions: Structured Outputs","sidebarTitle":"Structured Outputs","description":"JSON mode, JSON schema, Pydantic integration, and provider support for structured responses.","href":"/docs/ai-gateway/chat-completions-structured-outputs"},{"title":"Chat Completions: Caching \u0026 Reasoning","sidebarTitle":"Caching \u0026 Reasoning","description":"Prompt caching and reasoning models for the Chat Completions API.","href":"/docs/ai-gateway/chat-completions-advanced"},{"title":"Chat Completions: Extended Thinking","sidebarTitle":"Extended Thinking","description":"Thinking blocks, signatures, multi-turn reasoning, and Google Search grounding.","href":"/docs/ai-gateway/chat-completions-extended-thinking"}]},{"title":"Embeddings API (/embeddings)","sidebarTitle":"Embeddings","href":"/docs/ai-gateway/embed"},{"title":"Batch API (/batches)","sidebarTitle":"Batch Predictions","href":"/docs/ai-gateway/batch-predictions-with-truefoundry-llm-gateway"},{"title":"Finetune API (/fine_tuning)","sidebarTitle":"Finetuning","href":"/docs/ai-gateway/finetune"},{"title":"Responses API (/responses)","sidebarTitle":"Responses API","href":"/docs/ai-gateway/responses-api"},{"title":"Image Generation API (/images/generations)","sidebarTitle":"Image Generation","href":"/docs/ai-gateway/image-generation"},{"title":"Image Edit API (/images/edits)","sidebarTitle":"Image Edit","href":"/docs/ai-gateway/image-edit"},{"title":"Image Variation API (/images/variations)","sidebarTitle":"Image Variation","href":"/docs/ai-gateway/image-variation"},{"title":"Video Generation API (/videos-proxy)","sidebarTitle":"Video Generation","href":"/docs/ai-gateway/video-generation"},{"title":"Text to Speech API","sidebarTitle":"Text to Speech","href":"/docs/ai-gateway/text-to-speech"},{"title":"Audio Translation API","sidebarTitle":"Audio Translation","href":"/docs/ai-gateway/audio-translation"},{"title":"Speech to Text API","sidebarTitle":"Speech to Text","href":"/docs/ai-gateway/audio-transcription"},{"group":"Live / Realtime API","root":{"title":"Live / Realtime API (/live)","sidebarTitle":"Overview","href":"/docs/ai-gateway/live-api"},"expanded":false,"directory":"accordion","pages":[{"title":"Live / Realtime API (/live)","sidebarTitle":"Overview","href":"/docs/ai-gateway/live-api"},{"title":"Live API: Code Snippets","sidebarTitle":"Code Snippets","href":"/docs/ai-gateway/live-api-code-snippets"},{"title":"Live API: Tool Calling","sidebarTitle":"Tool Calling","href":"/docs/ai-gateway/live-api-tool-calling"},{"title":"OpenAI Realtime with WebRTC","sidebarTitle":"WebRTC (OpenAI)","href":"/docs/ai-gateway/live-api-webrtc"},{"title":"Amazon Nova Sonic (Bedrock)","sidebarTitle":"Nova Sonic (Bedrock)","href":"/docs/ai-gateway/live-api-nova-sonic"}]},{"title":"Files API (/files)","sidebarTitle":"Files API","href":"/docs/ai-gateway/file-endpoints"},{"title":"Rerank API (/rerank)","sidebarTitle":"Rerank API","href":"/docs/ai-gateway/rerank"},{"title":"Moderation API (/moderations)","sidebarTitle":"Moderation API","href":"/docs/ai-gateway/moderation"},{"title":"Compaction API (/responses/compact)","sidebarTitle":"Compaction API","href":"/docs/ai-gateway/compaction"},{"title":"Messages API (/messages)","sidebarTitle":"Messages API","href":"/docs/ai-gateway/messages-overview"},{"title":"Decisions API (/v1/decisions, /v1/systemone)","sidebarTitle":"Decisions API","href":"/docs/ai-gateway/decisions"},{"title":"Proxy API (/proxy)","sidebarTitle":"Proxy API","href":"/docs/ai-gateway/proxy-api"}]},{"group":"Making LLM Requests","pages":[{"title":"Making Your First Request","sidebarTitle":"Making Your First Request","href":"/docs/ai-gateway/making-llm-requests-via-gateway"},{"title":"Authentication","sidebarTitle":"Authentication","href":"/docs/ai-gateway/authentication"},{"title":"Custom Metadata and Headers","sidebarTitle":"Metadata \u0026 Headers","href":"/docs/ai-gateway/request-headers"},{"title":"Native SDK Support","sidebarTitle":"Native SDK Support","href":"/docs/ai-gateway/native-sdk-support"},{"title":"Model Discovery","sidebarTitle":"Model Discovery","href":"/docs/ai-gateway/model-discovery"}]},{"group":"Set Up Your IDEs","pages":[{"title":"Set Up Your IDEs","sidebarTitle":"Overview","href":"/docs/ai-gateway/personal-setup"},{"title":"Claude Code","href":"/docs/ai-gateway/ide-setup/claude-code"},{"title":"Codex (CLI \u0026 Desktop)","href":"/docs/ai-gateway/ide-setup/codex"},{"title":"Claude Desktop","href":"/docs/ai-gateway/ide-setup/claude-desktop"},{"title":"Cursor","href":"/docs/ai-gateway/ide-setup/cursor"},{"title":"VS Code","href":"/docs/ai-gateway/ide-setup/vscode"},{"title":"GitHub Copilot CLI","href":"/docs/ai-gateway/ide-setup/copilot-cli"},{"title":"Gemini CLI","href":"/docs/ai-gateway/ide-setup/gemini-cli"},{"title":"OpenCode","href":"/docs/ai-gateway/ide-setup/opencode"},{"title":"Goose","href":"/docs/ai-gateway/ide-setup/goose"},{"title":"Qwen Code","href":"/docs/ai-gateway/ide-setup/qwen-code"},{"title":"Grok Build","href":"/docs/ai-gateway/ide-setup/grok-build"},{"title":"Pi","href":"/docs/ai-gateway/ide-setup/pi"}]},{"group":"Deploy with MDM","pages":[{"title":"Deploy coding agents with MDM","sidebarTitle":"Overview","href":"/docs/ai-gateway/mdm-setup"},{"title":"Claude Code and Claude Desktop with MDM","sidebarTitle":"Claude Code \u0026 Desktop","href":"/docs/ai-gateway/mdm/claude"},{"title":"Codex with MDM","sidebarTitle":"Codex","href":"/docs/ai-gateway/mdm/codex"},{"title":"MDM configuration reference","sidebarTitle":"Configuration reference","href":"/docs/ai-gateway/mdm/configuration-reference"}]},{"title":"TrueFoundry AI Gateway Playground","sidebarTitle":"AI Gateway Playground","href":"/docs/ai-gateway/playground-overview"},{"group":"Virtual Models","pages":[{"title":"Virtual Models","sidebarTitle":"Overview","href":"/docs/ai-gateway/virtual-model"},{"title":"Create a Virtual Model","sidebarTitle":"Create a Virtual Model","href":"/docs/ai-gateway/virtual-model-advanced"},{"title":"Auto Routing","sidebarTitle":"Auto Routing","href":"/docs/ai-gateway/auto-routing"}]},{"title":"Access Control","sidebarTitle":"Access Control","href":"/docs/ai-gateway/gateway-access-control"},{"group":"Policies","pages":[{"title":"Rate Limiting","sidebarTitle":"Rate Limiting","href":"/docs/ai-gateway/ratelimiting-v2"},{"title":"Rate Limiting (V1, deprecated)","sidebarTitle":"Rate Limiting (V1)","hidden":true,"href":"/docs/ai-gateway/ratelimiting"},{"title":"Budget Limiting","sidebarTitle":"Budget Limiting","href":"/docs/ai-gateway/budget-limiting-v2"},{"title":"Budget Usage Recovery","sidebarTitle":"Budget Usage Recovery","href":"/docs/ai-gateway/budget-usage-recovery"},{"title":"Budget Limiting (V1, deprecated)","sidebarTitle":"Budget Limiting (V1)","hidden":true,"href":"/docs/ai-gateway/budgetlimiting"},{"title":"Logging Configuration","sidebarTitle":"Logging Config","href":"/docs/ai-gateway/logging-config"}]},{"title":"Caching (Exact and Semantic)","sidebarTitle":"Caching (Exact and Semantic)","href":"/docs/ai-gateway/caching"},{"title":"Cost Tracking","sidebarTitle":"Cost Tracking","href":"/docs/ai-gateway/cost-tracking"},{"title":"Cost Savings","sidebarTitle":"Cost Savings","href":"/docs/ai-gateway/cost-savings"},{"group":"Extras","pages":[{"title":"Session Management","sidebarTitle":"Session Management","href":"/docs/ai-gateway/session-management"},{"title":"Log Feedback on Requests","sidebarTitle":"Log Feedback","href":"/docs/ai-gateway/feedback-for-traces"},{"title":"First Chunk Gate for Streaming","sidebarTitle":"First Chunk Gate","href":"/docs/ai-gateway/first-chunk-gate"},{"title":"Anthropic Stream Overload Fallback","sidebarTitle":"Anthropic Stream Fallback","href":"/docs/ai-gateway/anthropic-stream-overload-fallback"}]}]},{"group":"MCP Registry and Gateway","pages":[{"title":"TrueFoundry MCP Gateway","sidebarTitle":"Overview","href":"/docs/ai-gateway/mcp/mcp-overview"},{"title":"MCP Server: Getting Started","sidebarTitle":"Getting Started","href":"/docs/ai-gateway/mcp/mcp-server-getting-started"},{"title":"MCP Protocol Support","sidebarTitle":"Protocol Support","href":"/docs/ai-gateway/mcp/mcp-protocol-support"},{"group":"Authentication and Security","root":{"title":"Authentication and Security","sidebarTitle":"Authentication and Security","href":"/docs/ai-gateway/mcp/mcp-gateway-auth-security"},"pages":[{"title":"Authentication and Security","sidebarTitle":"Authentication and Security","href":"/docs/ai-gateway/mcp/mcp-gateway-auth-security"},{"title":"Auth Overrides","sidebarTitle":"Auth Overrides","href":"/docs/ai-gateway/mcp/mcp-server-auth-overrides"},{"group":"Outbound auth integrations","pages":[{"title":"Okta app setup","sidebarTitle":"Okta App Setup","href":"/docs/ai-gateway/mcp/okta-app-setup"},{"title":"Microsoft Entra app setup","sidebarTitle":"Microsoft Entra App Setup","href":"/docs/ai-gateway/mcp/entra-app-setup"},{"title":"Microsoft Entra On-Behalf-Of (OBO) for MCP","sidebarTitle":"Microsoft Entra OBO App Setup","href":"/docs/ai-gateway/mcp/mcp-server-oauth-azure-obo"},{"title":"Register an AWS Bedrock AgentCore MCP Server","sidebarTitle":"AgentCore (AWS SigV4)","href":"/docs/ai-gateway/mcp/mcp-server-agentcore-sigv4"}]}]},{"title":"MCP Troubleshooting Guide","sidebarTitle":"Troubleshooting Guide","href":"/docs/ai-gateway/mcp/mcp-troubleshooting-guide"},{"title":"MCP Tool Approvals","sidebarTitle":"Tool Approvals","href":"/docs/ai-gateway/mcp/mcp-tool-approval"},{"title":"MCP Tool Management","sidebarTitle":"Tool Management","href":"/docs/ai-gateway/mcp/mcp-tool-management"},{"title":"Connect MCP Servers from Your IDE","sidebarTitle":"Connect MCP Servers from Your IDE","href":"/docs/ai-gateway/mcp/connect-mcp-from-ide"},{"title":"TrueFoundry Managed MCPs","sidebarTitle":"TrueFoundry Managed MCPs","href":"/docs/ai-gateway/mcp/tfy-managed-mcp-server"},{"title":"Virtual MCP Server","sidebarTitle":"Virtual MCP Server","href":"/docs/ai-gateway/mcp/virtual-mcp-server"},{"title":"OpenAPI to MCP Server","sidebarTitle":"OpenAPI to MCP Server","href":"/docs/ai-gateway/mcp/openapi-mcp-server"},{"title":"Hosted Stdio-based MCP Server","sidebarTitle":"Hosted Stdio-based MCP Server","href":"/docs/ai-gateway/mcp/stdio-mcp-server"},{"group":"MCP Server Guides","pages":[{"group":"Creating MCP Servers","pages":[{"title":"Create a Basic Calculator MCP Server","sidebarTitle":"Basic Calculator MCP Server","href":"/docs/ai-gateway/mcp/create-calculator-mcp-server"},{"title":"Add Okta OAuth to a Calculator MCP Server","sidebarTitle":"Okta OAuth MCP server","href":"/docs/ai-gateway/mcp/mcp-server-oauth-okta"},{"title":"Create an OAuth MCP Server with Azure Entra ID","sidebarTitle":"Azure Entra ID OAuth MCP Server","href":"/docs/ai-gateway/mcp/mcp-server-oauth-azure"}]}]}]},{"group":"Skills Registry","pages":[{"title":"AI Gateway Skills Registry","sidebarTitle":"Overview","href":"/docs/ai-gateway/skills/skills-registry"},{"title":"Create, Manage, and Use Skills","sidebarTitle":"Create, Manage and Use Skills","href":"/docs/ai-gateway/skills/getting-started"}]},{"group":"Prompt Registry","pages":[{"title":"Prompt Management","sidebarTitle":"Create, Manage and Use Prompts","href":"/docs/ai-gateway/prompt-management"}]},{"group":"Guardrails and Security","pages":[{"title":"AI Gateway Guardrails Overview","sidebarTitle":"Overview","href":"/docs/ai-gateway/guardrails-overview"},{"title":"AI Gateway Guardrails: Quick Start","sidebarTitle":"Getting Started","href":"/docs/ai-gateway/guardrails-getting-started"},{"title":"Configure Guardrail Policies","sidebarTitle":"Configure Guardrail Policies","href":"/docs/ai-gateway/guardrails-configuration"},{"title":"Guardrails Best Practices","sidebarTitle":"Best Practices","href":"/docs/ai-gateway/guardrails-best-practices"},{"group":"TrueFoundry Guardrails","root":{"title":"TrueFoundry Guardrails","sidebarTitle":"Overview","href":"/docs/ai-gateway/truefoundry-guardrails"},"pages":[{"title":"TrueFoundry Guardrails","sidebarTitle":"Overview","href":"/docs/ai-gateway/truefoundry-guardrails"},{"title":"Secrets Detection Guardrail","sidebarTitle":"Secrets Detection","href":"/docs/ai-gateway/secrets-detection"},{"title":"Code Safety Linter Guardrail","sidebarTitle":"Code Safety Linter","href":"/docs/ai-gateway/code-safety-linter"},{"title":"SQL Sanitizer Guardrail","sidebarTitle":"SQL Sanitizer","href":"/docs/ai-gateway/sql-sanitizer"},{"title":"Regex Pattern Match Guardrail","sidebarTitle":"Regex Pattern Match","href":"/docs/ai-gateway/regex-pattern-matching"},{"title":"Prompt Injection Guardrail","sidebarTitle":"Prompt Injection","href":"/docs/ai-gateway/tfy-prompt-injection"},{"title":"PII/PHI Detection Guardrail","sidebarTitle":"PII/PHI","href":"/docs/ai-gateway/tfy-pii"},{"title":"Content Moderation Guardrail","sidebarTitle":"Content Moderation","href":"/docs/ai-gateway/tfy-content-moderation"},{"title":"Metadata Validation Guardrail","sidebarTitle":"Metadata Validation","href":"/docs/ai-gateway/metadata-validation"},{"title":"Cedar Guardrails","sidebarTitle":"Cedar Guardrails","href":"/docs/ai-gateway/cedar-guardrails"},{"title":"OPA Guardrails","sidebarTitle":"OPA Guardrails","href":"/docs/ai-gateway/opa-guardrails"}]},{"group":"External Providers","pages":[{"title":"Akto Integration","sidebarTitle":"Akto","href":"/docs/ai-gateway/akto"},{"title":"Arthur AI","sidebarTitle":"Arthur AI","href":"/docs/ai-gateway/arthur-ai"},{"title":"AWS Bedrock Guardrail Integration","sidebarTitle":"AWS Bedrock Guardrail","href":"/docs/ai-gateway/bedrock-guardrails"},{"title":"Azure Content Safety Guardrail Integration","sidebarTitle":"Azure Content Safety","href":"/docs/ai-gateway/azure-content-safety"},{"title":"Azure PII Guardrail Integration","sidebarTitle":"Azure PII","href":"/docs/ai-gateway/azure-pii"},{"title":"Azure Prompt Shield Guardrail Integration","sidebarTitle":"Azure Prompt Shield","href":"/docs/ai-gateway/azure-prompt-shield"},{"title":"Bigeye","sidebarTitle":"Bigeye","href":"/docs/ai-gateway/bigeye"},{"title":"CrowdStrike AIDR Integration","sidebarTitle":"CrowdStrike AIDR","href":"/docs/ai-gateway/crowdstrike"},{"title":"Cato Networks","sidebarTitle":"Cato Networks","href":"/docs/ai-gateway/cato-networks"},{"title":"Cisco AI Defense Integration","sidebarTitle":"Cisco AI Defense","href":"/docs/ai-gateway/cisco-ai-defense"},{"title":"DeepKeep AI Firewall","sidebarTitle":"DeepKeep","href":"/docs/ai-gateway/deepkeep"},{"title":"Enkrypt AI Guardrail Integration","sidebarTitle":"Enkrypt AI","href":"/docs/ai-gateway/enkrypt-ai"},{"title":"F5 AI Security (CalypsoAI) Integration","sidebarTitle":"F5 AI Security","href":"/docs/ai-gateway/f5-calypsoai"},{"title":"Fiddler Integration","sidebarTitle":"Fiddler","href":"/docs/ai-gateway/fiddler"},{"title":"Fireworks Safety Model Guardrail for TrueFoundry AI Gateway","sidebarTitle":"Fireworks","href":"/docs/ai-gateway/fireworks"},{"title":"Google Model Armor Guardrail Integration","sidebarTitle":"Google Model Armor","href":"/docs/ai-gateway/google-model-armor"},{"title":"Gray Swan Cygnal","sidebarTitle":"Gray Swan Cygnal","href":"/docs/ai-gateway/grayswan-cygnal"},{"title":"Guardrails AI Integration","sidebarTitle":"Guardrails AI","href":"/docs/ai-gateway/guardrails-ai"},{"title":"HiddenLayer Integration","sidebarTitle":"HiddenLayer","href":"/docs/ai-gateway/hiddenlayer"},{"title":"CoreWeave Weave Guardrails Integration","sidebarTitle":"CoreWeave Guardrails","href":"/docs/ai-gateway/coreweave-weave-guardrails"},{"title":"Lasso Security","sidebarTitle":"Lasso Security","href":"/docs/ai-gateway/lasso-security"},{"title":"Noma Security","sidebarTitle":"Noma Security","href":"/docs/ai-gateway/noma-security"},{"title":"NVIDIA NeMo Guardrails Integration","sidebarTitle":"NVIDIA NeMo","href":"/docs/ai-gateway/nvidia-nemo"},{"title":"OpenAI Moderation Integration","sidebarTitle":"OpenAI Moderation","href":"/docs/ai-gateway/openai-moderations"},{"title":"Onyx Security","sidebarTitle":"Onyx Security","href":"/docs/ai-gateway/onyx-security"},{"title":"Palo Alto Prisma AIRS Integration","sidebarTitle":"Palo Alto Prisma AIRS","href":"/docs/ai-gateway/palo-alto-airs"},{"title":"Patronus AI Integration","sidebarTitle":"Patronus AI","href":"/docs/ai-gateway/patronus"},{"title":"Pillar Security Integration","sidebarTitle":"Pillar Security","href":"/docs/ai-gateway/pillar-security"},{"title":"Straiker","sidebarTitle":"Straiker","href":"/docs/ai-gateway/straiker"},{"title":"TrojAI DEFEND Integration","sidebarTitle":"TrojAI","href":"/docs/ai-gateway/trojai"},{"title":"Verra","sidebarTitle":"Verra","href":"/docs/ai-gateway/verra"},{"title":"Zscaler AI Guard guardrail for prompts and responses","sidebarTitle":"Zscaler AI Guard","href":"/docs/ai-gateway/zscaler-ai-guard"}]},{"title":"Custom Guardrail/Plugins Configuration","sidebarTitle":"Bring your own Guardrail / Plugin","href":"/docs/ai-gateway/custom-guardrails"}]},{"group":"Observability","pages":[{"title":"Metrics Dashboard","sidebarTitle":"Metrics Dashboard","href":"/docs/ai-gateway/analytics"},{"title":"Request Logging","sidebarTitle":"Request Logging","href":"/docs/ai-gateway/request-logging"},{"title":"Configure Data Access","sidebarTitle":"Configure Data Access","href":"/docs/ai-gateway/data-access"},{"title":"Configure Data Routing","sidebarTitle":"Configure Data Routing","href":"/docs/ai-gateway/data-routing"},{"group":"Export Logs and Metrics","pages":[{"title":"Export OpenTelemetry Data","sidebarTitle":"Export via OpenTelemetry","href":"/docs/ai-gateway/export-opentelemetry-data"},{"title":"Prometheus Grafana Integration","sidebarTitle":"Prometheus \u0026 Grafana","href":"/docs/ai-gateway/prometheus-grafana-integration"}]},{"group":"API Access to Metrics","expanded":false,"pages":[{"title":"Model Metrics","sidebarTitle":"Model Metrics","href":"/docs/ai-gateway/fetch-model-metrics"},{"title":"MCP Metrics","sidebarTitle":"MCP Metrics","href":"/docs/ai-gateway/fetch-mcp-metrics"},{"title":"Guardrail Metrics","sidebarTitle":"Guardrail Metrics","href":"/docs/ai-gateway/fetch-guardrail-metrics"},{"title":"Cache Metrics","sidebarTitle":"Cache Metrics","href":"/docs/ai-gateway/fetch-cache-metrics"},{"title":"Routing Metrics","sidebarTitle":"Routing Metrics","href":"/docs/ai-gateway/fetch-routing-metrics"},{"title":"Agent Metrics","sidebarTitle":"Agent Metrics","href":"/docs/ai-gateway/fetch-agent-metrics"}]},{"group":"API Access to Logs","root":{"title":"API Access to Logs","sidebarTitle":"Overview","href":"/docs/ai-gateway/fetch-request-logs"},"expanded":false,"directory":"accordion","pages":[{"title":"API Access to Logs","sidebarTitle":"Overview","href":"/docs/ai-gateway/fetch-request-logs"},{"title":"Request Logs: Trace Inspection","sidebarTitle":"Trace Inspection","href":"/docs/ai-gateway/fetch-request-logs-trace-inspection"},{"title":"Request Logs: Filtering","sidebarTitle":"Filtering","href":"/docs/ai-gateway/fetch-request-logs-filtering"},{"group":"Use Cases","expanded":false,"pages":[{"title":"Request Logs: Common Use Cases","sidebarTitle":"Basic Queries","href":"/docs/ai-gateway/fetch-request-logs-use-cases"},{"title":"Request Logs: Advanced Queries","sidebarTitle":"Advanced Queries","href":"/docs/ai-gateway/fetch-request-logs-use-cases-advanced"}]},{"group":"Span Attributes","expanded":false,"pages":[{"title":"Request Logs: Span Attributes","sidebarTitle":"TrueFoundry Attributes","href":"/docs/ai-gateway/fetch-request-logs-span-attributes"},{"title":"GenAI Span Attributes","sidebarTitle":"GenAI Attributes","href":"/docs/ai-gateway/fetch-request-logs-span-attributes-genai"}]}]}]},{"group":"Deployment","pages":[{"title":"Gateway Architecture","sidebarTitle":"Architecture","href":"/docs/ai-gateway/gateway-architecture"},{"title":"Deployment Options","sidebarTitle":"Deployment Options","href":"/docs/ai-gateway/modes-of-deployment"},{"title":"Globally Distributed SAAS Gateway","sidebarTitle":"Globally Distributed SAAS","href":"/docs/ai-gateway/globally-distributed-saas"},{"title":"Connect Private Models and MCP Servers","sidebarTitle":"Connect Private Endpoints","href":"/docs/ai-gateway/connect-private-models-mcp-servers"},{"title":"Install LLM Gateway (Standalone)","sidebarTitle":"Install AI Gateway","href":"/docs/ai-gateway/install-only-llm-gateway"}]},{"group":"Admin Guide","pages":[{"title":"Setup AI Gateway in Your Organization","sidebarTitle":"Setup AI Gateway in Org","href":"/docs/ai-gateway/setup-ai-gateway-in-org"},{"title":"Govern All AI Traffic Through the AI Gateway","sidebarTitle":"Govern All AI Traffic","href":"/docs/ai-gateway/govern-traffic-through-ai-gateway"},{"title":"Setup Dev, Staging and Prod Environments","sidebarTitle":"Setup Multiple Environments","href":"/docs/ai-gateway/setup-multiple-environments"},{"title":"Use Secret Manager in Integrations","sidebarTitle":"Use Secret Manager","href":"/docs/ai-gateway/secret-manager-in-ai-gateway"}]},{"group":"Chat","pages":[{"title":"Chat Completions","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /chat/completions","href":"/docs/api-reference/chat/chat-completions"}]},{"group":"Messages","pages":[{"title":"Messages","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /messages","href":"/docs/api-reference/messages/messages"},{"title":"Count Tokens","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /messages/count_tokens","href":"/docs/api-reference/messages/count-tokens"}]},{"group":"Embeddings","pages":[{"title":"Generate Embeddings","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /embeddings","href":"/docs/api-reference/embeddings/generate-embeddings"}]},{"group":"Rerank","pages":[{"title":"Rerank Documents","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /v2/rerank","href":"/docs/api-reference/rerank/rerank-documents"}]},{"group":"Responses","pages":[{"title":"List Model Response Input Items","deprecated":null,"version":null,"openapi":"/gateway-openapi.json get /responses/{id}/input_items","href":"/docs/api-reference/responses/list-model-response-input-items"},{"title":"Get Model Response","deprecated":null,"version":null,"openapi":"/gateway-openapi.json get /responses/{id}","href":"/docs/api-reference/responses/get-model-response"},{"title":"Delete Model Response","deprecated":null,"version":null,"openapi":"/gateway-openapi.json delete /responses/{id}","href":"/docs/api-reference/responses/delete-model-response"},{"title":"Model Responses","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /responses","href":"/docs/api-reference/responses/model-responses"},{"title":"Model Responses Compact","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /responses/compact","href":"/docs/api-reference/responses/model-responses-compact"}]},{"group":"Image","pages":[{"title":"Generate Images","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /images/generations","href":"/docs/api-reference/image/generate-images"},{"title":"Edit Images","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /images/edits","href":"/docs/api-reference/image/edit-images"},{"title":"Create Image Variation","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /images/variations","href":"/docs/api-reference/image/create-image-variation"}]},{"group":"Audio","pages":[{"title":"Transcribe Audio","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /audio/transcriptions","href":"/docs/api-reference/audio/transcribe-audio"},{"title":"Translate Audio","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /audio/translations","href":"/docs/api-reference/audio/translate-audio"},{"title":"Generate Speech","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /audio/speech","href":"/docs/api-reference/audio/generate-speech"}]},{"group":"Batch","pages":[{"title":"List Batches","deprecated":null,"version":null,"openapi":"/gateway-openapi.json get /batches","href":"/docs/api-reference/batch/list-batches"},{"title":"Create Batch","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /batches","href":"/docs/api-reference/batch/create-batch"},{"title":"Get Batch","deprecated":null,"version":null,"openapi":"/gateway-openapi.json get /batches/{id}","href":"/docs/api-reference/batch/get-batch"},{"title":"Get Batch Output","deprecated":null,"version":null,"openapi":"/gateway-openapi.json get /batches/{id}/output","href":"/docs/api-reference/batch/get-batch-output"},{"title":"Cancel Batch","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /batches/{id}/cancel","href":"/docs/api-reference/batch/cancel-batch"}]},{"group":"Files","pages":[{"title":"List Files","deprecated":null,"version":null,"openapi":"/gateway-openapi.json get /files","href":"/docs/api-reference/files/list-files"},{"title":"Upload File","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /files","href":"/docs/api-reference/files/upload-file"},{"title":"Get File","deprecated":null,"version":null,"openapi":"/gateway-openapi.json get /files/{id}","href":"/docs/api-reference/files/get-file"},{"title":"Delete File","deprecated":null,"version":null,"openapi":"/gateway-openapi.json delete /files/{id}","href":"/docs/api-reference/files/delete-file"},{"title":"Get File Content","deprecated":null,"version":null,"openapi":"/gateway-openapi.json get /files/{id}/content","href":"/docs/api-reference/files/get-file-content"}]},{"group":"Fine-tuning","pages":[{"title":"List Fine-tuning Jobs","deprecated":null,"version":null,"openapi":"/gateway-openapi.json get /fine_tuning/jobs","href":"/docs/api-reference/fine-tuning/list-fine-tuning-jobs"},{"title":"Create Fine-tuning Job","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /fine_tuning/jobs","href":"/docs/api-reference/fine-tuning/create-fine-tuning-job"},{"title":"Retrieve Fine-tuning Job","deprecated":null,"version":null,"openapi":"/gateway-openapi.json get /fine_tuning/jobs/{fine_tuning_job_id}","href":"/docs/api-reference/fine-tuning/retrieve-fine-tuning-job"},{"title":"Cancel Fine-tuning Job","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /fine_tuning/jobs/{fine_tuning_job_id}/cancel","href":"/docs/api-reference/fine-tuning/cancel-fine-tuning-job"}]},{"group":"Moderations","pages":[{"title":"Create Moderation","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /moderations","href":"/docs/api-reference/moderations/create-moderation"}]},{"group":"Video","pages":[{"title":"Poll or Download Video","deprecated":null,"version":null,"openapi":"/gateway-openapi.json get /videos-proxy/{providerAccountName}/{nativePath}","href":"/docs/api-reference/video/poll-or-download-video"},{"title":"Generate Video","deprecated":null,"version":null,"openapi":"/gateway-openapi.json post /videos-proxy/{providerAccountName}/{nativePath}","href":"/docs/api-reference/video/generate-video"}]},{"group":"Models","pages":[{"title":"List available models","deprecated":null,"version":null,"openapi":"/gateway-openapi.json get /models","href":"/docs/api-reference/models/list-available-models"}]}]},{"tab":"Agent Platform","pages":[{"title":"Agent Platform","href":"/docs/agent-platform/agent-harness/overview"}]},{"tab":"Platform","pages":[{"title":"Platform","href":"/docs/platform/overview"}]},{"tab":"AI Deployment","pages":[{"title":"AI Deployment","href":"/docs/ai-deployment-overview"}]},{"tab":"API Reference","pages":[{"title":"API Reference","href":"/docs/truefoundry_sdk"}]},{"tab":"Integrations","pages":[{"title":"Integrations","href":"/docs/ai-gateway/ecosystem"}]},{"tab":"Changelog","groups":[{"group":" ","pages":[{"title":"Changelog","href":"/docs/changelog"},{"group":"Announcements","pages":[{"title":"Deprecation of Rate Limiting V1 — 2nd November, 2026","href":"/docs/change-announcements/deprecation-of-rate-limiting-v1"},{"title":"Move to Named Guardrail Policies — v0.179","href":"/docs/change-announcements/named-guardrails-config-v0.179"},{"title":"Perplexity Models Move to the Agent API","href":"/docs/change-announcements/perplexity-agent-api-migration"},{"title":"Plan-Based Feature Gating — v0.174.0","href":"/docs/change-announcements/plan-based-feature-gating-v0.174.0"},{"title":"Agent Builder Moving to Trueforge \u0026 Agent APIs Deprecated — v0.172","href":"/docs/change-announcements/agents-moving-to-trueforge"},{"title":"Removing Collaborators from Resources — v0.172.0","href":"/docs/change-announcements/removing-collaborators-from-resources"},{"title":"Deprecation of MCP tool_settings — 26th October, 2026","href":"/docs/change-announcements/deprecation-of-mcp-tool-settings"},{"title":"Deprecation of truefoundry \u003c 0.17.0 and truefoundry-sdk \u003c 0.4.0 packages","sidebarTitle":"Deprecation of truefoundry \u003c 0.17.0 and truefoundry-sdk \u003c 0.4.0 packages - 19th August 2026","href":"/docs/change-announcements/deprecation-of-truefoundry-below-0.17-and-api-ml"},{"title":"Minimum Python Version Raised to 3.10 — truefoundry v0.17.0","href":"/docs/change-announcements/python-3.10-minimum-truefoundry-cli-v0.17.0"},{"title":"Budget Limiting V2 — v0.158.0","href":"/docs/change-announcements/budget-limiting-v2-v0.158.0"},{"title":"Deprecation of MCP Servers APIs — 30th September, 2026","href":"/docs/change-announcements/deprecation-of-mcp-servers-v1-apis"},{"title":"Deprecation of Images and Plots in Job Runs — v0.156.0","href":"/docs/change-announcements/deprecation-of-images-and-plots-in-job-runs-v0.156.0"},{"title":"Moving to External Secrets Operator for Secrets Management — v0.151.5","href":"/docs/change-announcements/moving-to-eso-for-secret-management-v0.151.5"},{"title":"MCP DCR client_id \u0026 Redirect URL Validation — v0.148","href":"/docs/change-announcements/mcp-dcr-client-id-validation-v0.148"},{"title":"Deprecation of Routing Config","href":"/docs/change-announcements/deprecation-of-routing-config"},{"title":"Identity and Access Revamp — v0.143","href":"/docs/change-announcements/identity-and-access-revamp-v0.143"},{"title":"AI Gateway Metrics Rename — v0.136","href":"/docs/change-announcements/refactoring-of-ai-gateway-metrics"},{"title":"Deprecation of MCP Servers in Prompts","href":"/docs/change-announcements/deprecation-of-mcp-servers-in-prompts"},{"title":"Deprecation of Common Tools in Playground and Settings — v0.135","href":"/docs/change-announcements/deprecation-of-common-tools-v0-135-0"},{"title":"x-tfy-routing-config Header Removal — v0.133","href":"/docs/change-announcements/routing-config-header-removal-v0.133"},{"title":"Simplified GitOps CI/CD with tfy apply — v0.132","href":"/docs/change-announcements/simplified-gitops-cicd-v0.132"},{"title":"Legacy MCP OAuth Routes Removal — v0.134","href":"/docs/change-announcements/legacy-mcp-oauth-routes-removal"},{"title":"MCP Gateway URL and Transport Changes — v0.130","href":"/docs/change-announcements/mcp-gateway-url-transport-v0.130"},{"title":"Gemini CLI Model Registration Requirement — v0.118","href":"/docs/change-announcements/gemini-cli-model-registration-v0.118"},{"title":"Guardrails YAML Schema Change — v0.116","href":"/docs/change-announcements/guardrails-yaml-schema-change-v0.116"},{"title":"MCP Server Groups Removal — v0.112","href":"/docs/change-announcements/mcp-server-groups-removal-v0.112"}]}]}]}]},"basePath":"","title":"Chat Completions API (/chat/completions) - TrueFoundry Docs","deploymentId":"dpl_DyXGnKeCjZyA3sUdfMJ7FjkdBefg","buildId":"dpl_DyXGnKeCjZyA3sUdfMJ7FjkdBefg","dynamics":[],"chunks":[]}