Prompt Management

Documentation Index

Fetch the complete documentation index at: https://www.truefoundry.com/llms.txt

Use this file to discover all available pages before exploring further.

​

Creating a Prompt

1

Open the Prompt Registry

Go to Prompts in the left navigation and click + Create Prompt in the top right.

2

Configure the Prompt

The New Prompt screen opens with a chat-style editor. Configure your prompt by:

• Choosing a model from the model selector (supports virtual models for load balancing and fallback)
• Editing the System prompt to define the AI’s role and behavior
• Adding a User message to provide the input or query
• (Optional) Configuring Logging Config, Cache Config, Input/Output Guardrails, Structured Output, and Metadata from the left panel

3

Add Input Variables (Optional)

Make prompts reusable by referencing template variables with the {{variable_name}} syntax inside your messages. Variables are auto-detected and appear under Input Variables in the left panel, where you can preview them with sample values before saving.

4

Save the Prompt

When you’re happy with the prompt, click Save New Prompt in the top right. A side panel opens — fill in:

• Repository — Select an existing repository where the prompt will be saved (see Repositories)
• Prompt Name — Give your prompt a descriptive name
• Commit message (optional) — Brief description of the change
• Store selected model in prompt template — Keep this enabled if you want the prompt to remember its chosen model

​

Viewing Your Prompt

• Prompt / Spec — View the prompt content and its full configuration
• Try Prompt — Open the prompt in a runnable view to test it interactively
• Edit v<N> — Make changes; saving produces a new version
• Version History — Browse and diff previous versions
• FQN — Copy the prompt-version FQN (needed when calling the prompt from code)
• Use Via SDK / Use Via AI Gateway — Get ready-made code snippets for using the prompt in production

​

Advanced Configuration

​

Versioning

• Roll forward and roll back across versions
• Compare exact changes between any two versions
• Pin specific versions to specific apps or environments

Edit and Create a New Version

Open any prompt and click Edit v<N> to make changes. Saving the edited prompt produces a new version (e.g. v2).

Browse Version History

Click Version History on the prompt detail page to see all versions of the prompt with their commit messages and authors.

Compare Versions (Diff Mode)

Inside Version History, switch to the Version Difference tab to see exactly what changed between two versions in a GitHub-style diff view.

​

Using the Prompt

• Use Via AI Gateway (server-side rendering)
• Use Via SDK (client-side rendering)

Pass the prompt version FQN in the request body. The Gateway renders the prompt and executes the call.Parameters

• prompt_version_fqn — The fully qualified name of your prompt version
• prompt_variables (optional) — Variables to substitute into the prompt template

Important considerations

• If the prompt version doesn’t have a model configured, pass the model in the request body via the model parameter
• If you specify a model in the request body and the prompt version already has one configured, the request body model takes precedence
• Any messages passed in the request body are appended to the messages defined in the prompt version

Open the Use Via AI Gateway tab on the prompt detail page to get a ready-to-paste snippet in your preferred client library (OpenAI, LangChain, Node.js, Curl, REST, and more).

from openai import OpenAI

client = OpenAI(
    api_key="your-tfy-api-key",
    base_url="{GATEWAY_BASE_URL}"
)

stream = client.chat.completions.create(
    messages=[],
    model="",
    stream=True,
    extra_headers={
        "X-TFY-METADATA": '{"your_custom_key":"your_custom_value"}',
        "X-TFY-LOGGING-CONFIG": '{"enabled": true}',
    },
    extra_body={
        "prompt_version_fqn": "chat_prompt:truefoundry/default/my-second-prompt:1",
        "prompt_variables": {
            "name": "John Doe",
            "age": "30"
        }
    },
)

for chunk in stream:
    if (
        chunk.choices
        and len(chunk.choices) > 0
        and chunk.choices[0].delta.content is not None
    ):
        print(chunk.choices[0].delta.content, end="", flush=True)

Fetch the prompt template, render it locally, then call any OpenAI-compatible client.Open the Use Via SDK tab on the prompt detail page to copy the snippet pre-filled with your prompt version FQN.

from truefoundry import client, render_prompt
from openai import OpenAI

# Fetch the prompt template
prompt_template = client.prompt_versions.get_by_fqn(
    fqn="chat_prompt:truefoundry/default/my-second-prompt:1",
).data.manifest

# Render the prompt with input variables
variables = {}  # Add your variables here
prompt = render_prompt(prompt_template, variables=variables)

# Send request to OpenAI directly
response = OpenAI().chat.completions.create(
    messages=prompt["messages"],
    model=prompt.get("model") or "gpt-4o-mini",
    **(prompt.get("parameters") or {}),
)

print(response.choices[0].message.content)