Deploying Your Agno Agent on TrueFoundry

In this tutorial, we will walk through deploying your first Agno-powered AI agent using FastAPI and TrueFoundry. Agno provides a flexible agent framework, allowing you to integrate OpenAI’s GPT-4o model and DuckDuckGo for web searches. TrueFoundry simplifies the deployment process, making it seamless to launch and manage your AI-powered applications.

Overview

Our project consists of the following key components:

• Dependencies: Installed using pip.
• Environment Variables (.env): Stores the OpenAI API key.
• Agent Configuration: Uses Agno’s Agent class to set up an AI model and tools.
• FastAPI Application: Defines an endpoint to receive user queries and return AI-generated responses.
• TrueFoundry Deployment: Enables easy deployment of the API.

Let’s dive into each piece.

1. Setting Up the Environment

First, create a new project directory and install the required dependencies:

pip install fastapi uvicorn python-dotenv agno openai pydantic

Next, create a .env file to store your OpenAI API key:

OPENAI_API_KEY="your_openai_api_key_here"

2. Implementing the Agno AI Agent

The core of our application is the Agno agent, which is initialized with the OpenAI model and DuckDuckGo search tools.

from agno.agent import Agent
from agno.models.openai import OpenAIChat
from agno.tools.duckduckgo import DuckDuckGoTools
import os
from dotenv import load_dotenv

load_dotenv()

api_key = os.getenv("OPENAI_API_KEY")
if not api_key:
    raise ValueError("OPENAI_API_KEY environment variable is not set")

agent = Agent(
    model=OpenAIChat(id="gpt-4o"),
    tools=[DuckDuckGoTools()],
    markdown=True
)

Here, we:

• Load the OpenAI API key from the .env file.
• Initialize an Agent with OpenAI’s GPT-4o model.
• Equip the agent with DuckDuckGo search tools for web queries.
• Enable Markdown output formatting.

3. Building the FastAPI Application

Now, we define a FastAPI application with an endpoint to handle user queries.

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
from typing import Optional

app = FastAPI(title="Agent API", description="API for interacting with the AI agent")

class Query(BaseModel):
    question: str
    stream: Optional[bool] = False

@app.post("/query")
async def query_agent(query: Query):
    try:
        response = agent.run(query.question)
        print(response)
        return {"response": response.content}
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

Explanation:

• Query Model: Uses Pydantic to validate the input data.
• query_agent Endpoint:
  • Receives a JSON payload with the user’s question.
  • Passes the query to the AI agent.
  • Returns the response.
  • Includes exception handling for robust API performance.

4. Running the Application Locally

To start the FastAPI server, run:

uvicorn <your_filename>:app --reload

This will start the API server on <http://127.0.0.1:8000.> You can test the endpoint using curl:

curl -X POST "<http://127.0.0.1:8000/query>" \
     -H "Content-Type: application/json" \
     -d '{"question": "What is the capital of France?", "stream": false}'

Expected Output:

{
    "response": "The capital of France is Paris."
}

5. Deploying Your Agno Agent on TrueFoundry

Before deploying, make sure to set your OpenAI API key in the environment variables on TrueFoundry. Without this, your agent will not function correctly.

To deploy your AI agent on TrueFoundry, follow these steps:

1. Navigate to the Deployments section.

2.Click Service at the bottom.

3.Select your cluster workspace.

4.You can deploy from your laptop, GitHub, or Docker.

If deploying from your laptop, run the following commands:

pip install -U "truefoundry"

tfy login --host "<https://app.truefoundry.com>"

5.The TrueFoundry platform will generate a deploy.py file. Ensure that your OpenAI API key is set in the environment variables on TrueFoundry; otherwise, the deployment will not work. The auto-generated deploy.py file will need to be edited to include your API keys and specify the command to run the FastAPI server. Add your API keys inside it, such as:

envs={"API_KEY": "your_api_key"}

# Ensure your deploy.py includes the following command:
command="uvicorn agent:app --host 0.0.0.0 --port 8080"  # Replace 'agent' with your filename and specify your desired port
``` Place this file in your project repository.
6. Run the following command to deploy your agent:

```bash
python deploy.py

Your Agno agent is now deployed and running on TrueFoundry!

To confirm everything is working, you can send a test request using curl:

curl -X POST -H "Content-Type: application/json" -d '{"question": "Hello, how are you?"}' <https://agno-agent-demo-8080.aws.demo.truefoundry.cloud/query>

If everything is set up correctly, you should receive a response from your deployed agent, similar to the following:

{
    "response": "Hello! I'm here and ready to help you with any information or questions you have. How can I assist you today?"
}

6. Add traces to your agent

Tracing helps you understand what’s happening under the hood when an agent run is called. You get to understand the path, tools calls made, context used, latency taken when you run your agent using Truefoundry’s tracing functionality by add very few lines of code.

You need to install the following

 pip install traceloop-sdk

And then add the necessary environment variables to enable tracing

"AGNO_VERBOSE": "true",  # For detailed Agno logs
"TRACELOOP_BASE_URL": "<your_host_name>/api/otel" # "https://internal.devtest.truefoundry.tech/api/otel"
"TRACELOOP_HEADERS"="Authorization=Bearer%20<your_tfy_api_key>"

In your codebase where you define your agent, you just need these lines to enable tracing

from traceloop.sdk import TraceloopTraceloop.init(app_name="agno")

Conclusion

In this tutorial, we deployed our first Agno-powered AI agent on TrueFoundry using FastAPI. We configured the agent with OpenAI’s GPT-4o model and integrated web search functionality using DuckDuckGo. TrueFoundry’s platform streamlined the deployment process, allowing us to get our AI agent up and running with minimal effort.

This setup serves as a strong foundation for more advanced AI-powered applications, such as chatbots, research assistants, or customer support tools.

‍