Last year, we likened our startup journey to building a rocket ship, and imagined 2024 as the year of ignition — the moment our rocket ship would leap into the orbit. And ignite we did! Were we thinking big when landing a Fortune 500 customer meant working with NVidia, or getting recognized as an emerging leader in Gartner’s magic quadrant?
But as always, this tale isn’t just a celebration of our achievements but also an acknowledgment of the challenges we’ve navigated, appreciation of the opportunities we have been presented with and the learnings we’ve embraced. Let’s take you on this exhilarating ride — from thrilling breakthroughs to uncharted territories — navigating both turbulence and triumphs in this stellar odyssey!
TrueFoundry and year 2023
TrueFoundry is building a cloud-agnostic PaaS on Kubernetes, that standardizes the training and deployment of Machine learning & Generative AI applications using production-ready, developer-friendly APIs — while taking an opinionated stance on how MLOPs / LLMOps and DevOps are feathers of the same flock!
2023 was the year of unprecedented opportunities and existential threat both at the same time for most organizations in the world. How they embraced the changed world would define their future or dictate whether they had any! And organizations were looking for any help possible to help them define use cases, run through experiments, fight the security policies, come up with press-releases to be knows as the forerunners of the GenAI world.
TrueFoundry was very well positioned to help organizations navigate these uncertain times, the choices between in-house vs closed source models, the complexity of GPU based infrastructure, the toolkit to ship applications to production. And that, right there, was the key — we knew any application that is not in production is a cost-center that delivers zero value. And while, the rest of the world was running experiments for PR purposes, a Fortune 100 customer of TrueFoundry was churning applications to prod at lightning speed! That head start in 2023 made them a leader in GenAI in their industry! We took a couple of key decisions in 2023 that set us up for success — we separated out what was urgent in the short term and what was important in the long run!
- First, we focused on organizations, that we believed had a clear, long-term differentiated business value out of GenAI and not just running after the craze!
- Second, we didn’t run after helping organizations with everything that was trending then - over-indexing on GPUs being scarce, or organizations not having permission to use best-in-class closed-source models. Don’t get me wrong- those problems needed solving, but they couldn’t be the only thing you focus on because it was a matter of time when these would go away! Rather, we focused single-mindedly on what are the challenges organizations would face in confidently shipping apps to prod in the long run!
This led us to two important learnings which we got right and have taken a note of.
1. Real test is not just putting your product out in front of a customer, but in the path of real business value generated for that customer!
2. Its okay if you wanna catch a trending wave, but when a shakedown happens, be prepared to emerge on the other side — and that, only happens, by focusing on the first principles.
Kicking off 2024
With that, we entered 2024 strong — which was a year of execution for us, doubling down on what we thought would work — and putting it through real tests — the TrueFoundry way!
We worked with some of the world’s largest organizations, and helped them bring to life applications that would bring $100Ms of dollar value- through automating customer support calls / optimal utilization of GPU clusters / helping sales-reps sell drugs better. We became a critical component of the GenAI stack of multiple Fortune 500 companies, double our team size and quadrupled our revenues- all within this year. Question is- what led to this categorized as what we got right, what we didn’t strategically and tactically? Let’s dive into this —
[Strategic, Worked] Uniformity of LLMOPs, MLOPs & DevOps —
We have talked about this in the past in detail. But its such a critical decision of our platform, that its important to mention! In 2022, when we focused on MLOPs we never thought of it as fundamentally different from DevOps — ML models are also applications that need to run on some compute — albeit more complicated. In 2023, we took the same stance, when GenAI hit the world- LLM fine tuning or ML model training or running a data pipeline job are all long running compute jobs and LLM model serving or ML model inferencing or a simple Rest API are all continuous running jobs. And so long as architecturally, they are the same, its a matter of building the right UX on how to maneuver the complexity of handling GPU resources, or distributed computing or large model sizes or very long running jobs — a lot of engineering for sure, but nothing fundamentally different.
In other words, TrueFoundry fundamentally takes any application or code, and translates that into a K8s manifest while hiding away the application specific complexity. This design had a profound impact on how customers viewed TrueFoundry.
TrueFoundry acted as a bridge from the unknown territory of GPUs, cross-cloud infrastructure, very large models to the known territory of their existing platform built on K8s.
TrueFoundry spoke their language, it fit in their existing stack and enabled organizations to leverage all the fundamental build and deploy pipelines they had painstakingly set up!
[Strategic, Worked] Core built of primitives — Services, Jobs, Helm Charts & Volumes
At TrueFoundry, we believe that any application in the world, can be fundamentally thought in terms of 4 primitives —
- Services — always up & running APIs
- Jobs — Long running offline jobs
- Helm Charts — Use, find and share appplications built on K8s.
- Volumes — Data mounts, filestore and storage layer.
These primitives became our core layer and then everything is built on top of it. For example, LLM hosting is nothing but a service, fine tuning is nothing but a job and VectorDB is nothing but a helmchart!
And the new paradigm of compound AI application can be thought of as a combination of these primitives. For example, a RAG application comprises of — reading source data from a volume, parsing, chunking and indexing as a job, VectorDB as a helm chart, LLM as a service, and RAG API as a service!
While the modus operandi in 2023 was RAG and 2024 were agentic applications, it could be different in the future. This architecture heps us be future safe and we are not tied to any specific ways of development. In 2024, we generalized this concept as a product and been received very well by our customers.
[Tactical, Worked] Future safe design — avoid vendor lockins
The standard of building GenAI apps is not out yet, and organizations are not willing to lock themselves into anything and this extends to cloud providers, models providers and framework providers.
TrueFoundry’s design of choosing any compute from any provider without worrying about the infra-management layer, choosing any model through Gateway without worrying about API signature or helping people orcehstrate deployments without prescriptive coding, saving dependency on any framework — including ours, has resonated very well with our customers. In fact, we go to the extent of making TrueFoundry redundant as well by exposing raw K8s manifests generated from TrueFoundry so the customer is never locked into TrueFoundry either!
I have to admit though, that while this is working now, we are not confident that in the long-run it’s gonna play this way. One potential example of catching a wave, but being mindful of what’s on the other side.
[Tactical, Worked] Focus on time to value
TrueFoundry’s focus has always been on reducing the time to ship production-ready applications for our end-users. Time to value is the core metric that we have always optimized for. In 2024, we spent enough time trying to optimize the time-to-value for us as a platform as well — which means, how quick is it to install TrueFoundry within a customer environment and how quick is it to derive final business value for end-users.
It’s clear in our G2 reviews, that our Time to go-live of 0.42 months is significantly better than others in our category at 2.29 months and the estimated ROI at 4 months compared to the average of 13.66!
[Tactical, Didn’t Work] Being closer to use cases
With our focus on getting the architecture right, we believe, there have been cases where we missed the mark on being super close to the end goal that the user is trying to achieve. What that means is its sometimes a bit of a lift for the end users to build the final application and we can make our product experience better aligned with that. For example, people today can build and ship any agentic application on TrueFoundry — because of how our architecture enables shipping any compound AI applications but is the experience as seamless as we would like? Probably not!
We had one major success in this area in 2024, with our first open-source launch, Cognita — a framework built to ship production-ready RAG applications which saw more than 3000 stars in the first couple weeks of launching! But I wanna say, this might be too-little, too-late! Ideally, this is an area we should have optimized for in 2023 itself and built a lot more in 2024! But now that we have realized this, we need to actively work towards this in 2025.
[Strategic, Didn’t Work] Focus on Buyer Persona
Common wisdom of startup world — if you intend to create a repeatable selling motion, you need to be laser focused on your Ideal-customer-profile and the buyer persona. We thought we knew this and “ruthlessly prioritized” this part to finally narrow down our buyers to be one of the two users — a Head of data science trying to ship a GenAI app to production, and a Head of platform building developer tooling for all the internal data science teams.
Our latest belief being — two is still not good enough. Laser focus means one and only one! This will help us optimize everything from the end UX of the product, to the Sales Enablement materials, to the product marketing and every function that builds the company. The jury is still not out, but as of now, it does seem like we might have to spend time in 2025 narrowing this down further!
Looking Ahead & Welcoming 2025
Armed with the learnings, a fundamentally & architecturally grounded product, a strong customer momentum and alongside a brilliant & relentless team who genuinely cares about solving this problem, we are excited to venture in 2025 — we feel we are positioned stronger than ever!
Most importantly, we are excited to embrace the changes GenAI has brought for ourselves as a startup and expanding our own vision! With the capabilities unlocked by GenAI, we believe that all the improvements in time-to-value, cost-savings, and being able to do more with less are just milestones. Eventually, everything will be instantaneous, most efficient and all on AutoPilot! Just like today, we don’t think about whether a compiler efficiently moved memory or allocated resources when we write a program, tomorrow, we won’t think about that the infrastructure managed by AI is done right.
If we are entering a world, where thousands of agents would interoperate with human beings to accomplish each task, it’s neither feasible not logical to have human beings become the bottle-neck for managing them. The central platform that manages world’s
AI will be managed by AI
With our eyes, set on this expanded vision, we welcome 2025 with open arms! Happy New Year everyone.
