May 8, 2024
An Overview: Sagemaker Vs TrueFoundry
Amazon SageMaker is a fully managed machine learning (ML) service and provides a range of functionality from data preparation to ML governance. It's functionality, performance, security, and scalability are closely tied to the underlying infrastructure and services provided by Amazon Web Services(AWS). A solid grasp of AWS services is preferred for effectively integrating various offerings and leveraging the ecosystem, including tools like AWS Glue, CloudWatch, etc.
Here is a preview highlighting the wide array of offerings that constitute SageMaker.
On the other hand, TrueFoundry focuses on model deployment automation. TrueFoundry's underlying architecture leverages Kubernetes. It enables us to leverage its advantages for optimizing infrastructure efficiently and pass those benefits on to you. We abstract all the complexities, allowing you to utilize the platform without needing any Kubernetes expertise. In Sagemaker, the deployment of models occurs on AWS-managed machines, wherein users have limited flexibility from an infrastructure optimisation perspective.
This architecture helps us capitalize on the advantages of self-managed clusters, enabling faster, simpler, and more cost-effective deployments. Also, Truefoundry's platform is engineered to facilitate seamless integrations and function across one or multiple clouds, as well as on-prem.
Key differences between Sagemaker and Truefoundry
Over 40% cost savings versus Sagemaker
TrueFoundry enables savings of more than 40% on total costs compared to running identical workloads on Sagemaker.
- Using Bare Kubernetes
SageMaker puts a markup of 25-40% on instances that are provisioned using SageMaker whereas TrueFoundry helps teams make use of raw Kubernetes through EKS
- Fractional CPUs and GPUs
TrueFoundry provides users with the flexibility to specify fractional CPU units, enabling requests as low as 0.1 CPU without the limitation of a minimum requirement of 1 CPU. This flexibility extends to GPUs as well, allowing users to utilize fractional GPU resources as needed.
- Reliability Layer On Spot Instances
Spot instances are provided at 40-60% cheaper by AWS, at the cost that it can be taken away whenever AWS needs it. TrueFoundry ensures that while using spot instances, the workloads stay reliable to serve production traffic without any request drops.
- Optimum Infrastructure Utilization
We have multiple complementary features designed to optimize costs further and minimize the risk of errors -
- Creating a reliable method of using spot instances with fallback on-demand
- Model caching to reduce transfer costs
- Autoscaling nodes as per traffic, pause our service & scale down to zero
- Time-based autoscaling (e.g. shut down development instances from 11 PM to 9 AM and on weekends)
- Culling notebooks when notebooks are not in use
- Creating Cost Visibility
Built in features for forecasting cost estimates, monitoring project level costs and fine-grained access control on resources for controlling costs.
You can look at this detailed product-tour to look at how the above cost optimzation features are integrated into our product.
Faster Startup Time
Truefoundry can deploy instances within one minute, whereas the same process takes approximately 2 to 8 minutes on Sagemaker, varying based on the instance type. This faster deployment time leads to improved autoscaling and enhanced reliability.
No constraint of libraries
Truefoundry doesn’t put any opinion on the code style or the libraries you are using to deploy the code. This allows complete flexibility to data scientists to use their favorite framework like FastAPI, Flask, Pytorch Lightning, Streamlit, etc to code up their apps. This also allows for easy portability of code which isn’t true in Sagemaker unless you are doing custom containers.
Cloud Native and No Vendor Lock-in
Truefoundry doesn't impose any restrictions on code style or libraries used for deploying code. This grants data scientists complete flexibility to utilize their preferred frameworks such as FastAPI, Flask, PyTorch Lightning, Streamlit, and more for developing their applications. Additionally, this flexibility facilitates easy portability of code—a feature not readily available in Sagemaker unless custom containers are utilized.
Fractional GPU
As mentioned above, Truefoundry support fractional GPU which makes it easy to maximize GPU usage.
The fractional GPU system allows data science and AI engineering teams to concurrently run multiple workloads on a single GPU, allowing companies to efficiently manage and execute a greater number of workloads.
Automated resource optimization
Truefoundry provides automated resource optimization insights which helps you run the applications in a reliable and cost-effective way.
Easier to get started and better UX
Many data scientists perceive Sagemaker to have a significantly steeper learning curve compared to Truefoundry. With Truefoundry, you can begin deployments in less than 10 minutes, making it more accessible and user-friendly for users.
Excellent level of support
Truefoundry guarantees a support response time SLA of under 10 minutes. Additionally, customer support reviews are available on G2 for further reference.We boast a 9.9/10 for customer support on G2.
Additional benefits for LLMOps
TrueFoundry extends the fundamental features of training and serving for LLMs as well, offering additional benefits that include the following-
LLM Gateway
Truefoundry offers an LLM gateway that enables developers to utilize various LLMs through a unified API, complete with cost attribution, rate limiting, and quotas. Sagemaker lacks this functionality.
LLM Model Deployment
Truefoundry can automatically determine the most optimal settings for any HuggingFace LLM model or embedding model, eliminating the need for manual configuration. Conversely, on Sagemaker, this optimization process has to be performed manually.
LLM Model Finetuning
Truefoundry can automatically identify the optimal settings for model fine-tuning, eliminating the need for manual intervention by the user. This streamlined process saves significant time during iteration.
About TrueFoundry
TrueFoundry is a cloud-native PaaS empowering machine learning teams to deploy and fine-tune ML and Large Language Models on their preferred cloud or on-premises infrastructure. Our platform offers:
- Cost Optimization: Achieve 30-40% reduction in cloud costs compared to alternatives like Sagemaker, along with full data privacy and security.
- Reliability and Scalability: Ensure 100% reliability and scalability, enabling teams to launch GenAI applications to production 80% faster than other methods.
- Comprehensive Ecosystem: Assist in deploying the entire ecosystem of components necessary to build end-to-end LLM applications. We provide native integration with popular LLM tools such as Langchain/LLamaIndex and Vector Databases like Milvus and Qdrant.
With TrueFoundry, machine learning teams can efficiently leverage their infrastructure while ensuring cost-effectiveness, security, and rapid deployment of AI applications.
