August 8, 2024
TrueML Talks #27 - GenAI and LLMOps for Customer Success @ Level AI
January 18, 2024
Share this post
We are back with another episode of True ML Talks. In this, we dive deep into GenAI and LLMOps strategy in Level AI as we are speaking with Abhimanyu Talwar
Abhimanyu is Staff AI Research Engineer at Level AI. Level AI a conversational intelligence company. They use machine learning to get insights out of conversational data.
📌
Our conversations with Abhimanyu will cover below aspects:
- Embracing Past Conversations for Customer Service AI
- Tackling the Hidden Challenges of Generative AI
- Open Source vs. Fine-Tuning
- Understanding Agents
- Demystifying Contact Center QA with AI
- Customer Service AI with GPT-4
- MLOPs Revolutionizing Customer Success
Watch the full episode below:
Embracing Past Conversations for Customer Service AI
Customer service AI is all the rage, but relying solely on static knowledge bases has its limits. Agent Assist and AgentGPT, two innovative tools fromLevel AI, that unlock the power of past conversations to boost agent efficiency and customer satisfaction.
- Agent Assist: Parses customer queries and suggests relevant resources from the existing knowledge base, empowering agents to respond effectively.
- AgentGPT: Takes it a step further. It analyzes past conversations, automatically extracting and indexing valuable information not found in the knowledge base. This creates a dynamic, ever-growing corpus of solutions to nuanced customer queries.
Retrieval: Mining Past Conversations for Answers
Forget keyword matching! Level employs a powerful retrieval pipeline:
- Pre-processing: Past conversations are transformed into a readily searchable format.
- Custom Indexing: Unique keys are designed to match current queries with relevant past interactions.
- Embeddings: Advanced embedding models (think RAG pipelines) capture the semantic meaning of conversations, enabling accurate retrieval.
To give better results, they further do the following:
- Re-ranking: Prioritize the most relevant results based on context and additional factors.
- Keyword Filters: Eliminate irrelevant or unhelpful past interactions from the search.
By harnessing both the power of static knowledge bases and the dynamic insights hidden in past conversations, Agent Assist and AgentGPT offer a glimpse into the future of customer service AI. This future is one where agents are empowered with the right information, leading to faster resolutions, happier customers, and a more efficient contact center.
Tackling the Hidden Challenges of Generative AI
Gen AI is a powerful tool, but its success hinges on careful data curation, robust evaluation methods, and a data-driven approach
Picking the right data from your vast corpus is like finding the perfect ingredients for a delicious dish. Poor data selection leads to a model that's, well, inedible.
First thing is focus on your data. Basically, have a very good handle on what your data mix looks like. What is the quality of your annotations? All of that really matters a lot. Otherwise it will be garbage in, garbage out
– Abhimanyu
After picking the right data, you will need to evaluate your Gen AI models. This isn't as straightforward as traditional AI tasks. Forget simple metrics like N-gram overlap – they miss the nuances of correctness. A single swapped word ("yes" to "no") can make all the difference.
For this you can use:
- Programmatic Metrics: Utilize tools like ROUGE and BERT scores to assess semantic similarity.
- Human Annotations: Employ real humans to evaluate qualitative aspects like coherence and completeness.
Don't rush into large models. Conduct experiments with smaller checkpoints to find the optimal data mix and task weightage.
Open Source vs. Fine-Tuning
It's tempting to stick with ChatGPT's apparent versatility. While GPT-4 shines in unconstrained scenarios, businesses operate on real-world limitations. High traffic volume demands efficient, cost-effective solutions without sacrificing performance or responsiveness.
This is where fine-tuning your own models can be advantageous:
- Hyper-focus: Instead of trying to be good at everything, tailor your model to excel at specific, business-critical tasks. Imagine a model adept at contact center interactions instead of writing graduate school essays!
- Cost-effectiveness: Smaller, focused models require less computational power, translating to lower operational costs.
- Control and Explainability: Fine-tuning grants you deeper understanding and control over your model's behavior, fostering trust and transparency.
ChatGPT as a Springboard
Don't write off options like ChatGPT entirely. They can be valuable allies! Consider using them as a starting point to:
- Validate your idea: Test the waters with open-source models before investing in custom development.
- Gather business traction: Use readily available tools to get real-world feedback and demonstrate viability before diving into fine-tuning.
Understanding Agents
What are agents?
Think of them as specialized teams of LLMs, each playing a specific role in a larger workflow. Instead of single API calls, tasks involve multiple "agents" collaborating through sequential API calls.
Why use agents?
Imagine writing a poem: you need creativity, rhyme analysis, and even grammar checking. One LLM might excel at generating initial verses, another at ensuring rhyme schemes, and a third at polishing the final draft. Agents let you leverage the unique strengths of different models to achieve superior results.
When are agents the right choice?
- Complex tasks: When a single LLM struggles with the full scope of your goal, agents can break it down into manageable steps.
- Precision and control: Sequencing different "agent" calls gives you more control over the process, allowing you to tailor the steps to achieve specific outcomes.
- Combining diverse skills: Leverage the unique strengths of various LLMs to create truly innovative outcomes.
Demystifying Contact Center QA with AI
One of the offering of Level AI is Agent Assist, a powerful AI tool powered by GPT technology. It helps automate QA by analyzing conversations and providing insights into agent performance.
Here's how it works:
- QA Rubric: Contact centers have specific criteria for good calls, like proactive problem-solving and successful resolution. Agent Assist learns these criteria from past conversations.
- AI Analysis: Each call is analyzed, and Agent Assist provides a score and explanation based on the QA rubric.
- Explainability with Evidence: Not just a black box, Agent Assist highlights specific parts of the conversation that support its score, giving agents valuable feedback.
- Human-in-the-Loop: QA managers can review Agent Assist's findings and adjust scores if needed, ensuring accuracy and fairness.
Benefits of Agent Assist:
- Faster QA: Saves time and resources compared to manual reviews.
- Improved Agent Performance: Provides targeted feedback to help agents excel.
- Data-Driven Insights: Reveals trends and areas for improvement in customer service.
- Increased Efficiency: Frees up QA managers for more strategic tasks.
Customer Service AI with GPT-4
GPT-4's arrival has sparked excitement in the AI world, but is it a one-stop shop for exceptional customer service experiences? Not quite. While its power is undeniable, there are a lot of hidden layers behind truly impactful AI solutions.
The Power and Pitfalls of Big Models:
GPT-4's raw potential is remarkable, surpassing open-source models and APIs in generating answers. However, relying solely on its output overlooks the crucial parts of the AI pipeline: data selection, feature extraction, aggregation, and business knowledge.
Building a Winning Team:
- Data Selection: Finding the right training data is key. Annotating past conversations specific to your industry teaches the AI what "good" and "bad" service looks like.
- Feature Extraction: Raw data needs distilling. Extracting key attributes from every conversation allows for analysis and comparison.
- Aggregation and Clustering: Millions of conversations with countless attributes are overwhelming. Smart clustering schemes present the data in a way humans can understand and utilize.
- Business Knowledge: Understanding what truly matters to your customers goes beyond metrics. Industry experience helps prioritize the right KPIs and guide AI development.
You can read more about how companies leverage human expertise and AI for Customer Service in the blog below.
MLOPs Revolutionizing Customer Success
The customer success landscape is being redrawn by MLOPs, the magic formula behind efficient AI deployments. Here's a sneak peek at its 5-year impact:
- Boost Efficiency: Say goodbye to slow queries and frustrated customers. AI-powered chatbots, fueled by MLOPs, will instantly answer questions, freeing agents for meaningful interactions.
- Human-AI Harmony: The future isn't robots replacing humans; it's about collaboration. MLOPs empowers agents with data-driven insights for smarter sales pitches, proactive problem-solving, and personalized experiences.
- Data-Driven Decisions: MLOPs unlocks the power of customer data. AI models will predict needs, personalize recommendations, and build trust through informed actions.
- Hyper-Personalization: MLOPs tailors everything from offers to support, cultivating lasting relationships and loyalty.
- Taming the Hallucination Beast: MLOPs tackles inaccuracies with focused training and evidence-based verification, ensuring reliable customer interactions.
MLOPs is the key to creating a customer-centric future, but responsible and ethical use is crucial. By embracing its potential while addressing challenges, we can build success stories worth shouting about.
You can read more about how Generative AI will shape the future of Customer Experience in the blog below.
Read our previous blogs in the True ML Talks series:
Keep watching the TrueML youtube series and reading the TrueML blog series.
TrueFoundry is a ML Deployment PaaS over Kubernetes to speed up developer workflows while allowing them full flexibility in testing and deploying models while ensuring full security and control for the Infra team. Through our platform, we enable Machine learning Teams to deploy and monitor models in 15 minutes with 100% reliability, scalability, and the ability to roll back in seconds - allowing them to save cost and release Models to production faster, enabling real business value realisation.
