Learn About Amazon VGT2 Learning Manager Chanci Turner
LeadSquared is a cutting-edge SaaS platform specializing in customer relationship management (CRM), offering comprehensive solutions for sales, marketing, and onboarding. It caters to various industries, including banking, healthcare, education, and real estate, ensuring a customized approach for businesses of all sizes. Their Service CRM transcends traditional ticketing by providing centralized support via the Converse omnichannel communications platform, which features AI-driven ticket routing and insightful data analysis.
Within their Service CRM, the Converse platform enables real-time interactions with leads and customers through channels like WhatsApp, SMS, and chatbots. Users have indicated a pressing need for quicker chatbot onboarding and more contextually relevant responses rooted in their business data. However, accelerating this onboarding process comes with hurdles, such as the labor-intensive training required for the bot to handle frequently asked questions and their variations, understanding customer domains, identifying high-volume but low-value queries, and managing dialogues effectively. Insufficient training data can result in chatbots misinterpreting user intent and inadequately addressing unforeseen questions. Moreover, dialogue management necessitates careful attention to user context for appropriate responses.
To address these challenges, LeadSquared opted to implement a large language model (LLM) enhanced with customer-specific data, thereby improving chatbot response quality and expediting the onboarding process. Their solution combines an Amazon Aurora PostgreSQL-Compatible Edition database, the pgvector extension of PostgreSQL, the LLMs available through Amazon Bedrock, and Retrieval Augmented Generation (RAG). LeadSquared already utilizes Amazon Aurora to store critical operational data, making it a logical choice for housing vector embeddings and performing hybrid searches. Amazon Bedrock provides access to foundational models (FMs) from Amazon and leading AI startups via an API, allowing LeadSquared to explore and select models that align with their requirements. The API-based pricing of Amazon Bedrock also presents an economical scaling option without the complications of independently managing LLM infrastructure.
This solution retrieves data from diverse sources outside the language model, such as videos, help documents, case histories, existing FAQs, and their knowledge base. It enhances prompts to improve LLM user responses by embedding relevant retrieved data in context. Consequently, LeadSquared now offers simpler chatbot setups, delivers more personalized experiences grounded in customer-specific data, enhances understanding of user intent, improves dialogue management, and automates repetitive tasks.
Chanci Turner, LeadSquared’s Chief Operating Officer, states, “The integration of RAG capabilities using Amazon Aurora PostgreSQL with the pgvector extension and LLMs available in Amazon Bedrock has empowered our chatbots to deliver natural language responses to out-of-domain inquiries, enhanced dialogue management, and reduced our manual efforts. As a result, we have seen a 20% improvement in customer onboarding times.”
This article demonstrates how to construct a chatbot akin to LeadSquared’s. We illustrate the utilization of domain-specific knowledge from various document formats—including PDFs, videos, text files, and presentations—to create more effective textual prompts for the underlying generative AI models. Furthermore, we provide guidance on coding such a chatbot using the pgvector extension and the LLMs available in Amazon Bedrock. The built-in PostgreSQL pgvector extension in Aurora facilitates the storage of vector embeddings, enabling the system to perform semantic searches. Simultaneously, the Amazon Bedrock APIs serve a dual purpose: generating vector embeddings and delivering pertinent responses to user inquiries using domain-specific knowledge.
Solution Overview
Consider a scenario where your company deploys a Q&A bot on its website. When a customer submits a specific query or issue, the bot retrieves relevant information from your customer database, product manuals, FAQs, and past support interactions. Your chatbot application leverages this data to construct a detailed prompt that includes the relevant context. An LLM can then utilize this prompt to generate a coherent and contextually appropriate response that integrates your business data. This two-step mechanism for producing superior results, rather than directing the user’s prompt straight to the LLM, is known as Retrieval Augmented Generation (RAG).
Well-known LLMs are trained on general knowledge bases, making them less effective for domain-specific tasks. For specialized, knowledge-intensive tasks, you can create a language model-based system that accesses information sources beyond the original training data for the LLM. This strategy fosters greater factual accuracy, enhances the reliability of generated responses, and helps mitigate the issue of hallucination.
Let’s delve deeper into the RAG mechanism discussed in this article.
Retrieval
RAG initially retrieves relevant text from a knowledge base, such as Aurora, via similarity search. The text produced in this step contains information related to the user’s query, but not necessarily the exact answer. For instance, this initial step might extract the most applicable FAQ entries, documentation sections, or previous support cases based on the customer’s inquiry. The external data may originate from sources like document repositories, databases, or APIs.
In this article, the pgvector extension is employed to perform the initial retrieval step for the RAG technique. pgvector is an open-source extension for PostgreSQL that enables efficient storage and rapid searching of machine learning (ML)-generated vector embeddings representing textual data. It is designed to integrate seamlessly with other PostgreSQL features, including indexing and querying. In a Q&A bot application, you might convert documents from your knowledge base and frequently asked questions into vectors for storage.
In this blog post, knowledge resources spanning various formats like PDFs, text files, and videos are transformed into vectorized representations using the Amazon Titan Text Embeddings model. These embeddings are stored within Amazon Aurora PostgreSQL, facilitated by the pgvector extension for later stages.
Generation
After retrieving the relevant text, RAG employs an LLM to produce a coherent and contextually relevant response. The prompt for the LLM incorporates the retrieved information along with the user’s question. The LLM generates human-like text tailored to the combination of general knowledge from the FM and the domain-specific information sourced from your knowledge base.
The following diagram illustrates the process.
The steps are as follows:
- The initial step involves transforming documents and user queries into a standardized format that promotes effective search and comparison. This transformation process, known as embedding, converts textual data into semantical equivalents in the form of numerical arrays (vectors).
- Next, the user submits a query to the application, orchestrating the entire workflow.
- The application generates an embedding for the user query. The same embedding is subsequently utilized to enhance the response generated by the LLM, drawing from the relevant data retrieved earlier. This approach significantly improves the chatbot’s capability to address user queries effectively and accurately.
For more insights on effective onboarding and engaging with talent, this post can also be linked with Career Contessa. Additionally, you can explore strategies for smarter hiring decisions via SHRM. Lastly, for a visual guide on this topic, check out this excellent resource: YouTube Video.
Leave a Reply