weaviate

An open-source, AI-native vector database enabling developers to build and scale AI-powered applications with advanced data retrieval capabilities.

Weaviate

Hi I'm from Weaviate. Retrieval augmented generation or RAG has quickly become a popular use case for large language models. In this video we'll go into different use cases for RAG systems and we'll look at how generative feedback loops can help improve performance and take your RAG application to the next level. 

First let's explore why doing RAG is necessary. Here's a scenario: I planned a business trip to Berlin and needed more details about my current booking. I had requested an early check-in for 11:00 a.m. but I have not received any response. The service operates entirely online with digital communication through chat bots, so I sent a follow-up message via their chat interface. 

Here's the message I sent: Hi I'm writing to follow up on my reservation for tomorrow. I'm super excited about my stay and I cannot wait to arrive. My reservation details are as follows: reservation number, the check-in date, and the checkout date. I had previously requested an early check-in at 11:00 a.m.

But I haven't received any confirmation yet. So could you please let me know what time the check-in will be possible? However, the automated response I received was rather disappointing. I didn't quite get that. I understand short messages best, max 140 characters ideally, one topic at a time. So as I try to shorten my message, the responses continue to be unhelpful like this one. Hm, I did not catch what you were saying. 

I am a virtual assistant. I understand short messages best, ideally one topic at a time. So after multiple attempts, I finally received a response, but I was still unsatisfied and also slightly frustrated. Please note that we are not able to guarantee an early check-in. We will only be able to check you in if the room is ready on the day of your arrival. We will do our best to prioritize your room according to your arrival. 

So what went wrong here? The chatbot couldn't understand my problem because the keywords in my query didn't match its database knowledge. I had to break down my text; otherwise, it couldn't understand my specific question. It took several rounds to get a response, leaving me frustrated and not hopeful for future improvements. The bot couldn't generate a conversational response, forcing me to select predefined options. As we can see, chatbots can have a lot of problems, especially if they rely on outdated systems like pure keyword search or predefined.

User interaction pass leading to misunderstanding and frustration. So how can rack help? Instead of struggling with exact keyword matching, we could use a vector database like Weaviate to perform semantic search. This approach allows the query, whether short or long, to be understood based on its meaning and context, not just on the specific words. 

With Factor search, we can retrieve the most relevant information by understanding the intent and the context of the question. Weate also offers hybrid search that combines both keyword and vector search to benefit from both techniques. With a rag system, we can use a generative large language model to generate a natural language response using the context provided by the vector database and the user's question. 

This means we don't have to rely on predefined options or user workflows and can lead to more efficient and effective interactions. At Weaviate, we've developed Verba, an open-source rag application that performs rack on your own data. For this demo, we've customized it as an Airbnb chatbot using Airbnb's customer documentation. 

Users can ask booking questions, receive relevant documents, and access original sources, making it feel more conversational, more natural, and more interactive. We'll start with a query that will then get vectorized and be used to search through our vector database for the most similar context using hybrid.

Search then that context along with with the original question will be included in a prompt for a generative large language model. The large language model will then use all of that information to generate a conversational response. While RX solves our problem pretty well, we can use AI agents to improve different aspects of our pipeline. 

The AI agents are autonomous systems that can perform tasks such as analyzing and processing user queries, making decisions, and executing actions to enhance application functionality. Agents are often used in rack to rewrite user queries based on previous conversations and decide what predefined action to do next. We'll show you how adding an agent performing generated feedback loops can increase speed and reduce cost of our pipeline. 

Generative feedback loops add one more step to our pipeline, taking large language model generated results and saving it back into the database. This technique can be used to automatically expand the chatbot's knowledge base over time and improve its ability to handle diverse queries. By saving generated answers, the system reduces costs by providing cached results to similar queries. 

For example, let's say we want to save time and money. High-end generative models like GPT 4 or Gemini often work better than smaller models like Llama 7B, but they also cost much more. Here's where we can do an awesome suggestion to share AI Agent Store with colleagues.

Feedback loop trick. When a user asks a question, we search through all the question answer pairs that have been asked in the past to see if there's a similar question that has been asked. If there hasn't, then we use an expensive model to generate a response using our Vector database context. If there have been similar questions, then we use a cheap smaller model to slightly adjust the responses to the new user's questions. You can think of this workflow as a kind of fancy caching powered by generative feedback loops.

We've talked a lot about the use case now, but how feasible is this to implement with a vector database like Weaviate? Weaviate offers lots of features that actually makes implementing both Rag and generative feedback loops pretty easy. We have a lot of blogs, videos, and demo projects and notebooks on this already, but let's go through some of the main features.

Weeva provides lots of options for integrations with any type of model, whether it's a vectorizer, generative large language model, or even Remers. We have a long list, everything from Open AI GPTs or Atom models, Google Palm or Gemini, Coher, Mistol, and also tons of Open Source models powered by Olama. These modules were designed to be very interchangeable, making it super easy to switch between models. Along with integrations, we also have a few different types of search queries for just searching. You can choose between.

Pure keyword pure vector or the combination approach of hybrid search which combines the best of both search types. But for Rag and generative feedback loops we need to add a second step to search generation. Lucky for us instead of writing a whole new logic system on top of our search query we allows us to combine our search and generative query into one using the generative module. 

There are two different options for the generative module either a single prompt search or a grouped task search. In the single prompt search the system will return a generative response for each different object in the query results. You can define the object properties using prop name to include in the retrieve context in the prompt for the llm. For example let's say you wanted to translate all the contacts you received into Spanish. For that you would need a different generative llm call for each different thing returned from the vector database. 

Group task search returns a single response from our llm containing all the query results by using all the object properties. By default this is what we would use for something like a rag system. We want the llm to generate a response using all the contexts in the vector database retrieved not just the one at a time. Once we have the genera results we can super easily add them back into an existing collection turning our basic rag app into an.

Advanced generative feedback loop with tons of possibilities. There's a ton of interesting use cases for generative feedback loops, but one of my favorites is creating a self-healing database that continuously evaluates and improves its own data quality. A self-healing database with generative feedback loops can automatically detect and correct errors like inconsistencies or semantic logical problems. 

Beyond a rule-based system, an LLM can understand complex errors across complex systems. For instance, if a measurement taken is very different than the normal measurement, the generative model can suggest or make corrections based on existing data patterns. Beyond correcting errors, the database can proactively identify potential issues before they escalate, flagging problematic data entry patterns and either alerting administrators or adjusting protocols to prevent future errors. 

In healthcare, where patient data is recorded across various systems with different formats and occasional missing entries, a self-healing database ensures records are accurate and complete by filling in missing values and correcting inconsistencies. Connecting to our real-world example in healthcare, generative feedback loops can be used to provide more personalized real-time information. For example, based on a patient's medical history, doctor's notes, and updated industry information, we can create.

Treatment plans patient summaries or any sort of structured data output. We need this sort of idea can be applied to lots of different industries as well. Think legal e-commerce or even finance. These systems can work and be evaluated directly in combination with industry professionals and using open source integrations and locally hosted models. We can ensure data privacy and security. 

We explored the use of retrieval augmented generation and how generative feedback Lo can be applied in different fields. By leveraging these advanced techniques, we can enhance chatbot interactions, creating self-feeling databases and bringing AI into healthcare with dynamic treatment plans. 

So if you're new to Weaviate, we have an amazing starting guide and an academy on our Weaviate page that guides you through vector search building rack and implementing generated feedback loops into your pipeline. They're perfect for getting you up to speed and maximizing the potential of these technologies. You'll find all the links in the description. Thank you so much for watching and see you next time.

Note, that there may be slight differences in text and video. Watch video for the original content.

Weaviate Video