Understanding Query Analysis

Understanding Query Analysis

Query Analysis is a set of techniques that helps optimize retriever search queries. Vector stores, by and large, get the fundamentals right with their in-built search implementations. But they are usually insufficient. Their search and re-ranking capabilities are found lacking in various scenarios.

Consider the following use cases:

For instance, retrievers search through documents but not the metadata attached to the documents. For a query that references metadata such as a date field, the search result won’t be able to filter by the date because it’s not found in the document itself.

Single queries are known to result in insufficient responses. You must ask many questions to get the right answers. This is somewhat related to the fact that retrieved relevant documents usually contain irrelevant information, too.

For queries that contain multiple questions, the search likely will return documents that answer the first or last question. They’re designed by default to answer one question at a time. It’s difficult for them to reconcile how to return results, especially when the questions aren’t related.

Vector-store searches struggle to get the right meanings of queries that have phrases and other literary components. They might be interpreted literally, which could lead to erroneous or irrelevant results.

Consider this last scenario in which multiple retrievers are involved in responding to a query. It becomes difficult to maintain proper context and ranking in such a situation.

All these constitute reasons why query analysis is necessary. The primary idea with most query analysis techniques is to refine the queries before conducting a final search. Other techniques could involve ranking the results for each of the regenerated queries to find the best possible response to the question. Other techniques also specialize in identifying phrases and keywords to maintain the right context during the search.

Below are several techniques LangChain supports:

• Query structuring: This LangChain technique identifies documents and their attributes to intelligently search through attributes when the queries refer to them.

• Query decomposition: This technique separates questions in an input into individual questions, which will be processed independently.

• Query expansion: For documents and queries that contain special phrases and keywords, LangChain uses query expansion to break down these phrases into simpler words, thus improving the search results.

• Query routing: LangChain uses this technique to direct queries to the relevant documents instead of searching through all available ones.

• Step back prompting: Sometimes, search quality and model generations can be tripped up by the specifics of a question. One way to handle this is to first generate a more abstract, “step back” question and to query based on both the original and step-back question.

Enhancing RAG Systems

To perfect a RAG app is quite involved. Apart from RAGs having many moving parts, each basic component also has multiple refinements you could apply to it. It’s fair to say that the solutions aren’t finite and probably never will be. What matters most is constant evaluation and polishing until it reaches acceptable levels based on the use case. For generic applications, the basic implementations are good enough. For others, many refinement techniques will be required to make them fit for purpose.

Below are several techniques that can significantly improve your RAG:

• Iterative LLM refinement: Because the LLM is a major part of a RAG, a refined LLM is key to improving your RAG. You can check out LLM leaderboards or look for LLMs with features such as self-critiquing.

• Speculative RAG: This is a RAG framework that uses a smaller specialist language model to generate draft texts for the generator (LLM) to verify and select the best draft. Speculative RAGs have high accuracy and efficiency.

• Query analysis: This optimizes queries through regeneration by LLMs before using them for retriever searches. It removes ambiguity, thus enhancing the original query. It’s effective at solving the single-query problem — that is to say, you’ll be able to get more relevant results with a single query.

• Knowledge filter: This focuses on trimming excesses from generated responses, thus improving accuracy.

• Memory knowledge reservoir: This technique involves caching previously retrieved knowledge to help speed subsequent similar queries.

• Re-ranking: Re-ranking is one of the most popular techniques for enhancing RAG performance. It assesses and reorders retrieved documents based on relevance.

• Hyper-parameter tuning: Hyper-parameters like chunk size, overlap, temperature, and the number of top retrieved documents can boost your RAG’s performance significantly. These parameters ensure high relevance in captured data, retained context, and improved searches.

• Embedding models: The right embedding model can make or break your RAG. Embedding is also a fundamental component in LLMs. Domain-specific models with the right parameters improve similarity searches and the overall performance of your RAG app.

• Fine-tuning: Fine-tuning involves training the LLM on carefully selected and structured datasets by domain experts on how to identify relevant information from retrieved content. This, though expensive, yields high performance when used in RAGs.

• Knowledge graph: This technique combines the power of structured graph data with unstructured vector searches to achieve accurate search results. Graph databases handle depth and relationships excellently, so this combination ensures that proper context is retained during the search. It’s also highly unlikely to ignore any relevant piece of data irrespective of where it’s located due to the ability of graphs to handle relationships.

• Summarization: Summarization extracts key information from responses based on the given context. By so doing, it removes noise from generated responses.

These are not all the techniques available for enhancing RAGs. Some are computationally expensive, and others require domain experts, which can be hard to find. Some use cases might benefit more from some techniques than others. Some techniques, though effective, are slow and thus cannot be used in real-time scenarios. Re-ranking is the most-used technique among the lot, followed by hyper-parameter tuning. This field is constantly evolving, so do check back occasionally for updated content.

In the next section, you’ll use one of the techniques above to enhance SportsBuddy. Excited? Move to the next segment.