You are here:
Retrieval Response with Enriched Search Index
Enriched indexing enhances retrieval response by matching a question to a passage with the best possible answer in your knowledge base. While Data 360 hybrid search combines the strengths of sematically aware vector search with the precision and speed of keyword search, enriched indexing closes the gap further by finding the chunks closest to the users' question.
Enriched Indexing and Retrieval Augmented Generation (RAG)
Enriched indexing closes the gap in two ways. First, it generates questions based on the original plain chunk – a question chunk–so vector search finds a near-perfect alternate question for the user question. Second, it extracts a rich metadata chunk for each original plain chunk — a title, summary, keywords, named entities, and topic categories. Because Data 360 embeds the metadata chunk as a vector, it improves both vector and keyword search scores. The distilled, concept-rich text gives vector search a better semantic match, and the explicit keywords help keyword matching. Both the derived chunks point back to the original plain chunk as the source at time of retrieval.
If a search retrieves a plain chunk more than once, the retrieval process retains only one instance of the plain chunk. This duplication occurs when multiple enriched chunks point back to the same plain chunk as the source chunk.
The Response: Your knowledge base includes an article with the right response. The article has a section written as a procedure, “Navigate to the recovery page and enter your email address.”
The Gap: The passage written as a procedure has a different meaning than the question. Data 360 ingests and indexes the knowledge article. During the retrieval process, this chunk can score much lower than expected because of this gap. As a result, the retrieval sometimes excludes this passage.
When you turn on enriched indexing using Large Language Models (LLMs), Data 360 creates a metadata chunk and a question chunk for the original plain chunk that includes this procedure content.
The metadata chunk includes rich metadata about the plain chunk, such as:
Title: "Password recovery"
Summary: "Steps to reset login credentials via the recovery page."
Keywords: recovery, credentials, reset link, email
Entities: recovery page, registered email (named things in the passage)
Topics: account management, authentication
The Questions chunk includes natural language questions that the plain chunk answers such as:
“How do I reset my password?"
"What do I do if I forget my login?"
"How long is a password reset link valid?"
"Where do I go to recover account access?"
All three chunks are part of the same index. So, when you search for a response any chunk type can match. In this example, the original query matches the question chunk. Data 360 search index returns the original plain chunk, the actual document text and not the derived enriched chunks. The original plain chunk that contains the right answers scores higher than other matches.
