Troubleshooting Knowledge Retrieval for Agents

My agent isn't returning information that exists in the knowledge store.

You’re expecting to see information that you’re certain exists in the knowledge store.

• First, confirm that the information exists in the knowledge store. Use citations to identify the source information used to generate the response.
• Content strategy and quality deficiencies are the most common cause of unsatisfactory knowledge retrievals. Audit your knowledge store to identify and remedy content quality issues associated with the requested information. Look for information that’s incorrect, obsolete, duplicate, overlapping, contradictory, and diffused. Identify ways to improve the clarity, focus, and organization of the information. For content optimization techniques, start with these resources:
  • Help: Prepare Your Source Content
  • Trailhead: Grounding an Agent with Data
  • 360 Blog: 10 Ways To Prep Your Knowledge Base for AI Grounding
• Verify that the user has sufficient permissions to access the information. For Standard objects (including Knowledge Articles), the agent user needs permission to view the data. Otherwise, that data is omitted from the agent’s response.
• If you’re using vector search, try hybrid search and compare retrieval results. Hybrid search can help improve your search results. For each search strategy, use Retriever Playground in AI Models (formerly Einstein Studio) with individual retrievers to score responses to the same query, and then compare them.
• Turn on Enhanced Event Logs to show more conversation details. Review the conversation in the enhanced event logs. Examine the user request and the retrieved content. How well did the agent recognize the user’s intent? Was the user request poorly formed? Was it too specific (narrow), too broad, or too ambiguous to find the right information? How well did the agent find and retrieve the information that pertained to the user’s intent?
• For individual retrievers, use Retriever Playground in AI Models to validate retrievals in a test environment with test/ground truth data. View retrieved chunks, source chunks, and source recall metrics.
• Use Knowledge/RAG Quality Data and Metrics to investigate retrieval quality issues and patterns. Metrics can guide your efforts to focus on the areas that can best increase answer accuracy and relevance. Context Precision scores reflect the quality of the retrieval. Faithfulness and Answer Relevance scores reflect the accuracy and relevance, respectively, of the generated response. See About Knowledge/RAG Quality Data and Metrics.
• Review the chunking strategy in your search indexes, which can greatly impact retrieval quality.
  • Examine the chunks retrieved in the response. Were the right chunks retrieved? Were the chunks too small or too large? How are the chunks ranked? For hybrid search indexes, tune the ranking factor settings (see Create a Hybrid Search Index with Advanced Setup).
  • Examine the chunks in the search index, as described later in this article. To inspect the data chunks in the search index, or to ask natural language query questions, you can use the Data 360 Query Editor or any SQL tool. Is the relevant information split across multiple chunks? Is important context lost in the chunking?
  • Evaluate and adjust your chunking strategy, such as changing the chunk size or using section-aware chunking. See Chunking Strategies, Optimizing Search Indexes: Field Selection and Chunking, and Enriched Indexing of Content Chunks in Data 360 help.
• In Prompt Builder (or a flow), compare the retrieved chunks in the prompt with the LLM’s response. What’s missing in the response? Use citations to identify the sources that the LLM used to generate the response. Is an important retrieved chunk ranked too low? Consider adjusting the instructions in the LLM prompt.

My agent returns incorrect or hallucinated information.

You’re seeing LLM responses that contain factually incorrect information.

• Examine the retrieved chunks. Are they semantically similar but factually wrong? Do they contain information that’s obsolete, contradictory, or incorrect? Were more accurate or relevant chunks not retrieved? Address content quality issues in the knowledge store. Add context to chunks by using prepend fields (see Optimizing Search Indexes: Field Selection and Chunking) and enriched indexing (see Enriched Indexing of Content Chunks). Increase the number of retrieval results (chunks) returned to the prompt.
• Examine the retrieved chunks and the associated source content. Was the source content parsed correctly? Are headings, paragraphs, and tables intact or scattered across chunks? If the latter, switch to processing your content with LLM-based Parsing and Preprocessing.
• Examine whether the LLM derived the answer from retrieved chunks or made up the information. If the latter, adjust the prompt to require citations.
• Use Knowledge/RAG Quality Data and Metrics to investigate quality issues and patterns. Faithfulness scores reflect the accuracy of the generated response.

My agent isn’t responding with recently updated information.

You’re seeing old information, not the latest information that you know was recently updated.

• For large knowledge stores, building a search index can take hours. Check the job status of the search index and the status state (see View Search Index Status and Process History ). Proactively monitor indexing jobs for progress and completion result (Succeeded or Failed). Use multiple search indexes separated by content type or domain.
• For hybrid search indexes, select RECENCY as the ranking factor and select the LastModifiedDate field in the Data Model Object (DMO). See Create a Hybrid Search Index with Advanced Setup.
• Check the status of the Knowledge Article in Salesforce. Archived articles are sometimes still indexed, which can result in contradictory, duplicate, or obsolete information. Add prefilters to the retriever to make sure that only the active articles are returned.
• Check for errors in indexing logs. Rebuild the index if the data stream stalled.

My documents aren’t being indexed at all.

Your documents aren’t appearing in the search index.

• Investigate which documents are missing from the index. Examine whether file attachments for Salesforce objects are configured for ingestion (is ContentDocument included in the data source?). Review your data source configuration and permissions and address setup issues.
• Check the document file format. If it’s not a supported format, convert the document to a supported file format. Refer to the file limits for unstructured data in Data 360.
• Check Salesforce object permissions and change permissions if needed. Make sure that the agent user has access to the required objects and Knowledge. Check data category filters and adjust filters as needed.
• Determine whether all DMO records have been indexed, or whether the incremental update of a search index has failed. Compare the results of record count using a query similar to the following example:

  SELECT 'INDEX' AS Location, COUNT(DISTINCT rc.SourceRecordId__c) AS ArticleCount, now() AS Timestamp 
  FROM
  <chunk DMO of the Search Index> rc
  UNION
  SELECT 'DMO' AS Location, COUNT(DISTINCT  kav.Id__c)  AS ArticleCount, now() AS Timestamp 
  FROM <DMO that was indexed, e.g. Knowledge Article Version> kav
  ORDER BY Location;

Table data isn’t being extracted correctly.

You're seeing errors in the extracted data.

• To avoid horizontal versus vertical detection errors, switch to processing and extracting your content using the Docling parser (see the blog: Turn Hidden Data into AI-Ready Insights: How the Docling Parser Harvests Value from Unstructured Information). For other parsing options, see LLM-based Parsing and Preprocessing.
• Examine the chunks and determine whether table information is intact or scattered across multiple chunks. If the latter, increase the chunk size or implement a section-aware chunking strategy.

Multi-lingual queries aren’t working.

You're getting poor agent responses to user queries submitted in different languages. For example, your knowledge store contains Spanish content and your user queries are in French and English.

• For RAG solutions involving non-English languages, make sure you’re using the multilingual-e5-large embedding model. This embedding model preserves semantic similarity across languages. A query in French can retrieve relevant articles written in Spanish. For the best results, consider using a single language for content in the knowledge store.
• Keyword search is less effective in multi-lingual solutions. If keywords are in a different language than the source content, test whether vector search performs better than hybrid search. For each search strategy, use Retriever Playground in AI Models with individual retrievers to score responses to the same query, and then compare them.
• Users expect responses to be in the same language as their user queries. Configured settings can affect responses. For example, instructions in a prompt template can specify to the LLM to generate a response in a specific language. To improve results in multi-lingual solutions, review supported languages and configured language settings for agents, prompt templates, and Einstein Trust Layer.
• LLMs and LLM versions can vary in multi-lingual support. Some perform better than others. To get the best results, test your prompts with different LLMs and LLM versions.

Other Considerations

• To help with debugging your RAG-powered solution, turn on Enhanced Event Logs to show more conversation details.
• Before you deploy your RAG-powered solution into production, use Retriever Playground in AI Models to help answer the important RAG questions. Does the right document exist? Were the most relevant chunks surfaced? Use no-code, individual retrievers to test and improve the quality of your responses. Tune retriever settings, examine intermediate outputs, score the quality of the retrieval results, and make adjustments to increase retrieval accuracy and relevance.
• Use Knowledge/RAG Quality Data and Metrics to investigate retrieval quality issues and patterns. Metrics can guide your efforts to focus on the areas that can best increase answer accuracy and relevance. Context Precision scores reflect the quality of the retrieval. Faithfulness and Answer Relevance scores reflect the accuracy and relevance, respectively, of the generated response. See About Knowledge/RAG Quality Data and Metrics.
• If you’re using vector search, try hybrid search and compare retrieval results. For each search strategy, use Retriever Playground in AI Models with no-code, individual retrievers to score responses to the same query, and then compare them.
• Filters can focus searches on the most relevant information but they can also preclude relevant content from retrieval. Make sure that your prefilter configurations in prompt templates and filter settings in retrievers are optimized for retrieving the right amount of content.
• Tune your chunking setup in your search indexes to optimize retrieval quality. See Chunking Strategies, Optimizing Search Indexes: Field Selection and Chunking, and Enriched Indexing of Content Chunks in Data 360 help.

Troubleshooting Solution Layers

Another way to investigate your RAG solution is to examine search and retrieval results in different layers of the solution stack.

Step 1: Investigate the Agent Layer

If you remedy the issue in the agent layer, you don’t need to troubleshoot RAG components. Use the Agentforce Builder or Agentforce Testing Center to investigate and diagnose the reasoning process.

For example, determine whether the Atlas reasoning engine is executing the right action within the right subagent. If the right subagent isn’t selected, or if the right subagent is selected but the right action isn’t run, then the problem most likely occurs in the agent configuration of the instructions and classification descriptions. Investigate and remedy the subagent instructions and action instructions as appropriate.

For other agent-related issues, see Troubleshooting Agents.

Step 2: Investigate Data Libraries

If your RAG-powered solution includes Agentforce Data Library, confirm permissions and check the data library status. See Agentforce Data Library Troubleshooting.

Step 3: Investigate the Search Index and Retrieval Layer

Determine whether the search index contains the content you expect.

• Use the Retriever Playground in AI Models with no-code, individual retrievers to test your retriever and investigate the returned chunks. Identify symptoms and patterns and remedy them accordingly.
• In Prompt Builder, use a prompt template with an embedded retriever to experiment with different prompts. Use preview mode to see what the retriever returns. Look for patterns of errors, omissions, and other anomalies.
• In the Data 360 Query Editor, run queries against the Index DMO. Be sure to use a LIMIT 10 or similar statement. For examples, see "Use the Data 360 Query Editor to Debug Chunks" later in this topic.

Step 4: Investigate the Generation Layer

Examine how the LLM takes the chunks in your grounded prompts and generates an answer.

• In Prompt Builder, compare the retrieved chunks in the prompt with the LLM’s response. What’s incorrect, missing, or hallucinated in the response? Use citations to identify the sources that the LLM used to generate the response. Is an important retrieved chunk ranked too low? Consider adjusting the instructions in the LLM prompt.
• See the recommendations in “My agent isn't returning information that exists in the knowledge store” earlier in this article.

Use the Data 360 Query Editor to Debug Chunks

Directly inspect and troubleshoot chunks in your search indexes with the Query Editor.

Find Chunks by Source Record

Examine all chunks created from a specific source (like an URL or Knowledge Article).

SELECT Chunk__c, ChunkSequenceNumber__c, DataSource__c, DataSourceObject__c, 
    InternalOrganization__c, KQ_RecordId__c, KQ_SourceRecordId__c, 
    RecordId__c, SourceRecordId__c 
FROM <Your_Chunk_DMO>__dlm 
WHERE SourceRecordId__c = '<source_record_id>'

Test Vector Search Queries

Examine the retrieved chunks for a given query.

SELECT v.score__c AS Score, c.Chunk__c AS Chunk, 
        c.SourceRecordId__c AS SourceRecordId
FROM vector_search(
  TABLE(<Your_Index>__dlm), 
     'your search query here', 
     '', 
     10
     ) v 
INNER JOIN <Your_Chunk_DMO>__dlm c 
   ON c.RecordId__c = v.RecordId__c
   ORDER BY Score DESC 
LIMIT 10

Test Hybrid Search (Vector + Keyword)

SELECT c.Chunk__c, h.hybrid_score__c, h.keyword_score__c, h.vector_score__c
FROM hybrid_search(
  TABLE(<Your_Index>__dlm),
      'your query here',
      '',
      100
      ) h
JOIN <Your_Chunk_DMO>__dlm c 
   ON h.SourceRecordId__c = c.RecordId__c
ORDER BY h.hybrid_score__c DESC            
LIMIT 10

Debug Chunk Creation

Examine whether chunks were created for specific content.

SELECT COUNT(*), SourceRecordId__c
FROM <Your_Chunk_DMO>__dlm
GROUP BY SourceRecordId__c

Find Duplicate Chunks

SELECT COUNT(RecordId__c) AS duplicates, SourceRecordId__c, Chunk__c  
FROM <Your_Chunk_DMO>__dlm  
GROUP BY SourceRecordId__c, Chunk__c  HAVING COUNT(RecordId__c) > 1