Loading
Feature degradation | Gmail Email delivery failureRead More
About Salesforce Data 360
Table of Contents
Filter by (0)Add
Select Filters

          No results
          No results
          Here are some search tips

          Check the spelling of your keywords.
          Use more general search terms.
          Select fewer filters to broaden your search.

            Search all of Salesforce Help
            Search all of Salesforce Help
            Customize, Test, and Validate Retrievers with Retriever Playground

            You are here:

            1. Salesforce Help
            2. Docs
            3. Data 360

            Customize, Test, and Validate Retrievers with Retriever Playground

            Fine-tune and validate retrieval logic for retrieval-augmented generation (RAG) systems by using retriever playground, reducing deployment risks, and verifying accurate AI outputs. Use AI Models (formerly Einstein Studio) retriever playground to configure pre-filters, select test data, and adjust result parameters to customize retrieval logic for domain-specific use cases, such as retrieving complaince documents or knowledge articles.

            Required Editions

            Available in: All Editions supported by Data 360. See Data 360 edition availability.

            Review intermediate outputs and key metrics, such as the Source Recall Score or Context Recall Score, to validate that the retriever finds information from expected source documents. Fine-tuning retrieval logic in the retriever playground is a cost-effective way to test and debug retrievers. Retriever playground improves agent performance and search relevance, establishing reliable and meaningful generative AI outcomes.

            Use Case: Agent Answers Field Service Questions on Installing Double Door Refrigerators with Incomplete Results

            A business administrator sets up an agent to respond to questions from contractors in the field who install their company refrigerators.

            Problem: Although the company knowledge base includes an article with complete step-by-step installation instructions, the field service provides feedback that responses from the agent are incomplete. The business admin tests the retriever set up for the agent in the retriever playground.

            The test shows that only five steps are retrieved from the eleven-step article.

            In reviewing the retriever results panel, the admin notices that the search index splits the article into two chunks. The top chunk includes the "double-door refrigerator" keyword. Because the second chunk dosen't include the necessary keywords from the question, it's scored lower in the result set and doesn't appear in the response.

            Solution: The admin increases the number of test results to include the chunk that contains the rest of the instructions. Then, the admin can test the new configuration with sample questions about installing double-door refrigerators to make sure that the response is complete.

            • Test a Retriever in Retriever Playground
              Test a retriever configuration to ensure accurate responses from your AI agents. Save retriever configurations as new versions of the retriever in AI Models (formerly Einstein Studio) and activate them for use in agents with prompt templates, Flow, and applications.
            • Retriever Metrics and Results
              Review retriever metrics and the retrieved data chunks to understand the results of your test. Modify the retriever configuration to try different test configurations and test data to get an optimal response. Save your configuration as a new version of the retriever.
            • Billing Considerations for Testing Retrievers
              When you test retrievers by querying search indexes, your data is processed in Data 360. The primary usage type for retriever testing is Data Queries. Use of retriever playground impacts the consumption of credits used for billing in these usage types.

            See Also

             
            Loading
            Salesforce Help | Article