Test an Ensemble Retriever

Customize an Ensemble Retriever in the Playground

1. In AI Models on the Retrievers tab, select Test Retriever. Alternatively, select Test Retriever for an ensemble retriever from the drop-down action menu.
   Ensemble retrievers that contain Agentforce Data Library (ADL) retrievers or Custom SQL retrievers can't be tested in the playground.
2. Select a mode for testing the retriever. In the basic mode, the retriever playground returns a source recall score and a response time. In the advanced mode, the playground returns an AI-generated context recall score based on an expected answer to a test question. For more information on metrics and costs, see, Retriever Metrics and Results and Billing Considerations for Testing Retrievers.
3. Select an ensemble retriever that you want to test and validate.
   When you search for retrievers, only active retrievers are listed. The retriever configuration panel shows the individual retrievers that are part of the ensemble retriever.
4. To customize or reconfigure the ensemble retriever, select Edit.
5. Select or deselect retrievers by data space.

   An ensemble retriever can contain a maximum of 10 individual retrievers. The Select Retriever tab shows the retrievers that are part of the ensemble.

6. Modify the number of results that you want the retriever to return.
   If the results to a user's question don't include the expected response, increase the number of results returned.
7. Customize the fields returned by the retriever by adding or removing existing fields.
   

   Note The retriever returns these new fields when you save the configuration and use it with agents.
8. (Optional) Turn on Citations Settings to preview and validate citations from individual retrievers.

Test Your Retriever

Configure test data and enter a natural language query to test your retriever. Test data is the desired source from where the correct answer is expected. With ensemble retrievers, the desired source records can be part of different retrievers. In the playground, you can select test data from each individual retriever that is part of the ensemble.

1. To test your retriever responses from expected sources, select Configure Test Data.
2. In the Configure Test Data window, select a retriever.
3. Select data model object if you know which records contain the response or select keyword search.

   For hybrid search retrievers with keyword matches, select test data by keyword search or data model object records. Otherwise, you can only select by data model object records. Select data model object records if you know which records contain the response to your test question. Select records based on a keyword search if you know what your response should include, such as, for a question related to paid time-off, you can enter “PTO or Time-off.”

4. Repeat steps two and three for each retriever that is part of the ensemble.
   You can select up to five test data records for each individual retriever in the ensemble.
5. Ask a test question. In the advanced mode, you can test your retriever responses based on an expected answer to the test question.
   After you run your test, the Test Retriever panel shows the number of documents retrieved by the retriever.