You are here:
Use Test Results
After a test run completes, review evaluation scores to identify where your agent or prompt template is falling short. Understand what each scorer measures, how scores are calculated, and what to do when results are low .
When a test run completes, results appear as scores in the test suite grid one row per test case, one column per scorer. Pay close attention to recurring failures: they often point to underlying issues like unclear instructions, missing context, or configuration problems.
For agent testing, manually retest failed utterances in Agentforce Builder and refine subagent instructions, agent actions, and knowledge until results improve. For prompt template testing, use the scores to identify where the template’s output falls short, then go to Prompt Builder to adjust your template instructions accordingly.
Agent Default Evaluations
Agent testing includes a set of default evaluations that run automatically on every test, plus optional quality evaluations you can add. Default evaluations are given a score of 1, pass, or 0, fail.
| Evaluation | Testing Significance | Recommendation |
|---|---|---|
| Response Evaluation | Scores of 3 to 5 indicates that the agent generally succeeds in achieving its goal. A score of 5 reflects a precise, complete, and brand-aligned response with no irrelevant content. Scores of 3 to 4 show decreasing levels of clarity and completeness. There may be minor omissions or partial understanding which introduces some ambiguity while the core intent is still addressed. Scores of 1 to 2 suggest the agent struggled to meet the goal. The response may be unclear, missing key elements, or include irrelevant information. It might also ask the user for information that should have been retrieved from the CRM. A score of 0 represents a complete failure to resolve the user’s query. The response is generic and doesn’t address the user’s intent. |
Check your agent configuration for issues related to subagent selection, instructions, or actions. Consider whether there are knowledge gaps, like outdated articles, or other issues preventing the agent from responding appropriately. |
| Subagent Assertion | A score of 1 indicates that the agent correctly identified the appropriate subagent to address an utterance. A score of 0 indicates that the agent selected an unexpected subagent to address an utterance. |
Manually retest any failed utterance in Agentforce Builder and review the agent's reasoning in the plan canvas. Refine the instructions for both the expected action and the subagent itself to clearly guide the agent toward the correct choice and restrict the use of the incorrect action. |
| Action Assertion | A score of 1 indicates that the agent correctly identified all of the appropriate actions within a subagent to address an utterance. A score of 0 indicates that the agent either chose the wrong actions or failed to select all necessary actions within a subagent to address an utterance. |
Manually retest any failed utterance in Agentforce Builder and review the agent's reasoning in the plan canvas. Refine the instructions for both the expected action and the subagent itself to clearly guide the agent toward the correct choice and restrict the use of the incorrect action. |
Response quality evaluations use an LLM judge to score responses on a 0–5 scale, with 3 or higher signifying a pass. Some evaluations apply to both testing types; others apply only to one. The reasoning behind each score is available in the detail panel when you select a result.
What is an LLM-as-judge?
LLM as Judge is when one large language model (LLM) evaluates the outputs of another LLM, serving as a scalable, automated, and objective evaluation tool for tasks like scoring summaries or ranking responses. A "judge" LLM receives a prompt that includes the task and evaluation criteria like, factual accuracy, relevance, coherence, and faithfulness to source. With these resources and guidelines, the LLM judge determines the expected response and compares it to the agent response then generates scores, rankings, or textual feedback. We’ve carefully designed our LLM-as-judge prompts to give you the most accurate and useful test results.
| Scorer | Agent Testing | Prompt Template Testing |
|---|---|---|
| Completeness | Scored 0–5. A score of 5 indicates a fully complete and accurate answer with no important omissions. Scores of 4 and 3 reflect decreasing levels of completeness, with minor to moderate gaps. Scores of 1 or 2 indicate a significantly incomplete answer. A score of 0 means the answer missed all important information. |
Scored 0–5. Same scale as agent testing. Pass or fail is returned based on whether the response fully addresses the query. |
| Coherence | Scored 0–5. Scores of 3–5 indicate the response transforms underlying information into clear, conversational language. Scores of 0–2 indicate the response delivered raw data, such as JSON structures or direct field content, rather than natural language. |
Scored 0–5. Same scale as agent testing. Pass or fail based on whether the response is logically structured and free of grammatical errors. |
| Conciseness | Scored 0–5. Scores of 3–5 indicate the response is appropriately brief and accurate. Scores of 0–2 indicate the response is lengthy, repetitive, or contains irrelevant content. |
Scored 0–5. Same scale as agent testing. Pass or fail based on whether the response avoids unnecessary length. |
| Factuality | — | Scored 0–5. Scores of 3–5 indicate the response contains accurate, well-supported information. Scores of 1–2 indicate errors or unsupported claims. A score of 0 indicates significantly inaccurate or fabricated content. |
| Latency | Not scored. A high or unusual latency suggests a problem with the utterance or underlying infrastructure. If agent adjustments don’t resolve the issue, contact Salesforce support. |
— |
| Instruction Adherence / Instruction Following | Scored as High, Low, or Uncertain. High: The agent fully follows the subagent instructions, addressing key points and providing required information. Low: The agent fails to follow at least one instruction, leading to an incorrect response. Refine instructions and set clearer constraints. Uncertain: Adherence can’t be conclusively determined due to an ambiguous response, incomplete response, or conflicting interpretations of the subagent instructions. |
Scored 1, 3, or 5. 5 (High): The response fully followed the prompt template instructions. 3 (Uncertain): Adherence can’t be conclusively determined. 1 (Low): The response deviated from the instructions. |
| Response Match | — | Scored 0–5. Measures how closely the response matches the expected output you provided in your CSV. A higher score indicates a closer match. Requires an Expected Output column in your test case CSV. |
Quality evaluations are assessed by the LLM judge based on fixed criteria. If a response receives a low score, review it carefully to see whether it truly falls short of your expectations. For example, the LLM judge may assign a low conciseness score, but you might feel that additional context better serves your customers or aligns with your brand voice. You can add extra instructions to better tailor the agent or prompt template to your goals. Always consider your specific goals and use case when interpreting scores. Quality evaluations provide guidance, but they are not absolute measures of success or failure.
