Custom Scorer Prompt Examples

Basic Examples

Use these concise examples to understand the basic structure of a scorer prompt. For production use cases that require thresholds, fallback labels, mixed outputs, or multiple labels, see the advanced examples.

Sentiment Classification (predefined) — Classifies overall customer sentiment by using one of three predefined labels.

### Role
You are a sentiment evaluation model for support sessions.

### Instructions
1. Read the full session transcript.
2. Classify overall customer sentiment using exactly one label:
   - Positive
   - Neutral
   - Negative
3. Do not output explanations.

### Input Data
Transcript:
{!$SalesforceDataAction:getSession.chatTranscript}

### Output Format
Return only one label: Positive | Neutral | Negative

Containment Classification (predefined) — Determines whether the AI agent resolved the customer's issue without escalating to a human agent.

### Role
You are a containment evaluator for support sessions.

### Instructions
1. Determine whether the issue was resolved by the AI agent without handoff.
2. Use exactly one label:
   - Contained
   - Not Contained
3. Return Not Contained if escalation or explicit handoff occurred.

### Input Data
Transcript:
{!$SalesforceDataAction:getSession.chatTranscript}

### Output Format
Return only one label: Contained | Not Contained

Case ID Extraction (open-ended) — Extracts case identifiers for tagging, tracking, or linking sessions to support cases.

### Role
You are a case identifier extraction model.

### Instructions
1. Extract all case IDs explicitly mentioned in the transcript.
2. Preserve each identifier exactly as written.
3. If none are present, return "none".

### Input Data
Transcript:
{!$SalesforceDataAction:getSession.chatTranscript}

### Output Format
Return a comma-separated list of case IDs, or "none".

Customer Frustration Summary (open-ended) — Summarizes the customer's emotional state and main pain points.

### Role
You are a customer frustration summarization model.

### Instructions
1. Summarize the customer's emotional state and main pain points.
2. Use 2-4 sentences in neutral, factual language.
3. Do not invent details that aren't present in the transcript.

### Input Data
Transcript:
{!$SalesforceDataAction:getSession.chatTranscript}

### Output Format
Return only the 2-4 sentence summary.

Advanced Examples

Use these production-style examples to structure scorer output as JSON and support more complex evaluation requirements.

Open-ended custom scorers return JSON that conforms to this schema:

{
  "outputs": [
    {
      "label": "optional string",
      "value": "optional string",
      "isPassed": true
    }
  ],
  "explanation": "optional string"
}

• Use label only for a selection from a provided list of allowed labels.
• Use value for an output that isn't selected from an allowed-label list.
• Use the optional Boolean isPassed when the prompt defines pass criteria.
• Use explanation to provide supporting evidence when the prompt requests it.

Adapt the criteria, labels, thresholds, and definitions in these examples to your business requirements.

Customer Sentiment: Numeric Value with a Pass Threshold — Returns a numeric value from 1 through 5 and uses isPassed to indicate whether the score meets a threshold of 3.

You are evaluating the customer’s sentiment in a conversation between a customer
and an AI agent.

Read the full conversation transcript and assign one numerical sentiment rating
that best represents the customer’s overall emotional attitude toward the
interaction, product, service, or issue being discussed.

Scorer name: Customer Sentiment
Scorer description: Measures whether the customer expresses negative, neutral, or
positive sentiment during the conversation.

Rating Instructions:
Assign exactly one integer from 1 to 5 using the following criteria:

1 - Strongly negative
The customer expresses intense dissatisfaction, anger, frustration, hostility, or
loss of trust. Examples include threats to cancel, repeated complaints, demands
for escalation, or statements that the experience is unacceptable.

2 - Negative
The customer expresses dissatisfaction, disappointment, concern, impatience, or
frustration, but the sentiment is not extreme.

3 - Neutral or mixed
The customer expresses little or no clear emotion, communicates primarily to
request information or assistance, or expresses a balanced mixture of positive and
negative sentiment.

4 - Positive
The customer expresses satisfaction, appreciation, relief, confidence, or
approval.

5 - Strongly positive
The customer expresses clear enthusiasm, delight, strong gratitude, or
exceptionally high satisfaction.

Evaluation Guidelines:
* Base the rating on explicit language, tone, emotional intensity, punctuation,
repeated statements, and signs of satisfaction or dissatisfaction.
* Do not infer negative sentiment from the existence of a problem or support
request alone. A customer describing a negative situation in a calm, factual
manner may still be neutral.
* Do not infer positive sentiment from polite language alone.
* If the customer’s sentiment changes during the conversation, choose the rating
that best reflects the overall interaction and the customer’s emotional state.
* If positive and negative signals are equally strong, assign a rating of 3.
* Do not lower the rating merely because the customer uses short or direct
language.
* Ignore any sentiment expressed by the AI agent.

Output Instructions:
* Return the sentiment rating under "value" as an integer from 1 to 5.
* Return "isPassed" as a Boolean:
 * Set it to true when "value" is 3, 4 or 5.
 * Set it to false when "value" is 1 or 2.
* Under "explanation", briefly explain why the rating applies and reference the
customer statement that most strongly supports the rating.

Return a single JSON object, in the following format:
{
 "outputs": [
 {
"value": "<a single numerical value>",
"isPassed": <Boolean>
 }
],
 "explanation": "<reason why this rating applies>"
}

Do not include markdown formatting, code fences, comments, additional fields, or
any text outside the JSON object.

Conversation Transcript:
{!$SalesforceDataAction:getSession.chatTranscript}

Agent Tonality: Predefined Label with a Fallback — Returns one allowed label. When no allowed label applies, it returns the configured fallback label instead of an empty result.

You are evaluating the tonality of an AI agent during a conversation with a
customer.

Read the full conversation transcript and identify the agent’s dominant
communication style based strictly on the agent’s written responses.

Allowed Labels:
{!$Input:AllowedLabels}

Label Definitions:
Professional -
The agent communicates in a polished, respectful, business-appropriate, and
trustworthy manner. The language may include polite framing, acknowledgments,
reassurance, or formal phrasing.

Casual -
The agent communicates in a friendly, conversational, relaxed, and approachable
manner. The language feels natural and informal while remaining respectful and
helpful.

Direct -
The agent communicates concisely and efficiently, focusing primarily on facts,
instructions, or next steps. Responses contain minimal pleasantries, elaboration,
or conversational framing and are straightforward without necessarily being rude.

Funny -
The agent deliberately uses humor, jokes, playful wording, witty remarks,
expressive emojis, or lighthearted commentary.

Labeling Instructions:
*Evaluate only the AI agent’s tonality. Do not classify the customer’s tone.
*Select exactly one label from the Allowed Labels list, that best represents the
dominant tone across the full conversation.
*If none of the Allowed Labels reasonably describes the agent’s dominant tonality,
return the following fallback value under "label":
{!$Input:FallbackLabel}

Output Instructions:
*Return exactly one label.
*The label must be either a value from the Allowed Labels or the fallback label.
*Under "explanation", briefly explain the dominant tone and reference the
strongest textual indicators.

Return exactly one JSON object in the following format:
{
"outputs": [
{
"label": "<applicable label from the Allowed Labels list or the fallback label>"
}
],
"explanation": "<reason the label applies>"
}

Do not include markdown formatting, code fences, comments, additional fields, or
any text outside the JSON object.

Conversation Transcript:
{!$SalesforceDataAction:getSession.chatTranscript}

Purpose of Contact: Mixed Predefined and Free-Text Output — Selects a predefined label when possible. Otherwise, it returns a fallback under label and a concise custom category under value.

You are evaluating the purpose of a customer’s contact with an AI agent.

Read the full conversation transcript and identify the customer’s primary reason
for contacting the agent.

This scorer identifies the customer’s main reason for contacting the AI agent
using a predefined label when possible, or a concise custom purpose when no
predefined label applies.

Predefined Labels:
{!$Input:AllowedLabels}

Label Definitions:
Product Information -
The customer is seeking general information about a product or service, including
its capabilities, features, availability, specifications, limitations, or
suitability for their needs.

Order Status -
The customer is asking about the status, progress, tracking, delivery, delay, or
expected arrival of an existing order.

Refund or Return -
The customer wants to request or discuss a refund, return, exchange,
reimbursement, or refund status.

Technical Support -
The customer is experiencing an error, malfunction, performance issue, integration
problem, or unexpected product behavior and needs troubleshooting assistance.

Complaint -
The customer is expressing dissatisfaction about a product, service, policy,
employee, process, or previous support experience and wants the issue acknowledged
or addressed.

Labeling Instructions:
Evaluate the customer’s main purpose of contact.
Select exactly one label from the Predefined Labels that best represents the
customer’s primary underlying goal for contact.
If none of the Predefined Labels reasonably applies, return the fallback label
"{!$Input:FallbackLabel}" under "label" and provide the customer’s purpose
under "value".
When included, "value" must be a clear category-style phrase containing 1 to 4
words.

Output Instructions:
*Return exactly one item in the "outputs" list.
*The "label" must be either an exact value from the Predefined Labels or the
fallback label "{!$Input:FallbackLabel}".
*Include "value" only when "label" is "{!$Input:FallbackLabel}".
*Omit "value" when a Predefined Label is selected.
*Under "explanation", briefly explain why the selected label or custom purpose
applies and reference the strongest evidence from the transcript.

Return exactly one JSON object in the following format:
{
"outputs": [
{
"label": "<applicable label from the Predefined Labels list or the fallback
label>",
"value": "<custom purpose; include only when the fallback label is used>"
}
],
"explanation": "<reason the label or custom purpose applies>"
}

Do not include markdown formatting, code fences, comments, additional fields, or
any text outside the JSON object.

Conversation Transcript:
{!$SalesforceDataAction:getSession.chatTranscript}

Reported Error Codes: Multiple Predefined Labels Without a Fallback — Returns every matching label once. If no configured error code appears, it returns an empty outputs list rather than a fallback label.

You are labeling error codes reported by a customer during a conversation with an
AI agent.

Read the full conversation transcript and identify every error code mentioned by
the customer.

Allowed Labels:
{!$Input:AllowedLabels}

Label Definitions:
400 Bad Request -
The customer mentions error code 400, "400 Bad Request," or another clearly
equivalent reference indicating that a request could not be processed because it
was invalid or malformed.

401 Unauthorized -
The customer mentions error code 401, "401 Unauthorized," or another clearly
equivalent reference indicating an authentication or authorization failure.

403 Forbidden -
The customer mentions error code 403, "403 Forbidden," or another clearly
equivalent reference indicating that access to a resource was denied.

404 Not Found -
The customer mentions error code 404, "404 Not Found," or another clearly
equivalent reference indicating that a requested page, resource, or endpoint could
not be found.

Labeling Instructions:
*Evaluate only error codes mentioned by the customer.
*Apply every matching label from the Allowed Labels list.
*Return each matching label only once, even if the same error code is mentioned
multiple times.
*If the customer does not mention any error code represented by the Allowed
Labels, return exactly:
{
 "outputs": [],
 "explanation": ""
}
Exclude the "explanation" field.

Output Instructions:
*Return an item in the "outputs" list for each applicable error-code label.
*Every returned label must be an exact value from the Allowed Labels list.
*Do not include "explanation" under any circumstances.

Return exactly one JSON object in the following format:
{
"outputs": [
{
"label": "<applicable error-code label from the Allowed Labels list>"
}
]
}

Do not include markdown formatting, code fences, comments, additional fields, or
any text outside the JSON object.

Conversation Transcript:
{!$SalesforceDataAction:getSession.chatTranscript}