Optimize and Refine Prompt Templates

The Seven-Step Iteration Cycle

Step 1: Write. Draft your initial prompt template. Don't aim for perfection — aim for "good enough to test." Start with a clear task definition, basic instructions (5–7 key points), essential merge fields, simple format requirements, and one or two key constraints.

Step 2: Test. Run your prompt template with 5–10 diverse real Salesforce records. Include typical cases, edge cases (unusual values, minimal data, maximum data), and records that have been problematic in the past. Save all responses for evaluation and document any errors or failures.

Step 3: Evaluate. Compare results against your success criteria using an evaluation rubric. Score each response: does it meet each criterion, what's the pass rate, and are failures consistent or random?

Step 4: Identify issues. Analyze failures and patterns. Common issue categories include:

• Missing information: Response doesn't include required elements because instructions didn't explicitly require them.
• Wrong tone: Response is too formal, too casual, or too robotic because tone instructions are vague or missing.
• Incorrect length: Response is too long or too short because the length constraint is missing or ignored.
• Poor structure: Response doesn't follow the expected format because format specifications are unclear.
• Inaccurate content: Response includes wrong information because merge fields are incorrect or context is insufficient.
• Inconsistent results: Quality varies dramatically across test cases because instructions don't handle edge cases.

Step 5: Refine. Update your prompt template based on what you learned.

• If information is missing, add explicit instructions listing required elements.
• If tone is wrong, add specific tone guidance with examples.
• If length is wrong, add an explicit length constraint with a word or character count.
• If structure is wrong, add detailed format specifications.
• If content is inaccurate, add more context or fix merge fields.
• If results are inconsistent, add explicit handling for edge cases such as empty fields.

Step 6: Re-test. Run the refined prompt against the same test data you used before. Measure whether the pass rate increased, whether previous failures are now passing, and whether fixing one issue created new ones. Keep iterating until you reach your target pass rate — typically 85–95% for production use.

Step 7: Deploy. Once your prompt consistently produces quality responses, deploy it to production.

Iteration Best Practices

• Keep a version history. Save each version of your prompt with notes about what changed and why, along with the pass rate for that version.
• Use the same test data across versions. This lets you directly compare improvements.
• Make one change at a time when possible. If you change three things and quality improves, you don't know which change helped. Early iterations can batch multiple fixes, but later iterations should make targeted changes.
• Document what didn't work. Note approaches you tried that made things worse. This saves time later.
• Share learnings with your team. What you learn optimizing one prompt often applies to others.

Common Iteration Mistakes

• Giving up too early. Expect 3–5 iterations before reaching production quality. Budget time for this process.
• Not testing with real data. Production data is messy. Always use actual Salesforce records, not made-up test cases.
• Changing too many things at once. In later iterations, make targeted changes so you can identify what works.
• Ignoring edge cases. Include edge cases in your test data and handle them explicitly in your instructions.
• No success criteria. Define 2–3 specific, measurable success criteria before you start iterating. Without clear goals, iteration is aimless.

Optimization Checklist

Before considering a prompt template ready for production, verify these items.

• Tested with at least 15–20 real Salesforce records
• Pass rate meets target for this use case (typically 85–95%)
• Edge cases are handled appropriately
• All success criteria are met consistently
• Version history is documented with pass rates and change notes
• Refinements show measurable improvement over initial version
• Remaining issues are rare and low-impact
• Team has reviewed and approved