Print this page

Success Insights: Marketing Cloud - Testing your Emails for Improved Metrics

Knowledge Article Number 000231073

Success Insights: Testing for Improved Metrics

  Goals of Testing

  1. Improve the overall user experience in ways that can be verified by response data

  2. Create a learning culture to foster continuous improvement and know what resonates with your customers
  3. Increase conversions through subscriber engagement with emails (opens, clicks)
  4. Reduce Attrition (Unsubscribers, complaints)
  5. Increase subscriber satisfaction

Variable Recommendations



Level of Effort

Potential Impact

Subject Lines: CopyLowLowMedium
Subject Lines: SymbolsLowLowLow
Subject Lines: PersonalizationMediumMediumMedium
Personalization: Dynamic ContentHighHighHigh
Day of WeekLowLowMedium
Preheader: CopyLowLowLow
Countdown ClocksLowLowMedium
List SegmentationMediumMediumHigh
Call to Actions (Buttons, Text)MediumMediumMedium
Responsive DesignMediumMediumHigh

Testing Best Practices


  • Plan for regular testing: develop a calendar but adjust as test results yield new theories
  • Understand the lag between getting results and applying them to messages in production
  • Determine the statisically significant sample size
  • Isolate one variable; hold all others constant. (The variable can be overall treatment) Once the winner is chosen, isolate specific components to test and optimize.

Review and Refine:

  • Ensure differences are statistically significant
  • To perserve institutional memory, capture and share test results regularly
  • Keep all information in a central file
  • Create a report with background, results, key discoveries, recommendations and screen captures

Important Questions:

  • What other theories spring up as a result of the test?
  • Should we retest?
  • Where factors cannot be held constant (seasonality), will we need to retest to verify results?
  • Can we be certain that the lift was from the test factors: does correlation = causation?
  • Can we state the conclusion of the test with confidence?
In conclusion - testing is the fastest and easiest way to give your customers the power to shape the communication you send to them. By creating a plan and being intentional about constant analysis - you can ensure you are sending your subscribers the right message and at the right time. 

promote demote