Einstein Engagement Scoring Model Confidence

Model confidence is a way to express the relative strength of a prediction generated by a model. Understanding the concept can help you interpret Einstein Engagement Scoring for email model confidence scores.

Model confidence provides the probability that a random positive sample has a higher probability of being positive than a random negative sample. Marketing Cloud Engagement uses the area under the Receiver Operating Characteristic curve, or a ROC curve, to evaluate the strength of subscriber engagement predictions. Scores are smoothed so they present an even, understandable view of your customer personas.

When model confidence is closer to 1.00, or 100%, there’s greater predictive strength for that engagement score. Lower model confidence indicates weaker predictive strength for that engagement score. The closer the confidence is to 0.50, or 50%, the more the model has about the same predictive strength of a coin toss. These ranges are the model confidence thresholds for Einstein Engagement Scoring.

• Excellent: 1.00–0.85
• Good: 0.84–0.75
• Fair: 0.65–0.50
• Poor: < 0.50

Each engagement prediction tile on the dashboard contains a model confidence indicator. The indicator consists of three circles that correspond with the model confidence thresholds:

• Excellent: three filled dots
• Good: two filled dots
• Fair: one filled dot
• Poor: no filled dots

Fair or poor model confidence can indicate insufficient subscriber data. It can also result from a change in email marketing strategy that influences subscriber engagement behaviors.