You are here:
A/B Testing for B2C Commerce
Using Business Manager, you can assign promotions or slot content to an A/B test with common qualifiers such as source codes or customer groups. A/B test statistics use a combination of two-sided t-tests and chi-square tests to weigh test segments against each other and compute the statistical significance.
Create and manage A/B tests, view test results, and change your site based on those results. Configure changes to your sites that affect only a subset of visitors so that you can determine from collected data the experience that produces the best results.
You create test segment, where each segment configures a unique experience for a set of participants, against which you can test other segments and a control segment. Define triggers that a customer accesses to qualify for participation in a particular test.
When creating A/B tests, understand that, relative to other promotions available in Salesforce B2C Commerce, A/B tests are considered to have high precedence.
You can also customize A/B tests using the B2C Commerce API, and use the A/B test import/export functionality to manage A/B tests across your company. Extract A/B test results in .csv output files, and use those files in your favorite spreadsheet application for further analysis.
A/B Testing vs. Personalization
Although the effect is similar, A/B testing segmentation is different from personalization segmentation. The regular personalization of a site, for example, done using customer groups, is segmented by criteria that you can define. Meanwhile, A/B test segmentation runs randomly. With both, the user sees an experience that is potentially different than other users. A/B testing, however, goes beyond personalization. A/B testing collects metrics to help determine which experience, among two or more, performs the best according to one or more segmentation criteria.
A/B Testing Eligibility
A shopper’s eligibility for A/B tests is determined by the Tracking site preference. The Tracking site preference is session-specific and only shoppers for whom tracking is allowed are eligible for A/B tests.
To set this preference, click App
Launcher 
, and then select .
The consistency of an A/B test experience varies slightly for eligible registered and unregistered shoppers.
- Eligible Registered–B2C Commerce stores the mapping between the A/B test group assigned to the shopper's customer ID in the database. As a result, B2C Commerce recalls the shopper's A/B test assignments, and therefore the shopping experience is consistent regardless of which device the shopper uses.
- Eligible Guest–B2C Commerce writes the shopper's assigned A/B test group to a cookie on the shopper's browser. As a result, B2C Commerce recalls the shopper's A/B test assignments. The shopping experience is consistent only when the shopper is on the same device and using the same browser.
Eligibility and A/B Test Sample Size
The consent requirement for A/B test eligibility and the possibility of a varied A/B test experience for unregistered shoppers can impact your test sample size. Consider planning longer test durations or altering your test criteria to capture statistically significant A/B test date.
A/B Test Data Deletion
Shopper data deletion requests cascade to A/B test participant records. When a shopper requests to have their records deleted from the system, all the shopper's A/B test participant records are deleted.
Set Up Incremental Indexing
Set up incremental indexing for B2C Commerce.
A/B Testing vs. Personalization
Although the effect is similar, A/B testing segmentation is different from personalization segmentation. The regular personalization of a site, for example, done using customer groups, is segmented by criteria that you can define. Meanwhile, A/B test segmentation runs randomly. With both, the user sees an experience that is potentially different than other users. A/B testing, however, goes beyond personalization. A/B testing collects metrics to help determine which experience, among two or more, performs the best according to one or more segmentation criteria.
A/B Testing Eligibility
The Tracking site preference determines a shopper’s eligibility for A/B tests. The Tracking site preference is session-specific and only shoppers for whom tracking is allowed are eligible for A/B tests.
Eligibility and A/B Test Sample Size
The consent requirement for A/B test eligibility and the possibility of a varied A/B test experience for unregistered shoppers can impact your test sample size. Consider planning longer test durations or altering your test criteria to capture statistically significant A/B test date.
