You are here:
A/B Testing for B2C Commerce Participation Triggers
With A/B testing, you can define triggers that a customer accesses to qualify for participation in a particular test. This topic applies to B2C Commerce.
Triggers can be:
- Site Entrance / Customer Login with customer group qualification
- Site Entrance / Customer Login without customer group qualification
- Category page view
- Home page view
- Pipeline call
Pipeline Triggers
The participation trigger can't be the same pipeline as one that collects data. Consider how you want to trigger the participation of a user in an A/B test. Reporting an A/B test participation can't be the trigger itself. The pipeline product-show is a good trigger. To use a pipeline as a trigger, use a different pipeline or call a simple pipeline through a remote include, at the appropriate place.
For example, use any of the existing pipelines as a trigger, except for a few reserved pipelines, such as ReportingEvent-Start. Or create a simple pipeline containing just a start node and an empty template, and include it in your code as appropriate. Also, to test pipeline triggering, call the pipeline directly in the browser via its pipeline URL.
Site Entrance / Customer Login Triggers
The customer triggers participation for an A/B test without customer group qualification as soon the customer accesses the site.
The customer triggers participation for an A/B test with customer group qualification when the customer logs in.
For example, a merchant wants to show two different banners within the storefront header. Banner 1 offers Free Shipping on Orders over $150 with a corresponding promotion. Banner 2 offers 10% off orders over $150 with a corresponding promotion. The merchant configures an A/B test as follows:
- Trigger = Site Entrance / Customer Login: A customer enters an A/B test when the customer accesses the storefront.
- Control Group = 10%: Represents the normal site experience header and promotion
- Segment A = 45%: Free shipping banner
- Segment B = 45%: 10% banner
When the customer accesses the site, the customer is immediately placed in one of the three test groups because the header banner is persistent throughout the storefront. The statistics are significant because customers always see the header.
Category Page View Triggers
The customer triggers participation when they view the category page for one of the configured categories. This means that a call was made to Search-Show?cgid=xxxx for one of the configured category IDs.
For example, a merchant wants to shows two different banners on the Kids category page within the storefront. Banner 1 offers Buy one kid's winter hat, get one free with a corresponding promotion. Banner 2 offers 20% off on kid's winter hats. The merchant configures an A/B test as follows:
- Trigger = Category Page View: A customer enters an A/B test when the customer accesses the category page.
- Control Group = 10%: Represents the normal site experience header and promotion
- Segment A = 45%: BOGO banner
- Segment B = 45%: 2% banner
The customer is only assigned to one of the three test groups if the customer actually views the Kids category page. If the customer doesn't view this page, no metrics are collected.
Home Page View Triggers
To trigger participation, the customer views the default storefront URL (that is, "/") or the Home-Show pipeline.
For example, a merchant wants to show two different banners on the home page within the storefront. Banner 1 offers Free Shipping on Orders over $150 with a corresponding promotion. Banner 2 offers 10% off orders over $150 with the corresponding promotion. The merchant configures an A/B test as follows:
- Trigger = Home Page View: A customer enters an A/B test when accessing the home page.
- Control Group = 10%: Represents the normal site experience header and promotion
- Segment A = 45%: Free shipping banner
- Segment B = 45%: 10% banner
The customer is only assigned to one of the three test groups by actually viewing the home page. If the customer doesn't view this page, no metrics are collected.
Pipeline Call Triggers
The customer triggers participation when a request calls a configured pipeline name and start node (for example, Account-Show). The use of triggers has no effect on the pipeline or scripting API because B2C Commerce request-handling logic executes A/B test triggers.
For example, a merchant wants to understand how well a particular promotion is doing for its registered customers. The merchant configures a banner within the My Account section as follows:
- Trigger = Pipeline Call
- Control Group = 0%: Represents the normal site experience header and promotion
- Segment A = 50%: (Promotion A, Banner A)
- Segment B = 50%: (Promotion B, Banner B)
The merchant only wants the customer to be assigned to an A/B test when the customer views the My Account section because that’s where the banners A and B appear. For example, a customer goes immediately to a product detail page, adds a product, and checks out. The customer is only put into one of the two A/B test groups by actually accessing My Account and seeing the banner. If this pipeline isn't accessed, the customer is never assigned to either test.
