A/B Testing in Optimizely CMS and How to Handle it when Cache is Implemented

In this blog post we are going to talk about A/B Testing in Optimizely/Episerver CMS, how to install it and how to avoid conflicts if your page has a cache layer in place. So, without further due, lets begin.

To be able to do A/B testing it is required to install the package EPiServer.Marketing.Testing which will also install the following dependencies:

• EPiServer.Marketing.KPI
• EPiServer.Marketing.Messaging

You can add them using the nuget package explorer or the nuget package console and the Episerver nuget feed.

After installing all the needed packages, to use it, you just have to go to a page, create a new draft and before publishing go to the publish button and then to A/B Test Changes.

This will create a new experiment which show the two versions, the published and the draft that we just created. If you press the start test button it will create the A/B test.

You can also add a description about the goal of the test, add conversion goals like site stickiness, time on page or landing page, the participation percentage (how many users are going to be part of the experiment), test duration in days and if the test is going to start immediately or scheduled for later.

Finally, in advanced options you can change the confidence level of the A/B test to reduce the possibility of incoherent data.

Now, lets assume that you are A/B testing a header but the header is too heavy so you have a cache layer on top of it to render it faster. What we want to implement is the following: If the page is involved in an A/B test show the A or B header without using the cache, if the page is not involved in an A/B test use the cache as usual. For this, we first need to use the following repository.

private readonly Injected<IMarketingTestingWebRepository> MarketingTestingWebRepository;

In order to check if the current page is involved in a A/B test experiment you can use the method GetActiveTestsByOriginalItemId to get the experiment, if any, using the content guid of the page.

 var anyActiveTests = currentPage?.ContentGuid != null && MarketingTestingWebRepository.Service.GetActiveTestsByOriginalItemId(currentPage.ContentGuid).Any();

Finally, using this Boolean variable, we will decide if we show the header from the cache layer or from the CMS.

var key = $"{SiteDefinition.Current.Id}-{currentPage?.ContentLink?.ID ?? 0}";

// Check if they are using ab - testing, disable cache if that is the case
if (anyActiveTests)
{
      return BuildHeaderNavigationItems();
} else 
{
      return MemoryCacheHelper.GetCacheItem(key);
}

The methods BuildHeaderNavigationItems and MemoryCacheHelper.GetCacheItem are just fictional methods which will generate the header component from scratch or get it from the Cache.

And that is it, You can now test if a page is being used in an experiment and decide what to do with that information. If you have any question let me know. I hope it will help someone and as always keep learning !!!