As an important tool in the Digital Manager’s toolkit, digital experimentation platforms deliver considerable business benefits but often at the expense of faster loading web pages. In this multi-part article we take a look at some of the inhibitors to web performance that come with these platforms and how you may potentially reduce their impact on customer experience.
Background
In the search for excellent customer experience, digital experimentation tools providing A/B testing and multi-variate testing (MVT), are important tools in the Digital Manager’s toolbox. These tools support the timely implementation of marketing campaigns, enable the testing and activation of feature updates, without the inherent development process delays, and also provide invaluable conversion reports to aid in understanding how the ecommerce platform is performing.
There are many vendors in this market but with a reported 35% market share, (source Datanyze: Figure 1), Optimizely has seen significant success. However, it is a technology that unless managed can be an inhibitor to achieving an organisation’s web performance objectives.
In this discussion we take a look at how the incorrect implementation of Optimizely can inhibit optimum web performance and what options exist to limit their impact on customer experience.
The importance of correct implementation
MVT systems can be implemented in several different ways, the most predominant being client-side in-browser (Optimizely Web) . Host-side server (Optimizely Full Stack) and edge-server options also exist with each option having its own set of benefits and drawbacks as well as different price points.
Optimizely Web is the easiest to implement and use and possibly why it has become a popular choice. The majority of the industry’s MVT vendors use similar techniques for this type of implementation.
Optimizely recommends that you insert a simple JavaScript snippet directly into the Head section of the HTML. Their exact recommendation is ‘as high up in the head tag as possible, generally after your <html> tag, charset declarations, and possibly other meta tags.’
Basically, the recommendation implies that the snippet is the first resource that is requested in the HTML stream. It is important as the result of any experiment that may run could change the webpage delivered and with it the customer’s experience.
However, this recommendation can be improved on.
It is not possible to avoid the synchronous approach taken to load and process the snippet as it will inject delay into the critical render path by blocking the browser until the JavaScript has been downloaded, parsed and evaluated. This is discussed further in the section ‘Synchronous Processing and the impact on the critical render path’ below.
Although the browser is prevented from progressing until the snippet has completed loading, parsing and evaluation, it is able to fill the time it must wait by processing previously initiated resource requests.
Consequently, requesting CSS resources prior to the snippet is recommended as these can be downloaded and the stylesheets parsed by the browser. The browser may be able to process any preload hints that have been defined as well.
Synchronous Processing and the impact on the critical render path
When implemented correctly in the HTML’s Head section, the Optimizely snippet processes synchronously with the objective of seamlessly determining and applying any changes needed to the server generated HTML.
Synchronous loading causes the browser to halt the parsing of HTML and any page rendering processes until the completion of execution of the requested JavaScript code. As this directly impacts the critical render path, possibly delaying rendering by several seconds, an irony exists as one of the purposes of MVT is to improve user experience.
Optimizely have worked to minimise delays by serving their JavaScript from an edge server location in a CDN. Unlike its predecessor Optimizely Classic, Optimizely X does not require jQuery to be loaded as a pre-requisite, but even with these enhancements a delay of ½ second is not unusual even over fast network connections and closer to 1.5 seconds on a 4G connected device.
Using Google Developer tools in the chrome browser we can see how the Optimizely snippet is processed. In Figure 4, the process breaks down into 3 separate sections. On an emulated 4G connection the first section begins as the Send Request is initiated. The browser then loads the snippet synchronously and observed time for loading the resource is 831ms.
Section 2 occurs as the script is parsed which in this example occurs in the parser thread, (not shown), in parallel to the loading process. In this example we benefit from the parallel processing with a time of 0ms.
However, this is not always the case especially if other tasks are being processed by the parser thread, such as jQuery or an APM library. Furthermore, not all browsers have parser threads, in which case parsing the script will occur sequentially following completion of the file load. Further insight on how the parser thread works in Chrome is covered here.
Section 3 is ‘Evaluate Script’ and is where the script is executed and the Optimizely experiment choices are determined. In this example, the ‘Evaluate Script’ task took 375.75ms execution time.
This example shows that the browser is unable to use valuable CPU processing time as it is prevented from progressing other activities for 1.18s while Optimizely loads and processes.
The time taken to load and evaluate Optimizely’s snippet is dependent not only on the capabilities of the network connection but also the speed of the processor of the client device used to parse and execute the script. As these attributes can vary considerably it is recommended that benchmarks on different devices and network connections are established so that the effect on customer experience can be observed as changes in the snippet size occur.
Beware the use of a Tag Manager
For many organisations, due to sprint priorities, implementing the snippet can sometimes be difficult to achieve in a timely manner. Digital and ecommerce managers circumvent this delay by using a Tag Manager which is often a resource they have directly under their control.
However, what would appear to be a quick win for the ecommerce team quickly becomes a loss for web performance and with it customer experience. Optimizely specifically states that the JavaScript snippet should not be implemented using a Tag Manager but this advice is often overlooked for reasons of ease of implementation and expediency.
When implemented through a tag manager, Optimizely warns of several issues such as the effect on customer experience through page flashing as an experiment overlays the original rendered page content with new content. It is also known that inconsistencies may occur in collected metrics and therefore the accuracy and value of reports and analytics may be reduced.
Some older tag manager’s use outdated methods and can load tags synchronously or by injecting code using ‘document.write’ directly into the HTML stream. As these are browser and render blocking events both of these actions will have a severe impact on web performance.
Unfortunately this practice is not uncommon and a key recommendation of this article is that for Optimizely Web, the Optimizely snippet should only be loaded directly from the Head section and never through a tag manager.
In Optimizing Optimizely Part 2 we will further investigate the impact Optimizely has on the critical render path.