Measure and optimize with mod_pagespeed experiments

JULY 30, 2012
Author Photo
By Jeff Kaufman, Software Engineer, PageSpeed Team

Making your site fast shouldn’t require lots of manual optimization. With mod_pagespeed, an open-source Apache module, you can automatically apply web performance optimization best practices like cache extension, image optimization, and css inlining to speed up your site without a lot of hassle. As of version 0.10.22.4, mod_pagespeed now supports A/B tests integrated with Google Analytics, allowing you to measure how much it speeds up your site on live traffic and experimentally determine the best settings.

When running an experiment, mod_pagespeed randomly assigns visitors to experimental configurations based on percentages you choose. You can run an experiment on 1% of your traffic, 100%, or anywhere in between without affecting other visitors. It also injects JavaScript to report experiment assignments back to your Google Analytics account in a custom variable. Within Analytics you can track the impact of experimental configurations on page load times, bounce rates, conversions, or any other Analytics metric.

We ran an example experiment, comparing mod_pagespeed running with default settings to mod_pagespeed in pass-through mode, on a small blog. This required adding the following lines to our pagespeed.conf:
ModPagespeedRunExperiment on
      ModPagespeedAnalyticsID "UA-XXXXXXXX-Y"

      # half the users get the pagespeed optimizations
      ModPagespeedExperimentSpec id=3;percent=50;default

      # half get an unoptimized site
      ModPagespeedExperimentSpec id=4;percent=50
While this site was static and contained mostly text, it did use some JavaScript and images and had not been manually optimized. We ran the experiment for a month, over which Analytics observed 11K page views, and we saw a 20% improvement in average page load time:


experiment results

Average page load time is sensitive to outliers, however, so to better understand the effects it’s helpful to check a histogram:


detailed experiment results

The clearest change is that mod_pagespeed moved about 7% of page loads from taking 1-3 seconds down to 0-1 second, but there is also an improvement in the long tail.

We encourage you to follow the experiment framework guide and start measuring the effect mod_pagespeed has on your site.


Jeff Kaufman works on mod_pagespeed, an open-source Apache module that helps make the web faster, and is interested in experiment measurement. He also plays for contra dances, organizes other dances, and blogs about dancing, giving, and tech.

Posted by Scott Knaster, Editor
posted in:
Previous
Next