Measure and optimize with mod_pagespeed experiments
By Jeff Kaufman, Software Engineer, PageSpeed Team
Making your site fast shouldn’t require lots of manual optimization. With mod_pagespeed, an
open-source Apache module, you can automatically apply web performance optimization best
practices like
cache
extension,
image
optimization, and
css
inlining to speed up your site without a lot of hassle. As of version 0.10.22.4,
mod_pagespeed now
supports
A/B tests integrated with
Google
Analytics, allowing you to measure how much it speeds up your site on live traffic
and experimentally determine the best settings.
When running an experiment, mod_pagespeed randomly assigns visitors to experimental
configurations based on percentages you choose. You can run an experiment on 1% of your
traffic, 100%, or anywhere in between without affecting other visitors. It also injects
JavaScript to report experiment assignments back to your Google Analytics account in a
custom
variable. Within Analytics you can track the impact of experimental configurations
on page load times, bounce rates, conversions, or any other Analytics metric.
We ran an example experiment, comparing mod_pagespeed running with default settings to
mod_pagespeed in pass-through mode, on a small blog. This required adding the following lines
to our
pagespeed.conf:
ModPagespeedRunExperiment on
ModPagespeedAnalyticsID "UA-XXXXXXXX-Y"
# half the users get the pagespeed optimizations
ModPagespeedExperimentSpec id=3;percent=50;default
# half get an unoptimized site
ModPagespeedExperimentSpec id=4;percent=50
While this site was static and contained mostly text, it did use some JavaScript
and images and had not been manually optimized. We ran the experiment for a month, over which
Analytics observed 11K page views, and we saw a 20% improvement in average page load
time:
Average page load time is sensitive to outliers, however, so to better understand the effects
it’s helpful to check a histogram:
The clearest change is that mod_pagespeed moved about 7% of page loads from taking 1-3 seconds
down to 0-1 second, but there is also an improvement in the long tail.
We encourage you to follow the
experiment
framework guide and start measuring the effect mod_pagespeed has on your site.
Jeff Kaufman works on mod_pagespeed, an
open-source Apache module that helps make
the web faster, and is interested in experiment measurement. He also plays for contra dances, organizes other dances, and blogs about dancing, giving, and tech.
Posted by Scott Knaster,
Editor
