Blog
Use compression to make the web faster
Monday, November 9, 2009
Every day, more than 99 human years are wasted because of
uncompressed content
. Although support for compression is a standard feature of all modern browsers, there are still many cases in which users of these browsers do not receive compressed content. This wastes bandwidth and slows down users' interactions with web pages.
Uncompressed content hurts all users. For bandwidth-constrained users, it takes longer just to transfer the additional bits. For broadband connections, even though the bits are transferred quickly, it takes several round trips between client and server before the two can communicate at the highest possible speed. For these users the number of round trips is the larger factor in determining the time required to load a web page. Even for well-connected users these round trips often take tens of milliseconds and sometimes well over one hundred milliseconds.
In Steve Souders' book
Even Faster Web Sites
, Tony Gentilcore presents data showing the page load time increase with compression disabled. We've reproduced the results for three highest ranked sites from the Alexa top 100 with permission here:
Data, with permission, from Steve Souders, "Chapter 9: Going Beyond Gzipping," in
Even Faster Web Sites
(Sebastapol, CA: O'Reilly, 2009), 122.
The data from Google's web search logs show that the average page load time for users getting uncompressed content is 25% higher compared to the time for users getting compressed content. In a randomized experiment where we forced compression for some users who would otherwise not get compressed content, we measured a latency improvement of 300ms. While this experiment did not capture the full difference, that is probably because users getting forced compression have older computers and older software.
We have found that there are 4 major reasons why users do not get compressed content: anti-virus software, browser bugs, web proxies, and misconfigured web servers. The first three modify the
web request
so that the web server does not know that the browser can uncompress content. Specifically, they remove or mangle the Accept-Encoding header that is normally sent with every request.
Anti-virus software may try to minimize CPU operations by intercepting and altering requests so that web servers send back uncompressed content. But if the CPU is not the bottleneck, the software is not doing users any favors. Some popular antivirus programs interfere with compression. Users can check if their anti-virus software is interfering with compression by visiting the
browser compression test page
at Browserscope.org.
By default, Internet Explorer 6 downgrades to HTTP/1.0 when behind a proxy, and as a result does not send the Accept-Encoding request header. The table below, generated from Google's web search logs, shows that IE 6 represents 36% of all search results that are sent without compression. This number is far higher than the percentage of people using IE 6.
Data from Google Web Search Logs
There are a handful of ISPs, where the percentage of uncompressed content is over 95%. One likely hypothesis is that either an ISP or a corporate proxy removes or mangles the Accept-Encoding header. As with anti-virus software, a user who suspects an ISP is interfering with compression should visit the
browser compression test page
at Browserscope.org.
Finally, in many cases, users are not getting compressed content because the websites they visit are not compressing their content. The following table shows a few popular websites that do not compress all of their content. If these websites were to compress their content, they could decrease the page load times by hundreds of milliseconds for the average user, and even more for users on modem connections.
Data generated using
Page Speed
To reduce uncompressed content, we all need to work together.
Corporate IT departments and individual users can upgrade their browsers, especially if they are using IE 6 with a proxy. Using the latest version of
Firefox
,
Internet Explorer
,
Opera
,
Safari
, or
Google Chrome
will increase the chances of getting compressed content. A recent editorial in
IEEE Spectrum
lists additional reasons - besides compression - for upgrading from IE6.
Anti-virus software vendors can start handling compression properly and would need to stop removing or mangling the Accept-Encoding header in upcoming releases of their software.
ISPs that use an HTTP proxy which strips or mangles the Accept-Encoding header can upgrade, reconfigure or install a better proxy which doesn't prevent their users from getting compressed content.
Webmasters can use
Page Speed
(or other similar tools) to check that the content of their pages is compressed.
For more articles on speeding up the web, check out
http://code.google.com/speed/articles/
.
By Arvind Jain, Engineering Director and Jason Glasgow, Staff Software Engineer
Labels
#freeandopen
#GooglePlay #AndroidDevStory #PlayStore #DeveloperConsole #StoreListingExperiments
#io12
#io13
#io14
#io15
#io16
#io17
#io2012
#io2013
#io2014
+1
20% project
3d
about.com
accelerator
accessibility
actions
actions on google
Administrative APIs
AdMob
adobe
Ads
adsense
advogato
AdWords
africa
agpl
AI
AIY
AIY Projects
AIYProjects
ajax
ajax apis
ajax search
ajax search books news apis
all for good
amarok
AMP
AMP Cache
analytics
android
Android Studio
Android Things
android wear
Announcement
apache
api
apis
apis console
apis explorer
apis. charts
app engine
app indexing
app indexing api
App Invites
apple
apps
apps script
area 120
Artificial Intelligence
asia
assistant
atom publishing protocol
Auth
authentication
authsub
awards
axsjax
barcodes
beacon
beacons
bespin
best practices
bigquery
Blockly
blogger
book search
books API
bootcamp
Brotli
browser
building ajax apps
business
buzz
c++
Cache
caja
caldav
calendar
camino
campfire one
caption
cardboard
CardDAV
cast
Certification
channel
chinese
chrome
chrome apps
chrome dev summit
chrome devtools
chrome experiment
chrome extensions
chrome os
chrome web store
chromecast
chromium
chronoscope
cifs
classes
classroom api
client libraries
closure tools
cloud
cloud datastore
cloud functions for firebase
cloud platform
cloud portability
cloud services
cloud sql
cloud storage
cms
CocoaPods
code for educators
code jam
code review
code-in
codeedu
codelabs
coffee with a googler
collada
color
commerce
community
competition
compression
compressorhead
computing heritage
conferences
contacts api
contest
contextual gadgets
conversations
couchdb
countdown to I/O 2012
courses
Crash Reporting
creative commons
cricket
crisis response
cryptography
css
css3
custom search
custom search api
danish linux forum
dart
Data Compression
datastore
design
devart
develop
developer
developer expert
Developer Keynote
developers
developers. meetup
devfest
devfest developer chrome maps social wave apps
DFP
discovery service
diversity
django
dns
docs
documentation
documents list api
dojo
doodles
dot net
doubleclick
dreamweaver
Drive
drupal
earn
earth
eclipse
eclipsecon
eddystone
education
email
EMEA
enterprise
Europe
event
events
evolution
execution api
extensions
Fabric
faster web
featured
feeds
finance
Firebase
Firebase Analytics
Firebase Cloud Messaging
Firebase Dynamic Links
firebug
firefox
firevox
fitness
font api
fosdem
freebsd
freenet
Fridaygram
fusion tables
G Suite
gadgets
Game Developers Conference
games
gcc
gci
GCP
GDA
gdata
GDC17
GDD
gdd07
gdd08
gdd09
GDD11
GDE
gdg
gdl
gdl weekly
gears
geo
geolocation
geoserver
getpaid
ghop
git
github
gmail
Gmail APIs
gnome
gnome women's summer outreach program
Go
goo.gl
Google
Google APIs
google apps
google apps api
google apps for your domain
google apps marketplace
google assistant
google assistant sdk
google buzz
google cast
google chart api
google checkout
google chrome
Google Cloud Messaging
Google Cloud Platform
google cloud storage
google code
google code project hosting
google code search
google code university
google compute engine
google data apis
google data protocol
google developer day
google developer days
google developers
Google Developers Academy
Google Developers Live
Google Developers site
Google Developers University Consortium
google docs
Google Docs API
google doctype
Google Drive
Google Drive SDK
google earth
google fit
Google Fonts
google friend connect
google gadgets
google gears
google grants
google health
Google I/O
Google Identity Platform
google io
google mashup editor
Google Noto fonts
Google Play
google play services
Google Science Fair
Google sheets
Google Sheets API
Google Slides
Google Slides API
Google Spreadsheets API
google storage
google summer of code
Google tech talk
google technoloy user groups
google tv
google visualization api
google wallet
Google Wave
google web elements
google web toolkit
google.org
google+
googlecast
googleio
googlenew
GooglePlay
googlewebelements googleio
GPE
GPT
green linux
grow
gsoc
GSuite
gtags
gtug
guest post
guice
gulp
GWSOP
gwt
gzip
hackathon
hacking
hackthon
hangouts
haproxy
hg
hibernate
howto
hpux
html
html5
I/O
I/O 17
I/O 2017
I/O Extended
I/O Live
ical
identity
ietf
ignite
igoogle
iguanas
iiw
Image Compression
image search
Imara
in-app payments
incubator
India
indie
internationalization
internet explorer
internet of things
interviews
IO17
IO2017
ios
iOS SDK
IoT
ipad
iphone
israel
Issue Tracker
jaiku
japanese
java
javascript
jetpack
joomla
joomladayus2007
joomladayusa
karaoke
KDE
KDE 4.0
kernel
kernel summit
keynote
khronos
kids
kids coding team
kml
korean
Krakow
labs
lanchpad
languages
latitude
launch
launchpad
lca
Leadership
lessons
licenses
linux
linux foundation
linux summit
linux virtual server
linuxconf eu
localization
LoCo
london
mac
MacFuse
machine learning
Makers
malware
maps
maps apis
Marketplace
material
material components
material design
MDL
meetup
mercurial
MIT CSAIL
mobile
mobile sites
mobile speed
mobile UX
mod_pagespeed
Moderator
monetize
MOOC
mozilla
mylar
myspace
MySQL
mythtv
named
narratives
native ads
native client
nearby
netbsd
non-profit
nonsense
nosql
notifications
Noto Serif CJK
nss
O3D
oauth
OAuth playground
OAuth2
objective-c
OCaml
ocr
ODF
office hours
oha
OOXML
open data
open source
open source blog
open web
openajax alliance
opengl
openid
opensocial
openssh
openssl
oreilly
orkut
oscon
oscon2007
osi
oss devs
ossjam
osx
pactester
page speed
PageSpeed
palette
payments
performance
phone
photos
picasa
picasa web
places API
play services
playground
plone
plone sprint
podcast
polymer
Polymer Summit
portugal
posix
PowerMeter API
prediction api
preview
prizes
programming
Progressive Web App
project hosting
Project Loon
Project Tango
proximity
pubsubhubbub
PWA
py3k
python
python sprint
rails
random hacks of kindness
Rasberry Pi
reader
Remote Config
research
result snippets
Rewarded Ads
Rewarded Video Ads
rhino
Saatchi
salesforce
samba
sandbox
Santa Tracker
scalability
scholarship
Scratch
screencast
sdk
sdks
search
security
serif
service worker
sessions
Sheets API
shindig
shopping
Shoreline Amphitheatre
showcase
sidewiki
sign-in
silverstripe
sitemaps
sites api
sixapart
sketchup
Slides API
Smart Lock for Passwords
soap search api
soc
social
social graph
solaris
souders
spa2007
spdy
speakers
speed
speed tracer
standards
startup
startups
storage
Street View
student programs
students
stuff
subscribed links
subversion
summer of code
Sundar Pichai
SVG
sxsw
syndication
tasks API
Team Drives (new)
techmakers
templates
TensorFlow
TensorFlow Research Cloud
Test Lab
testing
themes
tool
tools
topp
training
tranparency
transit
translate
translation
tutorials
tv
ubiquitous computing
ubiquity
ubucon
ubuntu
Udacity
UI
unicode
unit test
Unity
Universal App Campaigns
unix
URLs
video
videos
Vim
virtual keyboard
virtual reality
visualization
VR
wattpad
Wearables
Weave
web animations api
web apps
web components
web designer
web exponents
web fonts
web performance
web platform docs
webfonts
webgl
webmaster
WebP
website optimizer
weekly roundup
WhiteHouse.gov
Who's at Google I/O
win
windows
windows programming
Winter of Code
Women Tech Makers
women techmakers
wtm
xauth
yahoo
youtube
zlib
zurich
ZXing
Archive
2017
Jul
Jun
May
Apr
Mar
Feb
Jan
2016
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2015
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
2014
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2013
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2012
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2011
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2010
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2009
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2008
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2007
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2006
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Feb
Jan
2005
Dec
Nov
Oct
Sep
Aug
Jul
Jun
May
Apr
Mar
Subscribe
Google
on
Follow @googledevs
Visit
Google Developers
for docs, event info, and more.
