Using Google App Engine to power Web APIs
This post is part of the Who's @ Google I/O, a series of blog posts that give a
closer look at developers who'll be speaking or demoing at Google I/O. Today's post is a guest
post written by Steven Willmott of 3scale.Public APIs are a great way
to share data and functionality, both for existing applications and as useful resources in
their own right. Google Maps, Calendar and many other APIs lead the way in this and are now
powering tens of thousands of sites and applications. Flickr, Twitter and others also provide
great data and functionality which can be used to enhance and build new
applications.
While APIs can provide great value add, it's not always
obvious how to make one available if you're not Google or Yahoo. Although serving content via
an API is effectively no different than via a user-facing web app - just serving HTTP traffic
with XML, JSON or arbitrary data rather than HTML, CSS and Javascript normally destined for a
browser - it does come with some unique challenges: authentication works differently, inbound
traffic is often automated and response times need to be rapid for the API to be usable within
another site - all a bit different from a standard web app.
Since we're
in the API business we often get asked about how to quickly get a new API up and running, and
one neat new way to do this is using Google App Engine, which provides a lot of the core
ingredients needed to launch a new API and help it scale. To see how this works we put
together a quick and dirty trial service which exposes App Engine's Image manipulation
functions as a new Web API:
- First we set up a standard App
Engine account giving us the standard dashboard and deployment environment. This provides the
runtime environment plus a range of tools to monitor the application - all useful to track
resource usage which we'll need to worry about later.
- The
service itself in this case is a simple wrap of wrapping App Engine's available image
manipulation features which provide standard image manipulation features, but it could easily
be more complex.
- Next up, the functionality is wrapped in a
simple RESTful interface. There are multiple ways to do this, libraries such as the App Engine Rest Server and the Django rest interface or a port of the Ning-like REST interface can be used to do most of the heavy lifting. Gae-Rest
for example includes indexes and memcache and other features to optimise response times on the
API. These give the API standard GET, POST, PUT, DELETE operations we'll need to push images
in and out of the application.
- Finally, while we have the
service ready to roll we still need a way for developers to access the service, manage their
resource usage and track activity on the API. To do this we can use 3scale's API management platform which provides these features out of
the box. Connection works using a standard python plugin and the system creates a configurable
management portal for the service which handles signup keys, usage limits, traffic analysis
and the documentation needed to support developers using the new API. For heavy users we can
also add paid plans to cover any App Engine costs or make a small profit - check out the
appspotimage portal here:
The
service can now take advantage of App Engine's scalability - even under the constraints of the
plan, your API should comfortably serve up 500,000 - 1 Million hits per day (although note
that you need to check other resource limits - not just hits) which would be non-trivial to
arrange and potentially high cost on a lot of existing hosting services. If you're serving
traffic from appspot.com (for appspotimage we serve the management interface from
appspotimage.com and the API traffic from appspot.com) response times from App Engine tend to
be very fast (sub 10ms if you're on the west coast US) - which is great for API
users.
Although backend APIs are not quite as visible as a new Android or
iPhone App, they often provide a great resource and hopefully we'll see more and more datasets
and new types of processing on line as APIs.
You can find the service
itself
here
along with a
longer tutorial on how we built
it. Stay posted for some other services coming soon, as we start to play around with
Java language support on App Engine.
We'll be at
Google I/O
on May 27-28 and look forward to talking APIs and App Engine, so stop by the
Developer Sandbox. See you there!
Guest Post by Steven Willmott of 3scale
