Fixing app cache

by Andrew Betts,

At FT Labs, we are on a mission to fix app cache, for the good of mankind web developers.

Over the last few weeks a couple of important meetings have taken place bringing together developers and browser vendors to try and work out what went wrong and how we can fix an oft-reviled godawful mess that is the HTML5 application cache. We hosted a meeting of a number of developers and browser vendors here in London on the 14th of August, and Mozilla hosted a meeting on the 24th at their office in Mountain View in Silicon Valley.

At the FT Labs event, the aim was first to air our practical experiences with sites that the developers present were building – Facebook, GMail, the FT and Economist HTML5 apps and Lanyrd. It was great to explore the techniques that are being used, and the solutions that developers are trying to achieve. Google, for example, wants not only to update individual files in cache but to apply differential updates so that only new parts of a file are downloaded. That might sound nuts but it was one of a long list of things throughout the day that followed the same pattern – we want to do this on the web, and despite the fact that web standards are a million miles away from supporting us in our objective, we want it because it makes for a better app. And of course you can do it with a native app, yada yada.

For our part, we want to be able to cache media content – audio and video, individually, in some kind of storage that doesn’t require us to do crazy stuff just to make efficient use of the meagre quotas we have available. We want better control over what is cached and what isn’t, knowledge of whether we got served from cache or not, and we want to be able to ask for more quota. MUCH more. No standard tells a browser vendor how much storage they must allow a website to request, so it varies from unusably small to unlimited.

All these complaints came out and gave us a great wealth of shared experience to draw a better mental model of what is wrong with app cache. But the biggest realisation of all was that none of us actually understands it. A roomful of some of the smartest developers and browser makers on the planet, allowing an entire day to do so, cannot wrap their heads around the damn thing. We had no common understanding of either the app cache selection algorithm or the app cache download process.

So we started to use what we do know – the things we wanted to do and couldn’t – to build up a set of user stories that could feed back into the spec process and hopefully fix app cache for everyone.

Mozilla’s meeting over in California added some more experiences – Twitter, and Microsoft Outlook Web Access particularly, and drew out a number of high level ideas for what a fixed Appcache could look like. It’s a view I like a lot, particularly these points:

  • Get rid of master entries concept
  • When navigating, look for an appcache which contains that URL, rather than look for an appcache which has that URL as master entry.
  • JS-API for adding/removing/enumerating entries
  • JS-API for creating/deleting/enumerating appcaches
  • Knowing if you’re being served from appcache or not. And which version of the appcache.
  • Ability to write a HTTP server in JS which handle URI spaces.
  • Network:* should be default mode

The work continues. We’ve formed a new W3C community group called ‘Fixing appcache’, which Tobie and I are chairing, to move the process forwards, and many of the participants from these two gatherings are members. The notes from the London (as it happened and more refined) and Mountain View (as it happened) meetings are available online, so if you’re interested in helping out, go and read them, join the group, help us refine the use cases. Share your own experiences with app cache.

Together, we can fix appcache. Then we’ll do the world peace thing.