LinkedIn just released a brand new iPad app built using HTML5, backbone.js, and underscore.js. The app includes a rich stream of information which is a combination of network updates, group posts and news articles. The user can also switch to specific streams like Co-Worker updates or news articles from a specific category. In this post, I'll tell you how we leverage local storage to fetch all this data while still offering a snappy experience.
Client-side MVC and templates
Our views are rendered using underscore templates that are bound to backbone views and populated with data from backbone models or collections. Backbone models are responsible for storing the JSON data received from the server in a structured format and backbone collections are ordered sets of such models. The fetch function is executed on a model to make it receive, parse, and populate itself with data from the server. It does this via a RESTful JSON request through Backbone.Sync, a function Backbone calls every time it attempts to read or save a model to the server.
Each of these calls is an asynchronous network request which can take a few seconds. This latency can be partially overcome with some smart use of HTML 5 Local Storage, which is available in most modern mobile operating browsers. Recently fetched data can be considered fresh and served directly (and immediately) from local storage instead of making a roundtrip to the server.
Key Generation
HTML 5 Local storage is a store of key-value pairs. It is important to namespace keys to avoid potential conflicts. We use the following combination of identifiers:
<app-id>:user:<user-id>:<key-name>
Seamless use of local storage as persistence strategy
The idea is to override Backbone.Sync to use local storage when appropriate in a manner that hides all the internal workings from the regular Backbone fetch mechanism. This makes sure that the individual models and collections don't have to concern themselves with the persistence strategy and their parsing mechanism is untouched.
All collections inherit from a BaseCollection and all models inherit from BaseModel. Each model or collection specifies a boolean flag to indicate if it should be stored in local storage:
isStoredInLocalStorage : true
- The model/collection also implements a hash function to generate the unique key associated with that user and that model/collection.
- The BaseCollection and BaseModel implement the sync method which does all the hard work under the hoods
- The only method that is overridden is the read method for fetch requests
- The timestamp of the last fetch call is fetched from local storage and its difference from the current time is compared to the maximum time that data is considered fresh for that model/collection. A force refresh flag can be used to override this to enable a force fetch from the server if required.
- If the data is considered fresh by the above logic, it is fetched from local storage using the generated key and a normal async network call is simulated by calling the success handler within a setTimeout
- If however, the data in local storage is found to be stale, the normal Backbone sync method is executed and is passed a callback that would store the returned data in local storage using the generated key, store the current timestamp and pass along the data to the success handler.
- Another interesting scenario is the since call which adds to the current data rather than replacing it. In such a situation, the data received from the server is appended to the data read from local storage and written back to it.
Local Storage Reset
Most implementations of HTML 5 local storage have a size limit of about 5MB. As we quickly found out with our large stream of data, when the stored data starts exceeding this limit, setting a key value throws the QUOTA_EXCEEDED_ERROR. This can be handled by a reset mechanism. We chose to divide the key-value pairs in our storage system into volatile and non-volatile data. Non volatile data is something that cannot be purged during the course of an application's lifetime as it stores persistent information like the logged-in user information. On the other hand, volatile data like the last fetched stream data can be purged without any adverse effects on the application apart from a noticeable latency during the next stream request.
On receiving the quota exceeded exception, the local storage is reset to temporarily read in all the non-volatile data, clear local storage completely and write back the non-volatile data. This ensures that the store is now ready for further data storage by getting rid of all the stale data. This reset mechanism also kicks in when a user logs out to refresh local storage for brand new data.
Migration/Versioning
Any data store ends up requiring a versioning system to take care of application upgrades and the corresponding migration of the data store. To take care of this, a storageversion key stores the current version of the data store and a config file stores the expected current version. Before executing any of the data store functions, these two values are compared to make sure they are in sync. An inconsistency causes the data store to be cleared to start off with the new version. This ensures that data can be refreshed for the new data store. Thus, migration of the data store becomes as simple as updating the version number in the config file.
Conclusion
We've found HTML 5 local storage to be an effective way to improve an application's performance, both in terms of time and space requirements:
- Temporarily storing recently fetched data results in a tremendous improvement in application responsiveness. It provides the user the ability to move around the application quickly without waiting for network requests to finish for previously viewed data.
- More than one web view in the native application can share fetched data seamlessly.
- Memory constraints in mobile devices can be overcome by using local storage to store and populate temporary objects in memory when required.
- Storing complicated HTML document fragments from the stream to in local storage can decrease memory footprint and rendering time while scrolling through the stream.
Try out the LinkedIn iPad app and let us know what you think!