Leaving JSPs in the dust: moving LinkedIn to dust.js client-side templates

This is first of the three blog posts talking about how we build scalable, feature-rich, front-end apps for LinkedIn with an emphasis on site speed, performance, and developer productivity. Today, we'll tell you about our move from server-side templates, such as JSPs, ERBs, and GSPs, to client-side JavaScript templates powered by dust.js.

Where we started

All LinkedIn front-ends were originally built on top of a Java Servlet and JSP stack. As the site and engineering organization grew, so did the desire for front-end technologies that enabled rapid prototyping and development. As a result, we added support for other JVM based frameworks, including Groovy on Rails (Grails) and JRuby on Sinatra.

The dynamically typed and interpreted languages enabled faster prototyping and increased developer productivity. The statically typed Java framework made code more maintainable, easier to refactor, and proved reliable and performant for high-volume, consumer-facing apps at LinkedIn. Each LinkedIn engineering team adopted the front-end stack that best suited their product needs, so our architecture looked something like this:

Fragmented Stack

It seemed like we had the best of both worlds for a while, but we gradually realized how difficult it had become to write reusable front-end features. Each tech stack used a different technology to render pages: the Servlets used JSPs, Grails used GSPs, and JRuby used ERBs. This made it very difficult to share UI code. For example:

• There was no easy way to embed a skills widget, which is rendered via an ERB, in the profile page, which is rendered via a JSP
• The UI code from the profile page JSPs was difficult to reuse in the GSPs used by the Corporate Recruiter profile page
• Even displaying the same global nav across all apps took considerable effort

Client-side templates

Migrating all of our apps onto a single tech stack would've been a very expensive and time consuming project. Instead, we began to explore a unified rendering layer that is agnostic of the server-side technology: client-side templates.

Instead of using a JSP, GSP, or ERB to assemble a page server side and send back HTML, we have the server send back just the dynamic data as JSON and have the page assembled in the browser using a static client-side template served from a CDN. Moving the view logic to the browser meant that our different tech-stacks could share UI code:

Picking a templating technology

It turns out that there are many client-side templating solutions to choose from. Some are based on Resig's microtemplating (e.g. underscore.js), some are logic-less (e.g. mustache), some use a Haml syntax (e.g. Jade), and some are Java friendly (e.g. Google Closure).

All told, we evaluated 26 different templating technologies, scoring them on a variety of factors: DRY, JS library agnostic, mature, open source, easy to learn, documentation, flexibility, performance, and so on. After extensive testing and prototyping, dust.js came out the winner.

Leaving JSPs in the dust

dust.js templates offer huge benefits on many fronts:

• Caching: unlike server-side templates, client-side templates can be served via a CDN to reduce latency for your users and bandwidth and load for your servers. Moreover, the template files can be cached in the user's browser, so after the initial page load, the web server only needs to return the dynamic data as JSON, which is maximally efficient.
• Performance: dust.js templates can be precompiled into highly performant JavaScript functions.
• DRY: templates can be composed of partials, allowing reuse of mark-up and rendering logic, even across different tech stacks.
• Decoupling: dust.js templates emphasize presentation rather than the application and business logic.
• Cross-browser, cross-library: dust.js is compatible with all modern browsers and is JS library agnostic, working equally well with YUI and jQuery. It also escapes malicious template content by default, helping to avoid XSS attacks.
• Server side support: if you have a client that can't execute JavaScript, such as a search engine crawler, a page must be rendered server side. Once written, the same dust.js template can be rendered not only in the browser, but also on the server using node.js or Rhino.
• Progressive rendering: dust.js templates can render asynchronously and in parallel, so expensive operations can run in the background while the rest of the template is being rendered. When rendering server-side, we can also use the async support to early flush content in chunks so the user sees some content sooner.
• Simple, powerful, open: the dust.js template syntax is both easy to learn and to extend. We have written custom dust helpers that allow us to perform formatting, i18n, and rendering logic in a clean way.

Some examples

We're starting to push out client-side templates to all parts of the site. Here are some places you can see them now:

Adding skills on the profile page

See the dust.js code

Adding a position on the profile page

See the dust.js code

LinkedIn Today article on the homepage

See the dust.js code

Conclusion

Today, javascript templates are one of the key components of our front-end infrastructure strategy: they enable browser/CDN template caching, they allow for a cleaner separation of presentation tier concerns, and at the same time, they enable greater unification of our front-end stacks across the company.

We are actively building and migrating some of our core properties to client-templates to achieve our overall objectives. The next blog post will focus more on the challenges we faced in embracing dust.js templates, including the solutions we are building to extend dust.js for static content flushing, rendering logic, formatting, i18n, and A/B testing partials.

Acknowledgments

Special thanks to Brian Geffon, Eran Leshem, Brent Vincent, Avery Moon, Ganesan Venkatasubramanian, Erran Berger, Aleksandr Movsesyan, Yuri Kurland, Andrew Lottmann, Robert Martone, Brian O'Connor, Yevgeniy Brikman, and Lisa Maneresa.