Rest.li: RESTful Service Architecture at Scale

Co-author: Moira Tagle

Today we are announcing the open-sourcing of Rest.li, a piece of infrastructure developed and used here at LinkedIn. Rest.li is a REST+JSON framework for building robust, scalable service architectures using dynamic discovery and simple asynchronous APIs. We feel that Rest.li fills a niche for building RESTful service architectures at scale, offering a developer workflow for defining data and REST APIs that promotes uniform interfaces, consistent data modeling, type-safety, and compatibility-checked API evolution.

Rest.li was created to deal with a number of problems at LinkedIn. First, we needed a standard way to describe the resources available in our service-oriented architecture and enable access by diverse clients written in any language. We wanted to focus on standardizing common API operations, while still allowing developers to create non-standard operations. Secondly, we needed our solution to be scalable and asynchronous. Many of our services receive several thousand queries per second per instance, and we needed our solution to work under that sort of load. We wanted to ensure that writing APIs would be fast and easy for our developers, even if they are not well-versed with REST. We also needed our system to support long-term evolution and growth of our interfaces.

Figure 1. Rest.li Client Server Flow

Rest.li is a Java framework for building REST-style services using well known concepts such as an entity-oriented design, standard HTTP-style verbs to operate on those entities, and a flexible serialization scheme for entities. We've added standard batch operations to support efficient access to large data sets. It also includes Java client libraries and supports type-safe access to resources and entities on both the client and server side. Rest.li also specifies an IDL for describing resources.

R2 is a REST transport layer abstraction in Java. It exposes a low-level REST-style interface inspired by HTTP and a high-performance asynchronous API. We use Netty to build R2's HTTP implementation. R2 has also allowed us to experiment with mapping REST operations to non-HTTP transports.

D2 is our dynamic discovery and client-side load balancing layer. We use ZooKeeper as a registry for information about available services and the hosts that provide them. Clients automatically get the latest information from ZooKeeper and apply load-balancing algorithms on the client side to distribute load evenly across servers and reduce load to overloaded servers.

We've been using Rest.li for well over a year at LinkedIn. All our new services are built using it and we've converted many of our pre-existing services over. We think it's so important to have a uniform set interfaces that represent our data that we're aggressively migrating all, yes all, our services to Rest.li. And we're already well into this transition with many core services that power our site using Rest.li, including people following, our recommendation engine systems and the network update stream on the homepage.

We hope every Java developer working with RESTful APIs will give Rest.li a try. While a big benefit of Rest.li is that it can scale to the needs of world class data centers, it's designed to be lightweight, approachable and to foster REST best practices. This makes it a great way to get a RESTful API up and running for projects both small and large, even if you're not a REST expert.

Get started

1. Quick Start Guide
2. Rest.li source on github
3. Rest.li LinkedIn Group - Ask questions and meet the team