LinkedIn's Rest.li Moment

Slightly over a year ago LinkedIn announced the open source release of Rest.li, a framework we created for developing RESTful services at scale. Today we are excited to share a major engineering milestone regarding Rest.li: 50% of all remote calls between services at LinkedIn now use Rest.li!

Services using Rest.li for production traffic include external products such as University Pages, the Home Page feed, and Company Pages, as well as several widely used internal services such as our A/B testing service, our recommendations services, and Entitlements, a service that controls what premium features our members can access.

Why Rest.li?

Back 2002-2003 LinkedIn chose to use serialized Java objects over RPC as our communication mechanism. While this system made sense then, it does not in our current polyglot ecosystem with apps and services being written in Java, Node.js, Python, Ruby, and Scala. We created Rest.li in 2011 to address this issue (and several other issues mentioned below) and are halfway through migrating all our services to use it.

One of the biggest gains we have seen since we started this migration is in mobile development. It is now on par with desktop development in terms of internal APIs that are accessible. Our mobile services are primarily written in Node.js, which had made it very hard to communicate with Java object based RPC services. Since Rest.li is simply JSON over HTTP our software engineers can now develop mobile applications at a much faster rate than before.

In fact, using JSON over HTTP as the primary communication channel has several advantages: nearly all frameworks and languages support it, it is developer friendly (easy to read requests and responses), and you don’t need any special clients to use it. One can communicate with Rest.li services by using curl or even using the browser.

"Before we migrated our Python API endpoints to Rest.li, supporting downstream Java dependencies required a great deal of work. Now, we benefit from Rest.li by being able to not only support downstream Java dependencies with ease, but also to rapidly build services which clients can easily use” - Roman Inozemtsev, Staff Engineer, LinkedIn.

Using Rest.li for all services also lowers the learning curve for interacting with a new service. Our previous RPC system allowed for non-uniform interfaces, which meant that engineers had to potentially re-learn a new API each time they communicated with a different service. Since Rest.li is based on REST, this is not the case anymore, resulting in uniform service access semantics across LinkedIn.

“In order to prevent our Rest.li interfaces from looking RPC-like, it was important to establish conventions and design patterns. We achieved this by putting together a cross-functional group that would review every API as it was developed. This also ensured APIs weren't being designed for a specific use case, for example a representative of the mobile team could ask how the API would power a new feature in the tablet app. Clear and consistent data models means there is minimal transformation of data as it transitions between our internal systems and external APIs used by 3rd party developers, Kafka events produced by our services, and our offline systems like Hadoop and the data warehouse.” - Brandon Duncan, Director of Engineering, LinkedIn

Most LinkedIn Rest.li services use D2 for load balancing and dynamic discovery (though Rest.li does not depend on D2). Dynamic discovery frees us from having to configure URLs for each service that we need to talk to. D2 has allowed us to reduce the usage of hardware load balancers in our service deployments as the load balancing is now distributed amongst clients. Elimination of hardware load balancers also results in fewer network hops. Client side load balancing allows us to build service invocation features like scatter gather requests, sending a request to all partitions of a cluster etc. easily. It also gives us the ability to do things like graceful degradation and call dropping to help services recover.

Open Source

We are fully committed to open source and would urge people to give Rest.li a try and reach out to us if you have any questions. Please take a look at the Rest.li skeleton generator to see how easy it is to get started with a simple Rest.li service.

We are also pleased to announce the open source release of Rest.li API Hub. This began as a hack day project by Joe Betz and has now become an indispensible tool for LinkedIn’s Rest.li ecosystem. Once you configure it to work with the Rest.li services in your environment, the API hub allows you to view and search all your APIs, use an interactive console to send requests to actual services, share links to requests that you sent etc. Do check it out and let us know if you have any questions!

Next Play

So what’s next for Rest.li? For one, the team is working on Rest.li 2.0 which will change the way we represent URLs and make them easier to read as well as improve our performance. It will also feature easier to use and more consistent public APIs. Other plans include adding support for Rest.li level filters for Rest.li services and simplification of our request and response JSON structures.

(I would like to thank Joe Betz, Yevgeniy Brikman, Brandon Duncan, Christopher Eppstein, Steven Ihde, and Roman Inozemtsev for their help in writing this blog post)