infrastructure Articles

  • building-a-platform-for-assessments

    Building a platform: Learnings from our pursuit for leverage

    April 3, 2020

    Co-authors: Jefferson Lai, Vinyas Maddi, and Vijay Ramamurthy One of our core engineering principles here at LinkedIn is to create leverage. In practice, what this means is to build software that is easy to reuse by other teams. An effective way we are able to do this is to favor platforms that generalize long-term functionality rather than build inflexible,...

  • architecture-of-insearch

    InSearch: LinkedIn’s new message search platform

    March 17, 2020

    Co-authors: Suruchi Shah and Hari Shankar The rise of instant messaging has changed how we communicate. Compared to the back-and-forth of email, we send and receive messages at much higher volumes and speeds. In the instant conversations we have, we also expect to be able to search for important phrases, moments, or references with ease. This rapid increase in...

  • datahub-logo

    Open sourcing DataHub: LinkedIn’s metadata search and discovery platform

    February 18, 2020

    Co-authors: Kerem Sahin, Mars Lan, and Shirshanka Das Finding the right data quickly is critical for any company that relies on big data insights to make data-driven decisions. Not only does this impact the productivity of data users (including analysts, machine learning developers, data scientists, and data engineers), but it also has a direct impact on end...

  • The-new-collaboration-pattern-with-rest-li-and-pegasus

    Empowering our developers with the Nuage SDK

    November 19, 2019

    LinkedIn delivers value to more than 660 million members via thousands of microservices, most of which depend on data infrastructure...

  • lag-alert-graphs

    An inside look at LinkedIn’s data pipeline monitoring...

    October 30, 2019

    Co-authors: Krishnan Raman and Joey Salacup   Monitoring big data pipelines often equates to waiting for a long-running batch job to...

  • nuage-architecture

    Solving manageability challenges at scale with Nuage

    September 16, 2019

    Introduction LinkedIn is committed to providing economic opportunities for every member of the global workforce, and we’re growing at...