Performance Articles

  • diagram-showing-hadoop-dual-ingest-pipelines

    Theory vs. Practice: Learnings from a recent Hadoop incident

    August 6, 2020

    Co-authors: Sandhya Ramu and Vasanth Rajamani For companies and organizations, failure tends to be far more illuminating than success and the lingering effects of a failure can be harmful if the team moves too quickly and does not resolve the issue in a thorough and transparent manner. We recently ran into a large incident that involved data loss in our big data...

  • The impact of slow NFS on data systems

    June 23, 2020

    Espresso is LinkedIn's defacto NoSQL database solution. It is an online, distributed, fault-tolerant database that powers most of LinkedIn’s applications including member profiles, InMail (LinkedIn's member-to-member messaging system), sections of the main LinkedIn homepage, our mobile applications, and more. Since Espresso caters to many critical features, its...

  • diagram-showing-the-architecture-of-identity-services

    How we reduced latency and cost-to-serve by merging two systems

    April 22, 2020

    Co-authors: Xiang Zhang, Estella Pham, and Ke Wu Identity services are critical systems that serve data on profile and member settings to help power many other applications at LinkedIn. In this blog post, we’ll share how we merged two layers of the identity services that handle more than half a million queries per second (QPS) that drove a 10% reduction in...

  • architecture-of-insearch

    InSearch: LinkedIn’s new message search platform

    March 17, 2020

    Co-authors: Suruchi Shah and Hari Shankar The rise of instant messaging has changed how we communicate. Compared to the back-and-forth...

  • diagram-of-espressos-architecture

    How we improved latency through projection in Espresso

    March 5, 2020

    Co-authors: Xiang Zhang and Chuck Jerian Espresso is LinkedIn’s document-oriented, highly available, and timeline-consistent...

  • diagram-of-pinot-architecture

    Upgrading to RHEL7 with minimal interruptions

    January 14, 2020

    Pinot, an open source scalable distributed columnar OLAP data store, is used extensively across almost all areas of LinkedIn. It...