Performance Articles

  • fixing-linux-file-system-performance-regressions

    Fixing Linux filesystem performance regressions

    October 16, 2020

    As companies grow, adapt, morph, and mature, one item remains the same: the need for reinvention. Technical infrastructure is no exception. As our member community grew, our priorities were to keep up with that growth, or as we say, ensure continuous “site up.” (Read: adding servers to scale from hundreds to hundreds of thousands.) We ran into challenges about...

  • table-comparing-the-nexmark-benchmark-results

    Building a better and faster Beam Samza runner

    October 1, 2020

    Co-authors: Yixing Zhang, Bingfeng Xia, Ke Wu, and Xinyu Liu Since Beam Samza runner was developed in 2018 at LinkedIn, we now have 100+ Samza Beam jobs running in production. As our usage grew, we wanted to better understand how the Samza runner performs compared to other runners and identify areas of improvement. In general, for stream processing platforms,...

  • diagram-showing-hadoop-dual-ingest-pipelines

    Theory vs. Practice: Learnings from a recent Hadoop incident

    August 6, 2020

    Co-authors: Sandhya Ramu and Vasanth Rajamani For companies and organizations, failure tends to be far more illuminating than success and the lingering effects of a failure can be harmful if the team moves too quickly and does not resolve the issue in a thorough and transparent manner. We recently ran into a large incident that involved data loss in our big data...

  • The impact of slow NFS on data systems

    June 23, 2020

    Espresso is LinkedIn's defacto NoSQL database solution. It is an online, distributed, fault-tolerant database that powers most of...

  • diagram-showing-the-architecture-of-identity-services

    How we reduced latency and cost-to-serve by merging two...

    April 22, 2020

    Co-authors: Xiang Zhang, Estella Pham, and Ke Wu Identity services are critical systems that serve data on profile and member settings...

  • architecture-of-insearch

    InSearch: LinkedIn’s new message search platform

    March 17, 2020

    Co-authors: Suruchi Shah and Hari Shankar The rise of instant messaging has changed how we communicate. Compared to the back-and-forth...