infrastructure Articles

  • under-the-hood-updating-linkedins-ui

    Under the hood: Updating LinkedIn's UI

    October 22, 2020

    Co-author: Jitesh Gandhi and Eric Babyak Across LinkedIn, we recently launched a new design that brings a warmer, more inclusive feeling to our products. Throughout the planning phases of this process, we were met with challenging questions in implementing this redesign, such as, “How do you modernize and refresh a user interface (UI) across three applications...

  • architecture-diagram-of-magnet

    Magnet: A scalable and performant shuffle architecture for Apache Spark

    October 21, 2020

    Co-authors: Min Shen, Chandni Singh, Ye Zhou, and Sunitha Beeram At LinkedIn, we rely heavily on offline data analytics for data-driven decision making. Over the years, Apache Spark has become the primary compute engine at LinkedIn to satisfy such data needs. With its unique features, Spark empowers many business-critical tasks at LinkedIn, including data...

  • fixing-linux-file-system-performance-regressions

    Fixing Linux filesystem performance regressions

    October 16, 2020

    As companies grow, adapt, morph, and mature, one item remains the same: the need for reinvention. Technical infrastructure is no exception. As our member community grew, our priorities were to keep up with that growth, or as we say, ensure continuous “site up.” (Read: adding servers to scale from hundreds to hundreds of thousands.) We ran into challenges about...

  • test-management-deployment-and-evaluation-workflow-at-linkedin

    Our evolution towards T-REX: The prehistory of...

    September 24, 2020

    Editor’s note: This blog post is the first in a series providing an overview and history of LinkedIn’s experimentation platform. At...

  • title-card

    LIquid: The soul of a new graph database, Part 2

    September 16, 2020

    Co-authors: Scott Meyer, Andrew Carter, and Andrew Rodriguez Editor’s note: This is the second part of a two-part blog series. Part 1...

  • dark-cluster-architecture-diagram

    Production testing with dark canaries

    September 10, 2020

    The internet software industry has moved away from long development cycles and dedicated quality assurance (QA) stages, toward a...