infrastructure Articles

  • gif-showing-new-recruiter-and-jobs-experience

    New Recruiter & Jobs: The largest enterprise data migration at LinkedIn

    July 23, 2021

    Co-authors: Xiaoyang Gu, Xie Lu, and Xiaoguang Wang Introduction In August 2019, we introduced our members and customers to the idea of moving LinkedIn’s two core talent products—Jobs and Recruiter—onto a single platform to help talent professionals be even more productive. This single platform is called the New Recruiter & Jobs. Figure 1: New Recruiter & Jobs...

  • chart-showing-exponential-growth-of-data-metadata-and-compute-on-linkedins-largest-hadoop-cluster

    The exabyte club: LinkedIn’s journey of scaling the Hadoop Distributed File System

    May 27, 2021

    Co-authors: Konstantin V. Shvachko, Chen Liang, and Simbarashe Dzinamarira LinkedIn runs its big data analytics on Hadoop. During the last five years, the analytics infrastructure has experienced tremendous growth, almost doubling every year in data size, compute workloads, and in all other dimensions. It recently reached two important milestones. LinkedIn now...

  • host-wise-latency-to-detect-outliers-and-single-node-failures-this-graph-shows-four-outliers-from-three-hosts

    Rethinking site capacity projections with Capacity Analyzer

    March 16, 2021

    While site outages are inevitable, it’s our job to minimize both the duration of outages and the likelihood for an outage to occur. One of our preemptive measures is in the way we determine overall site capacity and health on an everyday basis, in that we load-test in production. There’s an elegant system to bucket and route members to specific data centers from...

  • diagram-for-variant-assignment-for-a-sample-population

    A/B testing at LinkedIn: Assigning variants at scale

    December 16, 2020

    Co-authors: Alexander Ivaniuk and Weitao Duan Editor’s note: This blog post is the second in a series providing an overview and...

  • behind-the-scenes-of-the-new-linkedin-jobs-experience

    Behind the scenes: The new LinkedIn Jobs experience

    October 28, 2020

    Co-authors: Hang Li, David Huang, Shuangfei Li, and Shannon Zhang A robust hiring ecosystem is one that serves the needs of both...

  • under-the-hood-updating-linkedins-ui

    Under the hood: Updating LinkedIn's UI

    October 22, 2020

    Co-author: Jitesh Gandhi and Eric Babyak Across LinkedIn, we recently launched a new design that brings a warmer, more inclusive...