infrastructure Articles

  • PartitionConsumer-objects-distribution

    Auto-Tuning Pinot Real-Time Consumption

    July 11, 2019

    Pinot, a scalable distributed columnar OLAP data store developed at LinkedIn, delivers real-time analytics for site-facing use cases such as LinkedIn's Who viewed my profile, Talent insights, and more. Pinot uses Apache Helix for managing cluster resources and Apache Zookeeper to store metadata. Pinot has wide adoption at LinkedIn, ranging from internal...

  • High-level-architecture

    Expediting Data Fixes and Data Migrations

    July 9, 2019

    With over 630 million members, the LinkedIn platform delivers thousands of features that individually serve and store large amounts of diverse data. Protecting, maintaining, and serving data has always been of paramount importance for enriching the member experience and ensuring service reliability. In this blog post, we’ll address a critical part of data...

  • Espresso-online-data-flow-with-Netty4

    Improving Performance and Capacity for Espresso with New Netty Framework

    June 27, 2019

    In this blog post, we’ll share how we migrated Espresso, LinkedIn’s distributed data store, to a new Netty4-based framework and achieved a large performance and capacity gain throughout the Espresso system as a result. In the larger scheme, this is particularly important since Espresso is a master data hub that serves many important applications across LinkedIn,...

  • settings-architecture-diagram

    Building Member Trust Through a Centralized and Scalable...

    May 29, 2019

    At LinkedIn, we are entrusted with protecting the privacy and data of our members. Storing members’ choices on how they intend their...

  • featuredimage5

    Coding Conversations: Interviews on Replacing...

    December 18, 2018

    LinkedIn’s infrastructure has grown remarkably over the years, and that has kept the Production Infrastructure Engineering teams on...

  • managingdependency3

    Managing Software Dependency at Scale

    September 6, 2018

    Co-authors: Szymon Gizecki, Yu Li, Chinmaya Dattathri, Ethan Hall, Irina Issayeva, and Deep Majumder   Introduction At LinkedIn, we...