• change-data-capture

    Open Sourcing Brooklin: Near Real-Time Data Streaming at Scale

    July 16, 2019

    Brooklin - a distributed service for streaming data in near real-time and at scale - has been running in production at LinkedIn since 2016, powering thousands of data streams and over 2 trillion messages per day. Today, we are pleased to announce the open-sourcing of Brooklin and that the source code is available in our Github repo!  Why Brooklin? At LinkedIn,...

  • PartitionConsumer-objects-distribution

    Auto-Tuning Pinot Real-Time Consumption

    July 11, 2019

    Pinot, a scalable distributed columnar OLAP data store developed at LinkedIn, delivers real-time analytics for site-facing use cases such as LinkedIn's Who viewed my profile, Talent insights, and more. Pinot uses Apache Helix for managing cluster resources and Apache Zookeeper to store metadata. Pinot has wide adoption at LinkedIn, ranging from internal...

  • High-level-architecture

    Expediting Data Fixes and Data Migrations

    July 9, 2019

    With over 630 million members, the LinkedIn platform delivers thousands of features that individually serve and store large amounts of diverse data. Protecting, maintaining, and serving data has always been of paramount importance for enriching the member experience and ensuring service reliability. In this blog post, we’ll address a critical part of data...

  • WomenConnect_table_discussion

    Driving Change with Authenticity and Allyship at...

    July 2, 2019

    WomenConnect is a series of events focused on bringing together women in tech to build meaningful relationships. Over the past four...

  • Espresso-online-data-flow-with-Netty4

    Improving Performance and Capacity for Espresso with New...

    June 27, 2019

    In this blog post, we’ll share how we migrated Espresso, LinkedIn’s distributed data store, to a new Netty4-based framework and...

  • FollowFeed-architecture

    Community-Focused Feed Optimization

    June 25, 2019

    Co-authors: Ali Mohamed and Zheng Li   LinkedIn’s feed stands at the center of building global professional knowledge-sharing...