scale Articles

  • stages-in-continuous-deployment

    Learnings from the journey to continuous deployment

    October 18, 2019

    As an engineer, your goal is for every commit to seamlessly land in production and provide a delightful experience for your customers. While frequent releases give you the ability to iterate and apply feedback quickly, they also require significant time, effort, and cost to achieve. In this blog post, we’ll explore the components of our development ecosystem...

  • LinkedIn-Kafka-ecosystem

    How LinkedIn customizes Apache Kafka for 7 trillion messages per day

    October 8, 2019

    Co-authors: Jon Lee and Wesley Wu Apache Kafka is a core part of our infrastructure at LinkedIn. It was originally developed in-house as a stream processing platform and was subsequently open sourced, with a large external adoption rate today. While many other companies and projects leverage Kafka, few—if any—do so at LinkedIn’s scale. Kafka is used extensively...

  • nuage-architecture

    Solving manageability challenges at scale with Nuage

    September 16, 2019

    Introduction LinkedIn is committed to providing economic opportunities for every member of the global workforce, and we’re growing at a rapid pace. Our platform is built on a collection of large-scale multi-cluster services functioning in harmony to offer a unified product experience to members.  The aim of several backend engineering teams is to allow other...

  • pipeline-cache

    Who Depends On Me? Serving Dependency Queries at Scale

    August 8, 2019

    Co-authors: Yu Li, Szymon Gizecki, and Chinmaya Dattathri To ensure we have significant flexibility in how our teams collaborate, our...

  • LinkedIn_Tech_Update

    Building the next version of our infrastructure

    July 23, 2019

    The pursuit of our mission to connect the world’s professionals to make them more productive and successful is deeply dependent on the...

  • change-data-capture

    Open Sourcing Brooklin: Near Real-Time Data Streaming at...

    July 16, 2019

    Brooklin - a distributed service for streaming data in near real-time and at scale - has been running in production at LinkedIn since...