Search results for "kafka"

  • top-blogs-logos

    The Top 2019 LinkedIn Engineering Blogs

    December 9, 2019

    As the year draws to a close, we’re taking a look back at ten of our most popular 2019 articles on the LinkedIn Engineering Blog. Examining the list, it’s clear that topics pertaining to open source and artificial intelligence are some of the most popular, as are posts that look at how we tackle technical challenges at scale. We’re excited to share new progress...

  • lag-alert-graphs

    An inside look at LinkedIn’s data pipeline monitoring system

    October 30, 2019

    Co-authors: Krishnan Raman and Joey Salacup   Monitoring big data pipelines often equates to waiting for a long-running batch job to complete and observing the status of the execution. The status can result in “Failed” or “Successful” or even “Incomplete.” From there, it’s the team’s job to understand the impact and troubleshoot the situation to identify a...

  • LinkedIn-Kafka-ecosystem

    How LinkedIn customizes Apache Kafka for 7 trillion messages per day

    October 8, 2019

    Co-authors: Jon Lee and Wesley Wu Apache Kafka is a core part of our infrastructure at LinkedIn. It was originally developed in-house as a stream processing platform and was subsequently open sourced, with a large external adoption rate today. While many other companies and projects leverage Kafka, few—if any—do so at LinkedIn’s scale. Kafka is used extensively...

  • FollowFeed-flow

    Auditing Content Features in FollowFeed

    August 27, 2019

    The LinkedIn feed relies on a ranked list of the most relevant content for a member. More than 80% of the feed is organic content...

  • change-data-capture

    Open Sourcing Brooklin: Near Real-Time Data Streaming at...

    July 16, 2019

    Brooklin - a distributed service for streaming data in near real-time and at scale - has been running in production at LinkedIn since...

  • ccfe1

    Introducing Kafka Cruise Control Frontend

    February 7, 2019

    At LinkedIn, Kafka is the de-facto messaging platform that powers diverse sets of geographically-distributed applications at scale....