Kafka Articles

  • concourse-1

    Concourse: Generating Personalized Content Notifications in Near-Real-Time

    May 25, 2018

    Co-authors: Vivek Nelamangala and PJ Xiao   Introduction to Notifications Social media are computer-mediated platforms that facilitate creation and sharing of information, ideas, career interests, and other forms of expression via virtual communities and networks. In the last few years, notifications have been proven to be the mainstay for surfacing social media...

  • samza-aeon-1

    Samza Aeon: Latency Insights for Asynchronous One-Way Flows

    April 19, 2018

    Co-authors: Max Wolffe and Akhilesh Gupta Introduction You can’t fix something if you don’t know there’s a problem. Measuring and tracking the latency of requests through your system is essential to identifying and resolving issues quickly. Many systems have built-in tooling which allows developers to monitor, set alerts on, and inspect the latency of a set of...

  • venice1

    Venice Hybrid: Doing Lambda Better

    December 20, 2017

    Over the last two years at LinkedIn, I’ve been working on a distributed key-value database called “Venice.” Venice is designed to be a significant improvement to Voldemort Read-Only for serving derived data. In late 2016, Venice started serving production traffic for batch use cases that were very similar to the existing uses of Voldemort Read-Only. In the time...

  • cruisecontrol2

    Open Sourcing Kafka Cruise Control

    August 28, 2017

    Apache Kafka's popularity has grown tremendously over the past few years. In fact, LinkedIn's deployment recently surpassed 2 trillion...

  • Inception1

    Inception: How LinkedIn Deals with Exception Logs

    December 16, 2016

    Coauthors: Toon Sripatanaskul and Zhengyu Cai In early 2012, the LinkedIn Performance team was trying to build a tool to validate the...

  • ShareAnalytics1

    New Analytics for Sharing on LinkedIn: See Who’s Viewed...

    November 1, 2016

    Coauthor: Bharath Kumarasubramanian   If you’re sharing content on LinkedIn, you’re positioning yourself as a thought leader among the...