Posts by Joel Koshy

This is the first post in a blog series adapted from my talk at the inaugural Kafka Summit. Apache Kafka is the backbone for various data pipelines and asynchronous messaging at LinkedIn and beyond. In fact, we were excited to learn at the Kafka Summit last month that a number of other big adopters like Netflix and Microsoft have hit similar levels of scale in...

Apache Kafka is a highly scalable messaging system that plays a critical role as LinkedIn’s central data pipeline. Kafka was developed at LinkedIn back in 2010, and it currently handles more than 1.4 trillion messages per day across over 1400 brokers. Kafka’s strong durability and low latency have enabled us to use Kafka to power a number of newer...

We are pleased to announce the release of Kafka 0.7.1. This is the second release of Kafka from the Apache incubator. Kafka is a distributed, persistent, high throughput messaging system for collecting and delivering a high volume of data with low latency. LinkedIn and several other companies use Kafka to track extremely large volumes of user-activity events. In...