Co-authors: Jon Lee and Wesley Wu Apache Kafka is a core part of our infrastructure at LinkedIn. It was originally developed in-house as a stream processing platform and was subsequently open sourced, with a large external adoption rate today. While many other companies and projects leverage Kafka, few—if any—do so at LinkedIn’s scale. Kafka is used extensively...
Open Source Articles
-
- Topics:
- scale,
- Kafka,
- Open Source
-
The Anti-Abuse AI Team at LinkedIn creates, deploys, and maintains models that detect and prevent various types of abuse, including the creation of fake accounts, member profile scraping, automated spam, and account takeovers. There are several unique challenges we face when using machine learning to prevent abuse on a large professional network, including:...
- Topics:
- machine learning,
- Open Source
-
Brooklin - a distributed service for streaming data in near real-time and at scale - has been running in production at LinkedIn since 2016, powering thousands of data streams and over 2 trillion messages per day. Today, we are pleased to announce the open-sourcing of Brooklin and that the source code is available in our Github repo! Why Brooklin? At LinkedIn,...
- Topics:
- scale,
- Stream Processing,
- infrastructure,
- Kafka,
- Open Source
-
Pinot is an open source, scalable distributed OLAP data store that entered the Apache Incubation recently. Developed at LinkedIn, it...
- Topics:
- Pinot,
- Distributed Systems,
- Open Source
-
At LinkedIn, our on-call incidents are managed using Iris and Oncall, two tools that we released as open source to the community about...
- Topics:
- Mobile,
- Open Source,
- SRE
-
Introduction Three years ago, LinkedIn announced and open sourced Ambry, a distributed, highly available and horizontally scalable...
- Topics:
- Storage,
- Open Source