Co-authors: Khai Tran and Steve Weiss Batch and streaming computations are often combined together in the Lambda architecture, but carry the cost of maintaining two different code bases for the same logic. We have previously shared on the blog a behind-the-scenes look at our approach into enabling the seamless translation of declarative batch code into streaming...
Stream Processing Articles
-
- Topics:
- Stream Processing,
- batch processing,
- Data,
- Pinot,
- Gobblin,
- Kafka,
- Samza
-
Brooklin - a distributed service for streaming data in near real-time and at scale - has been running in production at LinkedIn since 2016, powering thousands of data streams and over 2 trillion messages per day. Today, we are pleased to announce the open-sourcing of Brooklin and that the source code is available in our Github repo! Why Brooklin? At LinkedIn,...
- Topics:
- scale,
- Stream Processing,
- infrastructure,
- Kafka,
- Data,
- Open Source
-
If Apache Kafka is the lifeblood of all nearline processing at LinkedIn, then Apache Samza is the beating heart pumping that blood around. Samza at LinkedIn is provided as a managed stream processing service where applications bring their logic (leveraging the wide variety of Samza APIs), while the service handles the hosting, managing, and operations of the...
- Topics:
- Stream Processing,
- Testing,
- Samza
-
The existing Lambda architecture With the evolution of big data technologies over time, two classes of computations have been...
- Topics:
- Stream Processing,
- batch processing,
- Data
-
We are pleased to announce today the release of Samza 1.0, a significant milestone in the history of the project. Apache Samza is a...
- Topics:
- Stream Processing,
- Kafka,
- Open Source,
- Samza
-
A few years ago, we announced Rest.li 2.x and a Protocol Upgrade Story. Today, we are excited to share another major milestone: the...
- Topics:
- Stream Processing,
- rest.li,
- Data,
- Open Source