Authors: Bingfeng Xia and Xinyu Liu Background At LinkedIn, Apache Beam plays a pivotal role in stream processing infrastructures that process over 4 trillion events daily through more than 3,000 pipelines across multiple production data centers. This robust framework empowers near real-time data processing for critical services and platforms, ranging from...
Streaming Articles
-
-
It's not easy to quickly gather all the data that goes into a LinkedIn page view, particularly for something like our home page. LinkedIn benefits from a very distributed service-oriented architecture for assembling pages as quickly possible while still being resilient to failures from any particular bit of content. Each bit that ends up on the page is provided...
- Topics:
- Streaming,
- Performance,
- Samza,
- Open Source
-
At LinkedIn, we've been using the Play Framework in production for over a year. At the 2014 Ping Conference, I gave a talk about some of the techniques we've been experimenting with to deal with complexity and performance: Composition: the Play Framework is built around a functional programming paradigm, which makes it much easier to use composition to build...
- Topics:
- Performance,
- Streaming,
- Developer Productivity,
- Play,
- Tech Talk