Editor's note: This blog has been updated. Background Like many internet companies, LinkedIn has faced data growth challenges. Naturally, distributed storage systems became the solution to handle larger volumes of data and queries per second (QPS). But, aside from scaling issues, the variability in access patterns also grew quickly. For example, some scenarios...
Cluster Management Articles
-
- Topics:
- Venice,
- Distributed Systems,
- Cluster Management,
- helix
-
This post originally appeared as a contributed piece on The New Stack. Distributed data systems are used in a variety of settings like online serving, offline analytics, data transport, and search, among other use cases. These start off with a single node solution that provides the core functionality e.g. it can be a database, messaging, search index etc....
-
Two years ago we hit a wall. The scale of LinkedIn’s data was growing beyond what we could analyze. At the same time, our members needed their analytics and insights in real-time. We needed a solid solution that would grow with LinkedIn and serve as the platform to power all of our analytics needs across the company at web-scale. In this post, we will share how...
-
Apache Helix is a generic cluster management framework used for the automatic management of partitioned and replicated distributed...
- Topics:
- Cluster Management,
- Distributed Systems,
- Open Source
-
At LinkedIn, many individual services integrate together to deliver a reliable and consistent end-user experience. Although each...
- Topics:
- Cluster Management,
- Distributed Systems,
- Open Source
-
We are very pleased to announce the open source release of Helix- a generic cluster management system for managing partitioned and...
- Topics:
- Cluster Management,
- Distributed Systems,
- Open Source