We are proud to announce the open sourcing of Venice, LinkedIn’s derived data platform that powers more than 1800 of our datasets and is leveraged by over 300 distinct applications. Venice is a high-throughput, low-latency, highly-available, horizontally-scalable, eventually-consistent storage system with first-class support for ingesting the output of batch and...
Venice Articles
-
- Topics:
- Venice,
- Platform,
- Data,
- Open Source
-
Venice, which was developed in late 2015, is a key-value store platform built for serving read-heavy workloads and optimized for serving derived data. Since being deployed to production in 2016, it has become very popular in the recommendation world to serve derived datasets inside LinkedIn. Venice handles single-get and small batch-get requests very efficiently...
- Topics:
- Performance,
- Venice,
- scalability
-
Sometimes, an engineering problem arises that might make us feel like maybe we don't know what we're doing, or at the very least, forces us out of the comfort zone of our area of expertise. That day came for the Venice team at Linkedin when we began to notice that some Venice processes would consume all available memory and crash if given enough time to run....
- Topics:
- Performance,
- Venice
-
Over the last two years at LinkedIn, we have been working on a distributed key-value database called “Venice.” Venice is designed to...
- Topics:
- Performance,
- Venice
-
We build a lot of our own infrastructure systems here at LinkedIn. Many people have heard of Kafka, our distributed message buffer. We...
- Topics:
- Venice,
- Craftsmanship,
- Distributed Systems
-
Editor's note: This blog has been updated. Background Like many internet companies, LinkedIn has faced data growth challenges....
- Topics:
- Venice,
- Distributed Systems,
- Cluster Management,
- helix