LinkedIn delivers value to more than 660 million members via thousands of microservices, most of which depend on data infrastructure or mid-tier infrastructure platforms. This means that in order to launch a new application, developers traditionally had to request and set up an online database as a business data source of truth or stream data via Kafka topics,...
infrastructure Articles
-
- Topics:
- infrastructure,
- Data
-
Co-authors: Krishnan Raman and Joey Salacup Monitoring big data pipelines often equates to waiting for a long-running batch job to complete and observing the status of the execution. The status can result in “Failed” or “Successful” or even “Incomplete.” From there, it’s the team’s job to understand the impact and troubleshoot the situation to identify a...
- Topics:
- infrastructure,
- Gobblin,
- tools,
- Kafka,
- Data
-
Introduction LinkedIn is committed to providing economic opportunities for every member of the global workforce, and we’re growing at a rapid pace. Our platform is built on a collection of large-scale multi-cluster services functioning in harmony to offer a unified product experience to members. The aim of several backend engineering teams is to allow other...
- Topics:
- scale,
- infrastructure,
- Data
-
Co-authors: Mars Lan, Seyi Adebajo, Shirshanka Das As the operator of the world’s largest professional network and the Economic Graph,...
- Topics:
- Metadata,
- data science,
- infrastructure,
- Data
-
The pursuit of our mission to connect the world’s professionals to make them more productive and successful is deeply dependent on the...
- Topics:
- scale,
- infrastructure
-
Brooklin - a distributed service for streaming data in near real-time and at scale - has been running in production at LinkedIn since...
- Topics:
- scale,
- Stream Processing,
- infrastructure,
- Kafka,
- Data,
- Open Source