When dealing with massive amounts of data at scale, it’s important to have state of the art infrastructure and algorithms that can make sense of all that information. I lead the Data Analytics Infrastructure team at LinkedIn; we work on core infrastructure components such as Hadoop and Spark, as well as develop other infrastructure to support analytics use cases...
Cubert Articles
-
- Topics:
- Pinot,
- data analytics,
- Cubert,
- infrastructure,
- Gobblin
-
Authors: Maneesh Varshney, Srinivas Vemuri What do you do when your Hadoop ETL script is mercilessly killed because it is hogging too many resources on the cluster, or if it starts missing completion deadlines by hours? We encountered this exact same problem more than a year ago while building the computation pipeline for XLNT, LinkedIn’s A/B testing platform....
- Topics:
- Big Data,
- Hadoop,
- Cubert,
- Open Source