At LinkedIn, Apache Kafka is used heavily to store all kinds of data, such as member activity, log storage, metrics storage, and a multitude of inter-service messaging. LinkedIn maintains multiple data centers with multiple Kafka clusters per data center, each of which contains an independent set of data. Mirroring (i.e., replicating) Kafka topics across the...
Pinot Articles
-
- Topics:
- Pinot,
- Kafka,
- Open Source
-
Co-authors: Siddharth Teotia and Tim Santos Introduction LinkedIn Talent Insights (LTI) is a platform that helps organizations understand the external labor market and their internal workforce, and enables the long term success of their employees. Users of LTI have the flexibility to construct searches using the various facets of the LinkedIn Economic Graph...
-
Co-authors: Vincent Wang, Siddharth Teotia, Manoj Thakur, and Mayank Shrivastava As our LinkedIn Marketing Solutions Blog recently noted, companies and marketers “are once again peering ahead, setting their plans for success in a reshaped business environment.” One of the items businesses rely on to do this are insights, including the estimated reach of an...
-
Co-authors: Xiang Zhang and Jingyu Zhu Introduction The Lambda architecture has become a popular architectural style that promises...
- Topics:
- Stream Processing,
- Pinot,
- Profile,
- Architecture,
- Kafka,
- batch processing
-
Co-authors: Khai Tran and Steve Weiss Batch and streaming computations are often combined together in the Lambda architecture, but...
- Topics:
- Stream Processing,
- batch processing,
- Data,
- Pinot,
- Gobblin,
- Kafka,
- Samza
-
Co-authors: Timothy Santos and Jeremy Lwanga LinkedIn is a mission-driven organization, and we take our mission of “connecting the...
- Topics:
- Pinot,
- Product Design