Co-authors: Jun Shi, Mingzhou Zhou Introduction In the machine learning community, Apache Spark is widely used for data processing due to its efficiency in SQL-style operations, while TensorFlow is one of the most popular frameworks for model training. Although there are some data formats supported by both tools, TFRecord—the data format native to TensorFlow—is...
Open Source Articles
-
- Topics:
- Spark,
- machine learning,
- TensorFlow,
- Data,
- Open Source
-
Built at LinkedIn, Pinot is an open source, distributed, and scalable OLAP data store that we use as our de-facto near-real-time analytics service. We’ve previously discussed how and why we built Pinot to power a wide spectrum of use cases, including internal business intelligence dashboards to analyze highly-dimensional data and “Who Viewed My Profile” to...
- Topics:
- Pinot,
- metrics,
- Data,
- Open Source
-
Co-authors: Kerem Sahin, Mars Lan, and Shirshanka Das Finding the right data quickly is critical for any company that relies on big data insights to make data-driven decisions. Not only does this impact the productivity of data users (including analysts, machine learning developers, data scientists, and data engineers), but it also has a direct impact on end...
- Topics:
- Metadata,
- infrastructure,
- Data,
- Open Source
-
Nearly 20 years after the first release of Python 2 and 11 years after the first release of Python 3, the Python development community...
- Topics:
- Python,
- Open Source
-
If a picture’s worth a thousand words, then what about a video? In 2017, we launched video sharing to give our members the ability to...
- Topics:
- Android,
- Mobile,
- NYC Engineering,
- Data,
- Open Source
-
Co-authors: Jon Lee and Wesley Wu Apache Kafka is a core part of our infrastructure at LinkedIn. It was originally developed in-house...
- Topics:
- scale,
- Kafka,
- Data,
- Open Source