Co-authors: Weiwei Guo, Xiaowei Liu, Sida Wang, Huiji Gao, and Bo Long Natural language processing (NLP) technologies are widely deployed to process rich natural language text data for search and recommender systems. Achieving high-quality search and recommendation results requires that information, such as user queries and documents, be processed and understood...
Open Source Articles
-
-
Co-authors: Cong Gu, Abin Shahab, Chen Qiang, Keqiu Hu Editor's note: This post was updated on June 10, 2020 LinkedIn AI has been traditionally Hadoop/YARN based, and we operate one of the world’s largest Hadoop data lakes, with over 4,500 users and 500PB of data. In the last few years, Kubernetes has also become very popular at LinkedIn for Artificial...
- Topics:
- Hadoop,
- Security,
- Open Source
-
Co-authors: Jun Shi, Mingzhou Zhou Introduction In the machine learning community, Apache Spark is widely used for data processing due to its efficiency in SQL-style operations, while TensorFlow is one of the most popular frameworks for model training. Although there are some data formats supported by both tools, TFRecord—the data format native to TensorFlow—is...
- Topics:
- Spark,
- machine learning,
- TensorFlow,
- Data,
- Open Source
-
Built at LinkedIn, Pinot is an open source, distributed, and scalable OLAP data store that we use as our de-facto near-real-time...
- Topics:
- Pinot,
- metrics,
- Data,
- Open Source
-
Co-authors: Kerem Sahin, Mars Lan, and Shirshanka Das Finding the right data quickly is critical for any company that relies on big...
- Topics:
- Metadata,
- infrastructure,
- Data,
- Open Source
-
Nearly 20 years after the first release of Python 2 and 11 years after the first release of Python 3, the Python development community...
- Topics:
- Python,
- Open Source