Co-authors: Keqiu Hu, Jonathan Hung, Haibo Chen, and Sriram Rao At LinkedIn, we use Hadoop as our backbone for big data analytics and machine learning. With an exponentially growing data volume, and the company heavily investing in machine learning and data science, we have been doubling our cluster size year over year to match the compute workload growth. Our...
Hadoop Articles
-
- Topics:
- Hadoop,
- Data,
- Open Source
-
Co-authors: Keqiu Hu, Jonathan Hung, and Junfan Zhang Today, TonY is joining the LF AI & Data Foundation, an umbrella foundation of the Linux Foundation supporting open source innovation in artificial intelligence, machine learning, and deep learning. “We’re thrilled to welcome TonY into incubation in LF AI & Data. The project offers functionalities that are not...
- Topics:
- artificial intelligence,
- Hadoop,
- machine learning,
- Open Source
-
Co-authors: Konstantin V. Shvachko, Chen Liang, and Simbarashe Dzinamarira LinkedIn runs its big data analytics on Hadoop. During the last five years, the analytics infrastructure has experienced tremendous growth, almost doubling every year in data size, compute workloads, and in all other dimensions. It recently reached two important milestones. LinkedIn now...
- Topics:
- scale,
- Hadoop,
- Distributed Systems,
- infrastructure
-
Co-authors: Zihan Li, Sudarshan Vasudevan, Lei Sun, and Shirshanka Das Data analytics and AI power many business-critical use cases at...
- Topics:
- Stream Processing,
- Hadoop,
- Data,
- batch processing,
- Open Source,
- Gobblin,
- Kafka
-
Co-authors: Sandhya Ramu and Vasanth Rajamani For companies and organizations, failure tends to be far more illuminating than success...
- Topics:
- Performance,
- Hadoop,
- data center,
- Data
-
Co-authors: Cong Gu, Abin Shahab, Chen Qiang, and Keqiu Hu Editor's note: This blog has been updated. LinkedIn AI has been...
- Topics:
- Hadoop,
- Security,
- Open Source