Co-authors: Jonathan Hung, Keqiu Hu, and Anthony Hsu LinkedIn heavily relies on artificial intelligence to deliver content and create economic opportunities for its 575+ million members. Following recent rapid advances of deep learning technologies, our AI engineers have started adopting deep neural networks in LinkedIn’s relevance-driven products, including...
Hadoop Articles
-
- Topics:
- Hadoop,
- machine learning,
- TensorFlow,
- Open Source
-
Co-authors: Erik Krogen and Min Shen In March 2015, LinkedIn’s Big Data Platform team experienced a crisis. As the team was preparing to head home for the day, signs of trouble began trickling in: our internal users were reporting that their applications were stalling or timing out. Job queues were backing up, and SLAs would be missed. A bit of investigation...
- Topics:
- Testing,
- Hadoop,
- Open Source
-
Co-authors: Carl Steinbach and Vasanth Rajamani Big challenges in the big data ecosystem At LinkedIn, we have a number of challenges managing data in our complex data ecosystem. Changes to our infrastructure are often necessary to make progress, but they are difficult to accomplish without an expensive, large-scale, coordinated effort. Analytics processing...
-
Next Tuesday marks the start of the Spark Summit Conference in San Francisco. This year, LinkedIn engineers and data scientists are...
- Topics:
- Spark,
- Hadoop,
- research,
- Dr. Elephant,
- events
-
This post has been updated to note the release of Pepperdata's Application Profiler, a commercial project based on Dr. Elephant. Last...
- Topics:
- Spark,
- Hadoop,
- Dr. Elephant,
- Open Source
-
About a year ago, we open sourced Gobblin, a universal data ingestion framework that aimed to solve data integration challenges faced...
- Topics:
- Hadoop,
- Big Data,
- Open Source,
- Apache Kafka,
- Data Ingestion,
- Distributed Systems,
- ETL,
- Gobblin