• diagram-showing-the-act-listen-share-framework-that-real-time-feedback-enables

    How LinkedIn turned to real-time feedback for developer tooling

    January 12, 2021

    Over the last year, we have been using real-time feedback to evolve our tooling and provide a more productive experience for LinkedIn's developers. It’s helped us double our feedback participation, and more importantly, better tailor our recommendations and improvements. For any engineering organization looking to improve developer experiences, the following...

  • FastIngest: Low-latency Gobblin with Apache Iceberg and ORC format

    January 6, 2021

    Co-authors: Zihan Li, Sudarshan Vasudevan, Lei Sun, and Shirshanka Das Data analytics and AI power many business-critical use cases at LinkedIn. We need to ingest data in a timely and reliable way from a variety of sources, including Kafka, Oracle, and Espresso, bringing it into our Hadoop data lake for subsequent processing by AI and data science pipelines. We...

  • diagram-for-variant-assignment-for-a-sample-population

    A/B testing at LinkedIn: Assigning variants at scale

    December 16, 2020

    Co-authors: Alexander Ivaniuk and Weitao Duan Editor’s note: This blog post is the second in a series providing an overview and history of LinkedIn’s experimentation platform. The previous post on the history of LinkedIn’s experimentation infrastructure can be found here. Introducing variant assignment Previously on the blog, we’ve shared a look into how...

  • coral-a-sql-translation-analysis-and-rewrite-engine

    Coral: A SQL translation, analysis, and rewrite engine for ...

    December 10, 2020

    Co-authors: Walaa Eldin Moustafa, Wenye Zhang, Sushant Raikar, Raymond Lam, Ron Hu, Shardul Mahadik, Laura Chen, Khai Tran, Chris Chen...

  • explaining-metadata-architectures

    DataHub: Popular metadata architectures explained

    December 7, 2020

    When I started my journey at LinkedIn ten years ago, the company was just beginning to experience extreme growth in the volume,...

  • from-lambda-to-lambdaless-lessons-learned

    From Lambda to Lambda-less: Lessons learned

    December 1, 2020

    Co-authors: Xiang Zhang and Jingyu Zhu Introduction The Lambda architecture has become a popular architectural style that promises...