Open Source Articles

  • lightweight-hardware-accelerated-video/audio-transcoder-for-android

    LiTr: A lightweight video/audio transcoder for Android

    December 19, 2019

    If a picture’s worth a thousand words, then what about a video? In 2017, we launched video sharing to give our members the ability to share video content on the feed via the LinkedIn mobile app or a web browser. When posting a video from an Android device, the member could either record it using their device camera app or pick an existing video from the gallery....

  • LinkedIn-Kafka-ecosystem

    How LinkedIn customizes Apache Kafka for 7 trillion messages per day

    October 8, 2019

    Co-authors: Jon Lee and Wesley Wu Apache Kafka is a core part of our infrastructure at LinkedIn. It was originally developed in-house as a stream processing platform and was subsequently open sourced, with a large external adoption rate today. While many other companies and projects leverage Kafka, few—if any—do so at LinkedIn’s scale. Kafka is used extensively...

  • isolation-tree

    Detecting and preventing abuse on LinkedIn using isolation forests

    August 13, 2019

    The Anti-Abuse AI Team at LinkedIn creates, deploys, and maintains models that detect and prevent various types of abuse, including the creation of fake accounts, member profile scraping, automated spam, and account takeovers. There are several unique challenges we face when using machine learning to prevent abuse on a large professional network, including:...

  • change-data-capture

    Open sourcing Brooklin: Near real-time data streaming at...

    July 16, 2019

    Brooklin - a distributed service for streaming data in near real-time and at scale - has been running in production at LinkedIn since...

  • star-tree-data-structure

    Star-tree index: Powering fast aggregations on Pinot

    June 14, 2019

    Pinot is an open source, scalable distributed OLAP data store that entered the Apache Incubation recently. Developed at LinkedIn, it...

  • iris-mobile-view

    Iris mobile: An open source, mobile interface for incident ...

    May 9, 2019

    At LinkedIn, our on-call incidents are managed using Iris and Oncall, two tools that we released as open source to the community about...