scale Articles

  • How LIquid Connects Everything So Our Members Can Do Anything

    May 2, 2023

    Imagine a tool that can store and connect all the information you need to make decisions and solve problems. Most people would say it’s nice to think about, but not yet possible. The good news is this tool already exists - and it’s called a graph database. At LinkedIn, technologies like graph databases are essential to powering today's platform, while being...

  • Co-authors: Kenneth Tay and Xiaofeng Wang At Linkedin, we constantly evaluate the value our products and services deliver, so that we can provide the best possible experiences for our members and customers. This includes understanding how product changes impact key metrics related to those experiences. However, simply looking at connections between product...

  • new-dualip-project-logo

    DuaLip: Solving extreme-scale linear programs for web applications

    August 6, 2021

    Co-authors: Kinjal Basu, Yao Pan, Rohan Ramanath, Konstantin Salomatin, Amol Ghoting, and S. Sathiya Keerthi Building thriving internet or web products requires optimizing for multiple business metrics. In many applications, these business metrics can move in conflicting directions, and delicately balancing these tradeoffs can be vital for the success of a...

  • gif-showing-new-recruiter-and-jobs-experience

    New Recruiter & Jobs: The largest enterprise data migration...

    July 23, 2021

    Co-authors: Xiaoyang Gu, Xie Lu, and Xiaoguang Wang Introduction In August 2019, we introduced our members and customers to the idea...

  • current-architecture-of-the-daily-executive-dashboard-pipeline

    From daily dashboards to enterprise grade data pipelines

    July 20, 2021

    Within a matter of hours of each day beginning, we ingest tens of billions of records from online sources to HDFS, aggregated across...

  • chart-showing-exponential-growth-of-data-metadata-and-compute-on-linkedins-largest-hadoop-cluster

    The exabyte club: LinkedIn’s journey of scaling the Hadoop ...

    May 27, 2021

    Co-authors: Konstantin V. Shvachko, Chen Liang, and Simbarashe Dzinamarira LinkedIn runs its big data analytics on Hadoop. During the...