• Creating Community Around Open Source, Working With Legacy Code, Architecture Hoisting and More

    September 30, 2015

    LinkedIn’s publishing platform gives professionals a way to share their personal opinions about topical professional news and interests, including our engineers. Here, we regularly round up some of the best pieces written recently by LinkedIn engineers. "Building Communities" By Todd Palino, Staff Site Reliability Engineer at LinkedIn Todd discusses how creating an open and welcoming community is essential to the success and continued development of open source projects. He...

  • Espresso Onboarding Experiences: InMail

    September 29, 2015

    Fast growth is a happy problem to have, but not an easy one to solve. LinkedIn has experienced rapid member growth over the years and many of our engineers have witnessed the corresponding explosive data growth in awe. Until recently, LinkedIn relied on a traditional RDBMS as a primary data store for most of our data. Hundreds of terabytes of data were organized into Oracle shards that were incrementally provisioned as member growth continued. Several problems surfaced including:...

  • Bridging Batch and Streaming Data Ingestion with Gobblin

    September 28, 2015

    Genesis Less than a year ago, we introduced Gobblin, a unified ingestion framework, to the world of Big Data. Since then, we’ve shared ongoing progress through a talk at Hadoop Summit and a paper at VLDB. Today, we’re announcing the open source release of Gobblin 0.5.0, a big milestone that includes Apache Kafka integration. Our motivations for building Gobblin stemmed from our operational challenges in building and maintaining disparate pipelines for different data sources...

  • Rewinder: Interactive Analysis of Hadoop's Computational Resources

    September 23, 2015

    Co-authors: Teja Thotapalli Brian Jue Tu Tran Sandhya Ramu As LinkedIn continues to grow in size and stature, the data volume being generated continues to...

  • Video: Tools Team Revolutionizes Software Development

    September 22, 2015

    In the last four years, we’ve dramatically revamped the way we release software. One of the leaders of our tools team Jens Pillgram-Larsen shares what...

  • Transparency: Metrics You Follow and Ramp Alert

    The Evolution of A/B Testing Platform at LinkedIn

    September 21, 2015

    “Doubt the conventional wisdom unless you can verify it with reason and experiment” - Steve Albini At LinkedIn, we experiment with new ideas before...