Scalability

Managing documentation at scale

LinkedIn’s engineering teams are distributed around the globe: from California to Bangalore, our engineers are constantly taking a members-first approach to help the world’s professionals. Today, that comes with a focus on mobile, which is how the majority of our members access our platform. With that in mind, Project Voyager, the name of our current mobile application, requires the most optimal set of processes and systems for our engineering team to execute at the highest level to meet our members’ expectations. Within Voyager iOS, the iPhone and iPad app, engineers are organized by our member-facing products, and mobile engineers are sprinkled throughout our organizational structure. With iOS engineers placed among our myriad teams, and specialized teams creating libraries for the many orthogonal aspects of iOS at LinkedIn, documentation is key to enable engineers—both experienced and new—to understand how to further build our iOS products.

When the Voyager iOS project was started in 2015, keeping the documentation hub, which we dubbed vidocs (short for Voyager iOS docs), up-to-date was fairly straightforward. New things were being built, and they were appropriately documented in the hub. However, after three years of development and growth, vidocs devolved into barely organized chaos. Not only was there an innumerable number of lines of code written since Project Voyager began, but there were also hundreds of new engineers contributing to the effort. We had to scale our systems for both more code, and more people working on that code.

documentationatscale2

Or in this instance, a docs organization tip (Image Source: xkcd.com)
 

The natural tendency in these situations is that certain engineers became silos of knowledge (i.e., the go-to people for their respective topic). While that works for a period of growth, the approach struggled to scale well across geographies and increasingly became a problem as our engineering team grew around the globe. It also put an immense burden on these knowledge keepers, who were constantly being pulled away from their core work to answer questions. Over time, our shared documentation space slowly decayed, losing organization and clarity month over month. It was apparent we needed to both update the hub and, more importantly, adopt a process so we wouldn’t run into this problem again.

While searching for a solution, we realized that we had recently faced a similar problem around standardizing code style at scale. To solve that problem, we created a separate platform reviewer group of experienced engineers tasked with reviewing and determining what libraries should be used and which high-level coding conventions should be employed across the independent teams delivering improvements to the application. Since its creation, the platform group has achieved relative success towards its goals, as seen by the high net satisfaction marks around the primary intention of providing consistency. With this in mind, it seemed like a logical next step to request support from this team to apply what they had done to code to documentation and similarly solve our dilemma once and for all.

The group of platform owners already knew that the documentation hub desperately needed attention, so it took no convincing for them to jump in and solve this problem. We discussed implementation and put together a proposal for how to clean up the documentation hub and how to maintain it after the clean up. To ease ongoing updates, the top-level page structure would mirror the underlying org structure and each page would be assigned a clear owner. For example, the page on accessibility would be assigned to the accessibility team, the page on our architecture would be assigned to our Voyager infrastructure team, and the frequently asked questions would be assigned to the platform group. By having clear owners for each page, those owners were empowered to maintain their respective spaces within the vidocs hub. In addition, on a rotational basis, a platform owner would comb through vidocs each quarter, notifying page owners of anything that looked like it needed updates. These additional responsibilities for platform owners were quite cohesive with their existing responsibilities.

To achieve this new structure, we first had to select which topics were worthy of being top-level pages. We narrowed the vast ensemble of 50 pages that we previously had down to 10 topics corresponding to our organizational structure. All information on internationalization was gathered into a single page, and the vast expanse of documents that described how to test various things was also unified. From there, we had to standardize the format of these pages, which were previously an assorted mix of reStructuredText (rst) and markdown (md) files. Lastly, we had to remove pages that were completely out of date, of which there were several that no longer contributed value, but no one had felt comfortable in taking the first step to remove.

Since the update, several improvements have already been noticed, including:

  • We’ve seen Voyager developer satisfaction with the platform group increase from a mid-fifties percent positive sentiment rating to more than 75 percent positive sentiment over the past six quarters.

  • It’s become extremely easy to point someone to the appropriate documentation page with full confidence that it’s up to date.

  • Several engineers have already reached out to the platform group for guidance on updating a section or topic, ensuring the best fit for the information they are looking to add or modify.

  • The dependence on tribal knowledge has shrunk and engineers are now able to find answers in a completely decentralized manner.

While obvious in hindsight, aligning our solution for vidocs with the already functional platform owners group was key in not only successfully updating vidocs, but also putting into place a system to keep it that way. It may have been tempting to find a new and unique solution to this problem, but by turning to a pre-existing solution that had paid off in the past, we avoided having to reinvent the wheel and more easily ensured the present and future success of vidocs. Choosing to structure our documentation by team was also pivotal in assigning clear ownership to various parts of what is otherwise a huge documentation hub unmanageable for any single person to maintain. Furthermore, instilling a process agnostic of any individual will be instrumental in ensuring vidocs stays updated in the long term.

The lesson: When faced with a tough problem, the solution might be right around the corner. Leveraging existing solutions is almost always simpler and less risky than reinventing the wheel.