Under the hood: How we built API versioning for LinkedIn Marketing APIs
Co-authors: Nihit Purwar, Demian Lessa, and Shruthi Jeganathan
At LinkedIn, the goal of our API team is to create a world-class platform that enables partners to build, run, and grow sustainable businesses. Developers use our Marketing APIs to build new and innovative solutions that complement LinkedIn Marketing Solutions and enable marketers to achieve more. While the program has seen success, it has also experienced some growing pains as our approach to change management did not always take into account all of the potential impacts to our partners and customers.
As our business grows, we aim to quickly deliver increased value with platform improvements. However, we were releasing breaking changes almost monthly with different sunset dates – making it hard for developers to test and plan their roadmap without a predictable release schedule. Unversioned APIs also blocked customers from accessing the latest features and caused internal challenges with new feature development. Additionally, feedback from developers indicated we were releasing too many API changes with short warning times that disrupted planning.
As fellow developers, we understood the benefits that versioning our APIs would bring, as it’s an important foundation for an API platform and could significantly improve the developer experience. So we set out to build an API change management system with extensive documentation and a detailed changelog.
This is the story of how we built API versioning for LinkedIn Marketing APIs.
Principles of LinkedIn API versioning
Before we built API versioning, we aligned on five core principles:
Slow is smooth, and smooth is fast. This concept is taken from racing. When you go slow, it’s actually smooth—and when you’re smooth, you go faster. We wanted to invest in well-planned, smooth releases that would yield faster product velocity.
Future-first architecture. Architectural decisions we make today will impact our product for years to come. Every choice should be forward-thinking and enable enhancements.
API first. New features will either be launched simultaneously on our partner API and our UI, or on our API platform first.
Don’t disrupt customers. Ensure we don’t break existing supported versions so that customers aren’t blocked in their marketing workflows.
Decoupling. External and internal APIs evolve independently.
Legacy architecture: The past
Before we launched versioning, API externalization at LinkedIn was based on models and APIs that were constantly changing.
External requests were routed to the Rest.li Gateway, the component responsible for mapping and routing requests to internal services. These internal services were typically implemented by line-of-business (LoB) owned systems (these are Rest.li services). From an internal development perspective, there was a lack of separation between development for customers and external developers.
New architecture: The future
Our vision for API externalization at LinkedIn is one in which changes to models and APIs are exposed under specific versions to effectively isolate external applications from changes and allow partners to migrate to new versions at their own speed. Typically, each external application integrates with one or more API products. Each API product exposes its versioned models and APIs through a mid-tier that is dedicated to serving APIs for external partners. The mid-tier integrates with any number of backends, which implement the features that are ultimately exposed to the external applications.
In the new architecture, each mid-tier exposes its features (APIs and models) for one API product, and each product is implemented by just one mid-tier. The APIs and models of a mid-tier are exposed under specific versions, and their implementations may call any number of backends, which are version agnostic by design. External applications request specific versions of APIs; the API Gateway translates and forwards the request to the appropriate mid-tier.
API Gateway
At a high level, the API Gateway is responsible for taking versioned API requests from external applications, validating versions, dispatching them for execution, and returning the corresponding responses.
The Gateway also provides additional features on top of request execution, as shown in the figure above. These aspects include:
Authentication. Checks that the request has a valid authentication token, then populates the invocation content with user and application identifiers.
Request mapping (and validation). Determines the internal resource and method to invoke to satisfy the external request, which includes a version header. This validates that the external path and version can be mapped to an internal resource and method at the specified version, and that the specified version is active.
Throttling control. Checks that the requestor does not violate any rate-limiting rules.
Authorization. Checks that the requestor is allowed to perform the specified request to call the corresponding internal resource and method.
Anti-abuse control. Checks that the requestor does not violate any abuse prevention rules.
Request dispatching. Sends the request to the internal service, waits for a response, and forwards the response to the requestor.
Marketing Developer Platform (MDP) mid-tier
The mid-tier exposes all the APIs and schemas associated with the respective API product (see: Getting Started). Mid-tiers expose APIs and schemas at specific versions and allow multiple versions to coexist.
In the previous figure, the MDP mid-tier exposes two versions of its APIs and schemas. Version v202201.01 exposes resources Foo and Bar, whereas v202202.01 exposes Foo, Bar, and Alice. Resources Foo and Bar in these versions may be identical, differ slightly, or differ significantly—there are no constraints on what is allowed across versions. However, from a business standpoint, the closer these resource versions are to each other, the better the developer experience will be for the external partners using these APIs and schemas.
While conceptually there are two versions of the Foo and Bar resources, these would effectively be distinct resources in the mid-tier (e.g., FooResource_v20220101 and FooResource_v20220201). In the code, there exists a way to allow both these Foo resources to indicate that they represent the same logical resource (Foo) in different versions. Gateway would be able to map the external Foo resource at version 202201.01 to the internal resource FooResource_v20220101, as well as the Foo resource at version 202202.01 to FooResource_v20220201.
Service
The Service (see previous diagram) is responsible for making calls to the backend downstream services. Having version-specific code paths can result in bifurcated and duplicated code that leads to code maintenance problems. The Service makes use of the “Versioning Execution Framework” library to ensure these issues do not occur.
The two main tasks of the framework are to:
Convert the request to the latest possible version: The conversion is done with the help of user-defined transformers. The conversion reduces the code paths required to maintain the schemas for each version and makes code maintenance easier.
Convert the request to the internal model: After the request is converted to the latest possible version, it is then converted to the internal model using a user-defined internal model converter. The internal model will be used in the business logic of the service, including any calls to the downstream services to serve the request.
API versioning is available now
API versioning for LinkedIn Marketing APIs is the result of a concerted effort from many teams here at LinkedIn. We explored multiple ways to build versioning before finalizing the current approach to ensure the best possible experience for external developers, which means external developers do not see any unintended behavior or schema changes being published. To further protect our developers, we also built tooling and tests to automatically detect changes in the downstream services.
We hope that our experience of building API versioning at LinkedIn has provided you with food for thought when considering your own API versioning. Now that you know how it was built, get started with LinkedIn API versioning today.
Related articles