Last week, we released Connected, the latest addition to LinkedIn’s suite of apps focused on doing one thing really well. It surfaces contextually relevant relationship opportunities like job changes, work anniversaries and mentions in the news for people in your network. These are examples of what we call professional relationship opportunities. It is a chance to connect to people in your network and nurture your relationships.
We have introduced a beautiful new card based design for acting on these relationship opportunities which are ranked intelligently to present the most relevant opportunities up front. Notice that we also give you some insights about the person on the card which are a great way to start a conversation.
Anticipatory Computing & Pre-Meeting Intelligence
Connected is our first foray into anticipatory computing where members can choose to opt-in to receive push notifications that are used to provide timely updates about your network based on people you are going to meet or recently met. Here are a few examples of notifications that one of our members might receive.
- "You recently met Caitlin Crump. Connect and see more opportunities to reach out."
- "When you see Sara Adams in a few minutes, say happy work anniversary at LinkedIn."
- "Get to know Mauroof Ahmed before you meet in a few minutes."
- "Do a quick read on the news article Reid was mentioned in before you meet."
Our goal has been to make these notifications customized, relevant to the current context and useful. Working towards this presented some major engineering challenges.
- How do we make sure that notifications can be scheduled dynamically based on events in the calendar as they get added, updated and deleted?
- What is a good way to scale a scheduler that can schedule recurring notifications based on events for so many members?
- What happens when the scheduler fires? How do we look for relevant content between the member and the people she is about to meet and make intelligent guesses about important opportunities that she would like to know about?
In the first engineering post, we described the various components of our new Relationship Opportunities On Demand (ROPOD) platform. The Significant Context Recognition and Monitoring (SCRAM) system identifies significant changes to a member's context - starting with their calendar. Opportunist is the system that generates and sends push notifications at the right time. In this post, let us dive deeper into how these systems work together to enable pre-meeting intelligence by solving the challenges mentioned above.
Calendar Synchronization
The iOS calendar is a rich source of events from a lot of different sources. If the member opts-in to the meeting intelligence feature, specific calendar event information like title, location, time and attendees is periodically uploaded from the device via a secure SSL connection using the iOS 7 background fetch methodology. The uploaded calendar events are formatted by the front end and passed on to the calendar service.
Since calendar synchronization can be a time consuming job, the calendar service stores the uploaded calendar events temporarily in an incoming event store. It starts a calendar synchronization job and responds successfully without waiting for the job to complete.
The incoming calendar events now need to be merged with any existing events for the same time period. A simple merge is performed based on the IDs of the calendar events. Incoming events are preferred over existing ones in the case of conflicts. As part of the merge process, event attendees are mapped to their LinkedIn profiles using their email addresses so that we can use that information later to provide useful insights about the attendees. Finally, calendar data is stored securely by encrypting all the sensitive fields.
Now, the goal is to schedule the pre-meeting notification for the upcoming calendar event for the member. This is achieved by notifying our push generation system called Opportunist about changes in the member's calendar via a messaging system called Kafka. Kafka acts as a message broker to decouple processing by Opportunist from the data producer which is the calendar store. We made a conscious design choice to avoid strong coupling between the systems involved and you will see this pattern of managing our workflows again in this post.
Scheduling notifications
Based on the calendar change notification from Kafka, Opportunist wakes up and queries the calendar service for the next calendar event based on the newly updated calendar events and schedules a pre-meeting job on the scheduler approximately 15 minutes before the meeting. This ensures that at any given time, all members have a pre-meeting job in the scheduler for the latest upcoming meeting on their respective calendars.
| PRE_MEETING | JOHN SMITH (11111111) | 2014-06-08 11:45:00 |
As an example, this is a pre-meeting job scheduled for John Smith for a meeting at noon. It is important to note that we have only one entry per job per member only for the upcoming occurrence. This design choice enables us to scale the scheduler and not worry about all the possible events for a member at the same time. The scheduling of the next occurrence is handled by Opportunist when it processes this job. This is described next.
Notification generation
Fifteen minutes before the meeting, the scheduler supplies the job ID and the member ID to Opportunist which queries the calendar service for the attendees of the event that is going to start for the member. It then queries the Synthesizer, which generates and stores relationship opportunities, for interesting insights about the attendees of the event. These are returned by the Synthesizer in a sorted order ranked by how relevant they are to the member.
Opportunist picks the first item in the ranked list as the most relevant opportunity and constructs a Mobile Notification Event and outputs it to Kafka. The Mobile Notification Event contains a dictionary containing the details of the push notification like the type of the relationship opportunity, the target member, badge count, sound etc. Once again, we simply publish this message to Kafka so that it can be consumed asynchronously by the common push notification workflow to send out the actual push.
Opportunist then responds with a failure or success. If it's a failure, the job is retried. If everything is successful, Opportunist queries for the next event in the member's calendar and schedules the next occurrence of the job as part of its response. This ensures that the next pre-meeting push is taken care of.
The Mobile Notification Event is consumed by a service called the Mobile Notifications Consumer which formats the incoming event to the final localized alert text. The payload is then sent to the push notification service which sends it to the appropriate devices owned by the member.
Conclusion
Connected helps you stay in touch with the most important contacts in your network and makes it really easy to nurture your relationships with timely notifications about opportunities to connect. This was a significant engineering undertaking.
- We introduced a card based UI as bite-size presentations of concrete relationship opportunities where the member can take a quick action as a natural evolution of the newsfeed.
- We described the workflow that can keep up with the dynamic nature of a member's calendar.
- We highlighted how we are able to scale the scheduling of notifications by designing the system to only keep itself up to date for the upcoming calendar event.
- Finally, we delved into the details of how we achieved decoupling of the generation and ranking of relationship opportunities, scheduling based on the member's calendar and delivery of timely notifications by Opportunist through asynchronous Kafka messaging.
We are excited about additional context signals that we can process to generate intelligent notifications on the foundation we have setup for this app. Try out the brand new LinkedIn Connected app for iOS and let us know what you think.
Acknowledgements
Thanks to Greg Spurrier and Veena Basavaraj for a lot of ideas for this post.