Viral spam content detection at LinkedIn

On the LinkedIn platform, members from around the world share their knowledge, perspectives, and discuss topics important to them. Our goal at LinkedIn is to enable them to do so in a safe, trusted, and professional environment. 

We’ve previously discussed the various systems used to create a safe and trusted experience for our members and how we keep the LinkedIn Feed relevant for our members on LinkedIn. To provide a positive member experience, when content is posted or shared that goes against our LinkedIn Professional Community Policies, or when content is reported as violating those policies, we take action to remove the content. There are rare occasions when content uploaded on our platform goes undetected by our current defense mechanisms, which may result in sharing across the platform. Although such incidents occur infrequently they can have detrimental effects when they do. 

That’s why developing models for viral spam content detection is critically important. It’s extremely complex, especially when experiencing issues with data scarcity of viral content. To understand how viral spam content flows through member networks and is detected before it goes viral, it is necessary to stay updated with the latest forms of spam content being uploaded on the platform and modify our models accordingly. By understanding virality and spam content, we’ve been able to perform comprehensive analyses of features and signals that have been helpful in detection of this type of content, which has enabled us to address these challenges.

In this blog, we’ll discuss the steps we’re taking to create a trusted and safe environment by implementing various spam and policy-violating content detection measures. 

Understanding content virality

LinkedIn is not designed for virality but on occasion posts that result in significant engagement in the form of likes, reactions, comments, and reshares in a short period of time could be considered viral. 

Strategies to identify viral spam content

Viral spam detection is a larger effort where we have launched multiple AI models, based on the violative content and spam type present on the platform.   

The AI models used to detect violative content can be broadly classified into two main categories: 1) proactive defenses and 2) reactive defenses. In the proactive model, the detection happens as soon as the content surfaces on the LinkedIn feed, whereas in the reactive model, we monitor the activities happening around the posted content and then try to predict its potential to be shared widely.

Illustration of proactive and reactive defenses and their operating points

Figure 1: Illustration of proactive and reactive defenses and their operating points

Proactive defenses

On the platform, proactive defenses act to predict potential viral spam content as early as possible. These defenses consist of two types of classifiers. The first set of classifiers are trained on a specific spam category, such as hate speech; the second set of classifiers are trained on a particular content type, like videos or articles. Both types of classifiers have been trained on various features that provide an early signal for both viral and spam content, which will be discussed in a future section. These defenses are mostly deep neural networks that are trained using Tensorflow and are deployed using the centralized ML platform at LinkedIn, Pro-ML. These models are run on the platform every few hours and take appropriate action, such as filtering or sending content for manual human review.  

Reactive defenses

Reactive defenses serve as an additional layer of protection to the proactive defenses and act on the content after it has gathered engagement signals pointing to virality. The reactive classifiers prevent spam content from reaching large numbers of members and going extremely viral. 

Our reactive model is a combination of predictive machine learning model and heuristics, which makes use of member behavior, content features, and interaction patterns with content in order to predict the probability of viral content being spam. These defenses are trained on a Boosted Trees model and similar to our proactive defenses, are deployed using Pro-ML

Viral content detection pipeline 

The below diagram depicts the content’s detection journey on the platform after it has been posted. As soon as a piece of content surfaces, the existing ML classifiers act based on the immediate features that can be computed, such as author and content related features. If it is found to be spam or policy-violating, then we either take an automatic action or send it for human review to decide on the action to be taken. For the content that is still present on the platform, we monitor the engagement signals, temporal signals, and spam related signals to detect the potential for viral spam during the content lifecycle on the platform.

Diagram of Pipeline for viral content detection on LinkedIn’s platform

Figure 2: Pipeline for viral content detection on LinkedIn’s platform

Features to predict virality

From the time a piece of content is posted on the platform until the time it goes viral, there are a lot of factors that influence the engagement that the post receives. These factors include content type (text, image, video, article, etc.), members' interaction with the content, and how the content flows through the member network. We categorize these features into two main categories, post features and member features. 

Post features

Post features include content type, content polarity, and spamminess of the content. Looking at these factors to identify potential viral content allows us to view how content goes viral at different rates, observe a correlation between polarity scores and quality of the content, and utilize member reports as an indicator of spam content.

Graphic of Useful post features to predict viral spam content

Figure 3: Useful post features to predict viral spam content

Member features

Analyzing the features of members that have interacted with the post (shares, comments, or reactions) within a given timeframe provides an early detection signal for the virality of a given post. We derive following features from them:

  1. Network features: Features which quantify the influence and popularity of these members on feed as their action might expose the post to a lot more members creating a cascade effect which makes the post go viral. Here, we use features such as followers and connection counts, diversity in industry, location, and level of the network (connections and followers) of these members. 

  2. Activity features: These represent what kind of engagement their activities have gotten in the past, how active they are in the platform, and for how long they have been a LinkedIn member. 

Diagram of Useful member features to predict viral spam content

Figure 4: Useful member features to predict viral spam content

Engagement features

These features are based on the engagement a posts receives in the form of likes/reactions, shares, comments, and views. We derive various features from these such as temporal sequences of counts and velocity of likes/reactions, shares, comments, and views. These act as the strongest signal for the cascading effect happening in the network.

Graphic of Engagement actions on LinkedIn’s platform

Figure 5: Engagement actions on LinkedIn’s platform

Impact

Over time, all models have significantly contributed to reducing the number of unique viewers that encounter spam content. Through the implementation of both proactive and reactive models, we have successfully decreased the overall percentage of views on spam content by 7.3%. The proactive models have been especially effective in detecting spam content early, resulting in a reduction of spam views by 7.6%, whereas the reactive models have reduced it by 2.2%. As a secondary impact, we have observed a decrease in member and disinterest reports on onsite content. Moreover, our efforts have resulted in a decrease in views on policy-violating content by 12%.

By successfully reducing the number of viewers accessing spam and policy-violating content, we have taken a significant step towards creating a safer and more secure online environment. Our continued efforts towards improving content detection models will hopefully ensure a better user experience for all members on our platform.

Conclusion

Our goal at LinkedIn is to serve members with safe, productive, and relevant experiences across the Feed that will help them connect to opportunities and grow their professional presence. Viral spam detection is an evolving effort, and we are continuously working on improving the existing solutions to refine our coverage over content types and policies, while reducing model run time. Looking ahead, we’re working on a consolidated classifier, which will model viral spam content detection across different content types and policies. We’re excited to continue our work in this space and expand our capabilities to maintain LinkedIn’s reputation as a safe and trusted platform. 

Acknowledgements

We would like to thank all the people involved in the effort (alphabetically ordered):

Anirban Biswas, Jay Oza, Nithish Divakar, Shilpi Agrawal, Sumit Srivastava, Srinivasa Madhava, and Phaneendra Angara.