Leveraging behavior analytic computation for anti-abuse defenses

LinkedIn is committed to providing a trusted environment to keep our nearly 740 million members safe. Part of that commitment involves protecting against various types of abuse on the platform. Fighting abuse presents many challenges; one example is when bad actors use bots for large-scale attacks, while another is when attack signals constantly evolve to adapt to existing defenses. In these scenarios, the adaptability of the adversary can limit the effectiveness of our existing anti-abuse models over time, which is why it's important to have both high-quality features and a faster response time to enhance our defense resilience.

Behavior analytic computation is a newer concept that utilizes simple arithmetic and logarithmic operations on one or a few existing raw features to capture the key anomalous patterns from bot-driven attacks. It can extract more robust features with well-preserved automation behavior, which improves defense resilience against fast-changing attack signals. Due to the lightweight computation, the extracted signatures can be used directly as the label to defend against large-scale attacks and provide fast defense turnaround. At LinkedIn, behavior analytic computation has not only been used to create various robust features, enhancing machine learning models to detect bots, but has also brought significant improvement in response time and defense coverage on sudden and new abuse attacks.

In this blog post, we will first review the challenges and limitations of applying machine learning in the anti-abuse domain. Then, we will introduce the concept of behavior analytic computation and show how it can mitigate the gaps to improve overall anti-abuse defense.

Label and feature limitations

Machine learning has been an important technique to tackle various types of large-scale internet abuse at LinkedIn. Supervised machine learning approaches are commonly used, as abuse behaviors unperceived by humans can be learned from the given labelled data. However, because abuse is determined by both intention and behavior, the ground truth is difficult to obtain in many abuse scenarios, and normally only partial abuse data can be labeled, which could lead to biased or underperforming models.

As the labeled abuse data is limited and more unsupervised approaches are explored, the feature quality becomes the key for machine learning models to succeed. The features used by models normally belong within two categories: 1) value features and 2) counting features. Value features are the most common category; they directly use the value of raw features, such as IP address, name, email, etc. Some abuse patterns can be found by finding identical or similar feature values among massive requests/users (e.g., clustering); some abuse patterns can be found by identifying different feature values from the same entity (e.g., account takeover). Counting features are based on the feature counting numbers within a certain time period, which are commonly used for large-scale abuse prevention. Various machine learning approaches (supervised/unsupervised) basically try to find the complex relationship between these features.

When abusers start to change their signals once anti-abuse defenses are applied, we often see both value features and counting features are subject to be compromised. Rules and models based on value features often need to adjust parameters or retrain if feature values are changed/spoofed. Counting features will lose the visibility to differentiate bot and human traffic once abusers lower their request rate. The fact that features are constantly altered or changed can significantly degrade the performance of rules and models. 

As models are only able to learn from the information captured by the features, any information missed by the features will be overlooked by the models as well. Higher quality features to capture more insights and provide higher resistance to attack variability are essential for successful anti-abuse defense. There are other technologies that could serve the purpose, but they often require collecting additional user information.

Behavior analytic computation

Behavior analytic computation, on the other hand, can provide high quality features for bot-driven attacks without collecting extra user data. The idea of behavior analytic computation is to capture the key bot/automation signatures hidden among the existing feature values or numbers—these signatures can be preserved even if the features are constantly changed. It intelligently utilizes simple arithmetic and logarithmic operations to transform existing features into more robust ones in constant time. Two key operations of behavior analytic computation are ratio and logarithm. Ratio can capture the relative change of numbers instead of absolute numbers; logarithm can amplify particular ratio values that differentiate bot/automation traffic and human traffic patterns.

Here are some examples to demonstrate the concept of behavior analytic computation:

Fig. 1. Request distribution of top values of Y feature when a certain value of X feature is selected  

When two features are independent of each other, statistically we will see some kinds of natural distributions between them. Assuming X and Y are two feature types, x_i and y_i are values of the feature, respectively. When a particular value x₁ is selected and the request numbers from top values of Y within a certain time period are sorted descendingly, the distribution of different y_i count can be obtained. Fig. 1(a) shows an exponential decay distribution of top ten counts of Y, which normally indicates the selected x₁ is a legitimate one used by humans, as it is independent to the values of Y used. On the other hand, if we see the distribution like Fig. 1(b) where one Y value, y₁, dominates the majority of the traffic, it indicates x₁ and y₁ are highly correlated and it is possibly from bots. Fig. 1(c) shows another distribution where several y_i counts are abnormally similar, which is an evidence of potential bots uniformly iterating over requests with multiple Y values.

Behavior analytic computation captures such a distribution signature into a single score by applying several arithmetic and logarithmic operations on those y_i counts. More natural distributions (Fig. 1(a)) will have lower scores, and abnormal ones (Figs. 1(b) and (c)) will have higher scores. When bad actors change the X or Y values/numbers, the automation behavior presented by the distributions can still be captured by the extracted distribution scores. The lightweight computation on any arbitrary time period makes such distribution signature scores powerful as both features and labels. At LinkedIn, these types of extracted distribution signatures have improved our defense response time for large-scale bot-driven attacks by more than an order of magnitude.

Fig. 2. Request temporal plots of two different users

Another example where behavior analytic computation can contribute is by improving the quality of counters. Counters are commonly used against large-scale abuse, as higher counter values typically reflect the likelihood of automation-driven abuse. However, certain behavioral and structural information that is not captured by counters will also be missed by models if only typical counters are used as the features. Fig. 2 shows two examples of the requests from a single user within one hour where the x-axis is time and y-axis is the request index. The counter values for both cases are identical, but different behaviors are shown. While there is no obvious automation evidence found on Fig. 2(a), the requests on Fig. 2(b) contain certain regular patterns indicating automation behind the scenes.  

Behavior analytic computation again captures such a plot signature into a single automation score where more human-like plots (Fig. 2(a)) have lower scores and automation-like plots (Fig. 2(b)) have higher scores. The extracted score is able to capture the key automation pattern even if users constantly adjust different regularities within the same time period. The same lightweight computation continues to provide speed and flexibility to the defense. More importantly, the extracted automation score is able to fill the gap where typical counters fail to differentiate between human requests or bot-driven automation requests at low counting numbers. Due to its capability to detect low-frequency automation, this extracted automation score has almost doubled our machine learning model recall on detecting automation-driven abuse at LinkedIn.

Overall, we are able to leverage behavior analytic computation for our anti-abuse defense from three different aspects:

Robust features
Behavior analytic computation can capture automation signatures as robust features, thus improving model and rule resilience against fast-changing attacks. When defending against an actual dynamic attack, the extracted feature can bring significant robustness improvements - greatly extending the time of protection by more than two orders of magnitude.

Label expansion
The other key merit of behavior analytic computation is the ability to expand abuse labels. The typical anti-abuse defense is based on supervised models. However, for many bot-driven abuse types, such as scraping or account takeover, often only a small portion of all abusive requests/users are labeled, which limits the effectiveness of the models. Behavior analytic computation provides an unsupervised approach where each extracted automation signature can perform as an automation detection classifier. Those signatures have helped expand the overall recall when defending against bot-driven large-scale attacks in many abuse verticals at LinkedIn.

Prompt response
As bad actors can persistently adapt to penetrate existing defenses and additional applications on the platform can provide entry points for new abuse scenarios, anti-abuse defense inevitably needs to continually handle new abuse scenarios and behaviors. Quick turnaround to the new abuse activities becomes critical to properly contain the overall damage. Unlike other robust features, such as browser or device fingerprinting, which often require longer engineering time and higher cost to onboard, behavior analytic computation can provide a prompt response to evolving attack signals to mitigate the threat. When facing new abuse scenarios where early assessment of existing features is critical to define anti-abuse strategy, behavior analytic computation’s lightweight extraction can also offer fast offline analysis and quick preliminary defense onboarding.

Conclusions

Bad actors will always continue to change and evolve their behaviors, so resilience of the defense and fast turnaround are essential to ensure long lasting anti-abuse success. Behavior analytic computation is a newer concept that offers a lightweight and quick defense onboarding based on existing features, while preserving member privacy. The extracted automation signatures can not only enrich the feature set, enhancing defense robustness, but also provide expanded abuse labels to significantly improve incident response time and coverage for large-scale bot-driven attacks.

Acknowledgements

Fighting abuse at LinkedIn is an ongoing effort requiring extensive collaboration between multiple teams. I would like to acknowledge our partners from the following teams, without whom this would not have been possible: Abuse Research and Response, Abuse Prevention Infrastructure, Trust AI, Trust Data Science, Trust and Safety, Legal, and Information Security.