Predicting social media users’ indirect aggression through pre-trained models

Participants and data collection

The steps in this study are illustrated in Fig. 1. This study was reviewed and approved by the Human Research Ethics Committee of the School of Economics and Management at Beihang University. The study used the online surveys for data collection. Students from the University of Beijing in China were invited to participate in this survey. Informed consent was obtained from all participants. The consent process was conducted in written form, ensuring participants were fully informed about the study’s purpose, procedures, and their rights. A total of 456 valid questionnaires were collected. All participants provided their Weibo IDs, enabling us to collect their Weibo activities and profile data. To guarantee the quality of the user data, we only select active users with more than 20 Weibo tweets. The selected users must have posted since 2020 and must have been obviously active for at least 2 months on Weibo. Three hundred and twenty active accounts are selected for this research on indirect aggression in social media, including 74 males and 246 females.

We employ the Indirect Aggression Scale Aggressor model (Mladenović, Ošmjanski & Stanković, 2021; Forrest, Eatough & Shevlin, 2005) to measure participants’ indirect aggressive behaviors, including three types of social exclusion (10 items), malicious humour (nine items), and guilt induction (six items) strategies. Specifically,

• social exclusion refers to behaviors that work by socially excluding the victim, such as withholding information, leaving out of activities, and turning the community against someone;

• malicious humour refers to behaviors in which humour is used to harm the victim, such as the use of sarcasm as an insult, intentional embarrassment, and practical joke playing;

• guilt induction refers to behaviors whereby guilt is intentionally induced, such as the use of emotional blackmail, undue pressure, and coercion.

Participants are first told to “Please answer the questions below according to how you interact with your contacts on Weibo.” The answers are measured using 7-point Likert scales, from “1 = Never behave this way” to “7 = Always behave this way”.

Three subscales of Alpha coefficients range from 0.76 to 0.92. The social exclusion ( $\mu =2.44$, $\sigma =1.11$), malicious humour ( $\mu =1.69$, $\sigma =0.77$), and guilt induction ( $\mu =2.03$, $\sigma =1.02$) are obtained by computing the average of each relevant item ( $\mu$ as Mean, $\sigma$ as SD).

Subjects categoring

Inspired by previous studies (Zhou, Xu & Zhao, 2018; Yu & Zhou, 2022), it is reasonable to divide the subjects into three categories (high, neutral, and low) according to the psychological prediction goals. The distribution of scores unexpectedly follows a long-tail distribution instead of a Gaussian distribution (Fig. 2). This implies that most individuals have low scores and few have high scores. The clustering algorithm K-means is applied to classify the individuals’ scores into three categories. The subject is grouped into the closest category according to the distance from the cluster centroid. Table 1 shows the cluster centroid and the number of users in each category. By classifying 320 active users into three categories, we obtain a training set used next to analyze and establish the prediction models.

Users’ features

Using the three prediction targets, namely social exclusion, malicious humour, and guilt induction, we extract and analyze three groups of features in order to determine a user’s characteristics. These features serve as inputs for the prediction model. The first and second group of features, respectively, contain a user’s basic and dynamic features. The third group primarily depicts the text in a user’s tweets. We calculate the three groups of features below.

Basic features $X_{b1}\sim X_{b13}$ reflect the user’s demographics and static numeric information on social media. The demographics $X_{b1}\sim X_{b4}$ include gender (male or female), educational background, occupation and hobbies. Additionally, the static numeric information includes $X_{b5}\sim X_{b13}$:

• $log(DAYS+1)$, where DAY S represents the number of days from the user’s registration to the survey;

• $log(NT+1)$, where NT represents the total number of the user’ tweets;

• $log(NT/(DAYS+1))$;

• $log(NF+1)$, where NF represents the number of the user’s followers;

• $log(NFE+1)$, where NFE represents the number of the user’s followees;

• $log((NFE+1)/(NF+1))$;

• $NT/(NF+1)$;

• $NT/(NFE+1)$;

• the length of the user’s self-description text.

Dynamic features $X_{d1}\sim X_{d125}$ are designed to reflect the intricate patterns of social interactions on Weibo both daily and weekly. Social interaction refers to posting, mentioning, and retweeting, which are key behaviors related to psychological traits in previous research. First, we calculate hourly and daily feature vectors based on the occurrences of interactions at each hour of the day and each day of the week. Here, hourly dynamic feature vector contains 24-dimensional hour-level feature and daily dynamic feature vector contains 7-dimensional day-level feature. These vectors are integral components of the dynamic features.

Second, we extract statistical measures from the hour-level and day-level features. For example, the following five features of “posting behavior” are extracted from the hour-level feature vector:

• the average hourly posting counts;

• the maximum of hourly posting counts;

• the hour with the most posts;

• the hour with the least posts;

• the variance of posting counts at different hours.

The proportions of tweets with mentions and retweets are also considered as features that reflect a user’s intensity of interaction.

User-generated content provides rich information about a user’s behaviors, which can also reflect their psychological traits. Previous studies on personality traits show that a user’s language styles on social media can effectively predict psychological traits (De Choudhury et al., 2013; Panicheva et al., 2022). In our study, we first utilize 300-dimensional pre-trained word embeddings $X_{w1}\sim X_{w300}$ to represent each word in Weibo. The word embeddings are taken from the pre-trained Chinese Word Vectors using Positive Pointwise Mutual Information (PPMI) (Li et al., 2018). We average the word embeddings to create the content vector at the Weibo level and subsequently average the Weibo-level vectors to obtain 300-dimensional content vectors at the user level. It should be noted that the text in a user’s descriptions is also considered part of the content features.

The extraction results in a 13-dimensional basic feature ( $X_{b1}\sim X_{b13}$), 125-dimensional dynamic feature ( $X_{d1}\sim X_{d125}$), and 300-dimensional content feature ( $X_{w1}\sim X_{w300}$). We extract 512-dimensional content vector representations ( $X_{p1}\sim X_{p512}$) from the text content using pre-trained models, which are detailed in the Models section.

Correlation analyses between features and indirect aggression

Based on the aforementioned user’s features, we first compute and examine the Pearson correlation coefficients between features of different categories. Applying a coefficient threshold of 0.1, we find that an individual’s scores of indirect aggression from the three categories is related with both the basic and dynamic features ( $p$-value $<0.05$).

Gender, as a demographic characteristic, emerges as one of the most significant features of malicious humour (Coef. = 0.240) and guilt induction (Coef. = 0.147). In terms of basic interactions, social exclusion shows a positive correlation with the number of followers (Coef. = 0.125), suggesting that users experiencing high social exclusion have more followers. When looking at dynamic features, social exclusion and malicious humour are positively correlated with social media activities (Coef. = 0.958 and Coef. = 0.119, respectively), such as tweeting frequency at different times. We observe that the three categories significantly and positively correlate with individual mentions and retweeting activity.

Textual content analyses

From the perspective of the text content, key words in three subcategories are computed and analyzed. We choose the significant word results from the word frequency in the high group and the low group (shown in Table 1) using analysis of variance. The word cloud in Fig. 3 depicts the key words and the size of the words represent the significance (1/ $p$-value). It also shows the different emotional levels and aggressive behaviors.

Instances of social exclusion are direct and aggressive, requiring lower levels of social manipulation. The words such as “accusation” (), “threaten” (), “rape” (), and “victim” () are used to express exclusion. The expression of malicious humour is rational-appearing aggression with the use of euphemistic and gentle words and thus requires a higher-level social manipulation. The names of tarnished celebrities, such as “Yifan Wu” (), “Yaxuan Song” (), and “Yitong Li” () are used to express maliciousness due to their controversial behaviors or evil doings. Finally, guilt inducting expressions apply psychological pressure, prompting individuals to reflect on their emotions and behaviors. This level of social manipulation falls between that of social exclusion and malicious humour. The words like “apologize” (), “pressure” (), and “connotation” () express guilt.

Models

Recently, we have witnessed the fast growth of large-scale pre-trained models in the natural language processing field. They are often pre-trained with an extremely large amount of data and have demonstrated good performance across various prediction tasks (Vaswani et al., 2017; Han, Zhang & Ding, 2021; Min et al., 2023). The Transformer architecture has the potential to represent text content, which can substantially improve the research on psychological prediction. Bidirectional Encoder Representations from Transformers (BERT) (Devlin et al., 2018), as a pre-trained model, was used to generate the text represented for each social media user. We first utilized the pre-trained BERT based on the Chinese corpus to predict the users’ indirect aggression in our study.

However, the existing pre-trained language models rarely integrate knowledge graphs, which offer rich structured knowledge and entities to enhance understanding of natural language. Consequently, we use ERNIE, a sophisticated modification of the BERT architecture, to incorporate linguistic and world knowledge (Sun et al., 2021). Figure 4 shows the architecture of ERNIE. The current version, ERNIE 3.0, introduces a framework for large-scale pre-trained knowledge-enhanced models with 175 billion parameters. Empirical results demonstrate that the ERNIE 3.0 model outperforms the state-of-the-art models on multiple Chinese NLP tasks.

As shown in Fig. 4 (right), the knowledgeable encoder (K-Encoder) is a key module in this model. It is designed to effectively integrate heterogeneous information from the content of the text and from the knowledge graphs. The K-Encoder integrates additional token-oriented knowledge into content features at the attention layer. It consists of stacked aggregators where the input word embeddings $\{w_1^i,...,w_n^i\}$ and entity embeddings $\{e_1^i,...,e_n^i\}$ from the previous aggregator are fed into multi-head self-attentions at the $i$-th aggregator. The operation of $i$-th aggregator operation is denoted as follows:

(1) $$\{w_1^{i+1},...,w_n^{i+1}\},\{e_1^{i+1},...,e_n^{i+1}\}=\mathrm{A}\mathrm{g}\mathrm{g}\mathrm{r}\mathrm{e}\mathrm{g}\mathrm{a}\mathrm{t}\mathrm{o}\mathrm{r}(\{w_1^i,...,w_n^i\},\{e_1^i,...,e_n^i\}).$$

Specifically, the $i$-th aggregator adopts an information fusion layer for combining the token and entity sequence, and computes the output embedding for each token and entity. For a token $w_j$ and its aligned entity $e_k$, the information fusion process is as follows,

(2) $$\begin{array}{rl}{h}_j& =\sigma ({\tilde{W}}_t^i{\tilde{w}}_j^i+{\tilde{W}}_e^i{\tilde{e}}_k^i+{\tilde{b}}^i),\\ w_j^i& =\sigma (W_t^i{h}_j+b_t^i),\\ e_k^i& =\sigma (W_e^i{h}_j+b_e^i)\end{array}$$

Here, $h_j$ is the hidden state that integrates the information from both the token and the entity. ${\tilde{w}}_j^i$ and ${\tilde{e}}_k^i$ are output from multi-head self-attentions with input of $w_j^i$ and $e_k^i$.

The output computed by the last aggregator serves as the final output embeddings of the K-Encoder. This allows us to input the heterogeneous information of words and entities from user-generated content into the content vectors. The information fusion can enhance the prediction of users’ indirect aggression.

Moreover, an increasing number of promising pre-trained models based on Transformer architecture have emerged including BART (Lewis et al., 2019), RoBERTa (Liu et al., 2019), and ELECTRA (Clark et al., 2020). Researchers evaluate the design decisions of these pre-trained models. The findings indicate that performance can be improved by training the model longer, with bigger batches, and by dynamically changing the masking patter applied to the training data. This leads to the proposal of RoBERTa (Liu et al., 2019). In addition, ELECTRA outperforms MLM-based models (masked language modeling) such as BERT and RoBERTa when considering the same model size and data. This is attributed to its compute-efficient and parameter-efficient ability to distinguish real data from challenging negative samples (Clark et al., 2020). We use the competitive models RoBERTa and ELECTRA for comparison with BERT and ERNIE in this study.

In this study, we compute the average word embeddings from large pre-trained models that generate 512-dimensional content embeddings to represent the users’ behavior. In the output layer, we incorporate both Basic + Dynamic features in order to fine-tune the model. We aim to develop the ultimate prediction model. The experiments are performed with PyTorch (v1.12) and Transformers (v4.7.0) on a Linux server equipped with two GPUs (NVIDIA V100) and two CPUs (Intel Xeon Silver 4216). The anonymous dataset and code have been released publicly. The pre-trained language models in this study are obtained from HuggingFace.

Experimental design

In this section, we employ various models to predict and assess the predictive power regarding three specific forms of indirect aggression among social media users: social exclusion, malicious humour, and guilt induction.

The experiment is structured in two phases: the first involves pre-trained and benchmark models; the second evaluates advanced pre-trained models. Initially, we compare benchmark models such as LR, SVM, and MLP with pre-trained models like BERT and ERNIE, demonstrating that the latter significantly outperform traditional machine learning approaches. This phase evaluates the predictive power of user-generated content using two feature sets: Basic + Dynamic and Basic + Dynamic + Content.

Subsequently, we include additional advanced pre-trained models, RoBERTa and ELECTRA, to identify the most effective model for predicting users’ indirect aggression. This comprehensive analysis not only underscores the superior performance of advanced models but also enhances our understanding of their applicability in real-world scenarios. Performance evaluation for each model is rigorously conducted using the dataset split into 70% training and 30% testing segments, facilitating robust validation.

Evaluation metrics

The prediction performance of the aforementioned models is verified by Accuracy (ACC), F1-score ( $F1$), and AUC (Area Under the Receiver Operating Characteristic Curve). We build 3-category classifiers for users’ indirect aggression and then obtain true positive (TP), true negative (TN), false positive (FP), and false negative (FN) for the category $i=-1,0,1$. $N_{samples}$ is the total number of samples. The expressions for ACC and $F1$ are below.

(3) $$ACC=\frac{{\sum }_{i=-1,0,1}T{P}_i}{N_{samples}}.$$

Calculations for the precision and recall for each label $i$ are below:

(4) $$Precisio{n}_i=\frac{T{P}_i}{T{P}_i+F{P}_i},$$

(5) $$Recal{l}_i=\frac{T{P}_i}{T{P}_i+F{N}_i},$$

(6) $$F1=\frac{1}{3}\sum _{i=-1,0,1}\frac{2(P{r}_i\times R{e}_i)}{P{r}_i+R{e}_i}.$$

The AUC of each category can be measured against the rest of the categories, namely the one-vs-rest AUC (Li et al., 2020). $F1$ and AUC serve as proxies for model validation even though the testing dataset is not unbalanced.

Results of pre-trained models and benchmark models

Table 2 details the performance of various models in predicting users’ indirect aggression within the three categories: social exclusion, malicious humour, and guilt induction. The models include LR, SVM, and MLP as the benchmarks, and BERT and ERNIE represent the pre-trained models. The models are evaluated based on two groups of features: Basic + Dynamic and Basic + Dynamic + Content.

The BERT and ERNIE models’ performance with Basic + Dynamic + Content features is outstanding in predicting social exclusion. They achieve $ACC=57.50\mathrm{\%}$, $F1=50.99\mathrm{\%}$, $AUC=67.57\mathrm{\%}$ and $ACC=63.75\mathrm{\%}$, $F1=48.48\mathrm{\%}$, $AUC=61.88\mathrm{\%}$ respectively. These two pre-trained models, especially ERNIE, significantly outperform MLP and other benchmark models.

While using only Basic + Dynamic features, the SVM model leads with an accuracy of 54.69% and AUC of 53.18% for predicting malicious humour. With the addition of content features, there is an improvement in the LR’s performance, which shows an accuracy of 50.31%. MLP achieves the highest $F1$ score at 45.16%. The pre-trained models surpass these results, with ERNIE leading with an ACC of 65.00%, an $F1$ of 55.63% and an AUC of 64.43%.

The pre-trained models demonstrate superior performance in predicting guilt induction, with ERNIE achieving the highest accuracy of 60.31%, an $F1$ score of 54.73%, and an AUC of 67.71%. The Basic + Dynamic features yield the highest accuracy with LR at 46.88% and the highest AUC score of the SVM at 58.25%. When incorporating content features, the LR’s accuracy improves to 48.44%, but the $F1$ scores drop across all models.

Figure 5 shows that BERT and ERNIE with Basic + Dynamic + Content features have the most noticeable performance improvement compared with the other benchmarks, including prediction in the three categories of social exclusion, malicious humour, and guilt induction. We obtain these results consistently. Pre-trained models perform consistently highest in our study.

We find ERNIE is significantly better than BERT because of its richly structured knowledge. ERNIE substantially raises the prediction performance, improving 15.31%, 13.31%, and 11.87% of the ACC compared with the best baseline model. The results of the $F1$ score and the AUC support this conclusion as well. ERNIE consistently shows the best performance across all metrics and categories, indicating its effectiveness in understanding and predicting indirect aggression online.

When adding the content features to the traditional machine learning models, there is limited improvement, as shown by the blue and orange lines in Fig. 5. The ACC, $F1$ score, and the AUC of the MLP with Basic + Dynamic + Content features (45.63%, 42.39%, 60.06%) are slightly better (+1.25%, +2.77%, and +1.58%) than those of the MLP with Basic + Dynamic features. This demonstrates that the text content cannot be adequately mined by the simple and traditional word embeddings normally used for predicting psychological traits.

Additionally, we evaluate the time efficiency of the prediction models. Our optimal model leverages the pre-trained ERNIE model, significantly reducing the need for extensive training from scratch. This combination of pre-trained models and optimized feature extraction enhances both prediction accuracy and efficiency. Specifically, ERNIE completes training in 68.12 s and achieves an inference rate of 8.43 users per second. In comparison, the MLP model completes training in 35.33 s with an inference rate of 6.34 users per second. This demonstrates that in our study, ERNIE requires only slightly more training time compared to the traditional machine learning solution. Considering application scenarios of psychology, this time efficiency is acceptable.

Results of advanced pre-trained models

We argue that leveraging advanced pre-trained models can increase the potential of automating textual coding and potentially improve the ability to predict indirect aggression. The fact that the model is trained on a small batch of samples, a concept commonly referred to as few-shot learning, should not be overlooked. In addition to the BERT and ERNIE models, we construct and evaluate two pre-trained models: RoBERTa and ELECTRA.

Figure 6 presents a clear comparison of the accuracy of the four models across the three prediction targets. ERNIE demonstrates the highest accuracy in predicting users’ social exclusion and malicious humour behaviors, achieving an accuracy of 63.75% and 65.00%. Although ELECTRA achieves a slightly higher accuracy rate (62.19%) than ERNIE (the second-best 60.31%) for predicting guilt induction, ERNIE achieves the highest $F1$ score (54.73%) and AUC (67.81%).

Table 3 shows the comprehensive performance of four pre-trained models (BERT, ERNIE, RoBERTa, and ELECTRA) across three categories of indirect aggression online. ERNIE consistently demonstrates superior performance scores across all categories of indirect aggression online. Informative entities, such as entertainment stars and controversial figures, are often the targets of indirect aggression. ERNIE’s knowledge graphs contain rich information about entities and the relationships between them. The results imply that knowledge graphs play a pivotal role in predicting indirect aggression, thereby contributing to the enhancement of the model’s performance.

Theoretical contribution

This study makes a notable theoretical contribution by expanding the understanding of indirect aggression. By going beyond the conventional reliance on self-reporting methodologies, we addressed the need for innovative approaches to predict indirect aggression using pre-trained models. This pioneering step opens up new possibilities for examining and comprehending the complexities of indirect aggressive behaviors in online social media contexts. By harnessing the power of computational techniques, we provide a fresh perspective on indirect aggression, shedding light on how it manifests itself and the nuances involved that were previously challenging to discern. Our research offers clear guidelines for using the pre-trained model to predict indirect aggression. Incorporating basic, dynamic, and content features, we provide insights into the roles these features play in predicting different types of indirect aggression. These guidelines not only contribute to the specific task of predicting indirect aggression but also have a broader application in the field of natural language processing and behavior analysis. Researchers and practitioners can benefit from this structured approach to harnessing ERNIE’s capabilities in various predictive tasks beyond aggression.

A significant theoretical finding of our study is the outperformance of pre-trained models compared to traditional machine learning models that lack external pre-trained information. Demonstrating the superiority of pre-trained models like ERNIE in predicting social media users’ behaviors provides compelling evidence of the effectiveness of using these pre-trained techniques to capture the intricate patterns and subtleties found within social media interactions. We present three reasons why ERNIE is suitable for our prediction research. First, ERNIE often surpasses BERT in few-shot learning and fine-tuning with limited data (Sun et al., 2019; Zhang et al., 2019). Second, informative entities, such as entertainment stars and controversial figures, are often targets of indirect aggression. ERNIE’s knowledge graphs contain rich information about entities and their relationships, improving predictions of users’ indirect aggression. Third, as shown in Fig. 4, the K-Encoder integrates token-oriented knowledge into textual information from the underlying layer, representing heterogeneous information of tokens and entities in a unified feature space. This integration can enhance the prediction of indirect aggression. This result extends the theoretical boundaries of prediction models to other psychological contexts, particularly when dealing with limited data samples, and highlights the potential of pre-trained models to advance advancing behavioral research in the digital age.

Additionally, the novelty of our dataset lies in its extensive collection and annotation of indirect aggression instances on social media platforms, which is a relatively unexplored area. The innovative aspects of our method include the integration of multiple feature types and the application of a sophisticated pre-trained model, which together significantly enhance prediction accuracy and reliability. These contributions not only fill a gap in the current literature but also set a foundation for future research to build upon.

Practical implications

From a pragmatic standpoint, our research offers a workable system for predicting indirect aggression online (shown in Fig. 7). By moving beyond the limitations of conventional self-reporting questionnaires, our pre-trained model, based on ERNIE 3.0, provides a more objective and data-driven means to detect and understand indirect aggressive behaviors online.

Social media platforms can leverage our findings and methodology to implement proactive measures for identifying and addressing instances of indirect aggression. This can enhance their capacity to promote a safe and respectful online environment. Additionally, this study extends to social media platforms’ organizational strategies. By monitoring and recognizing users’ indirect aggression through our predictive model, platforms can intervene early to mitigate potential harmful behaviors. This proactive approach empowers platforms to take appropriate actions, such as providing supportive interventions, issuing warnings, or offering educational resources to users engaging in indirect aggression.

As a result, fostering a more inclusive and respectful user experience becomes feasible, with the potential to reduce instances of cyberbullying and negative interactions. In addition, the optimization of platforms’ organizational strategies, based on our predictive model, can lead to a more positive user experience and improved well-being for their user base. By curbing instances of indirect aggression, social media platforms can create an environment that promotes healthy communication, encourages constructive interactions, and reduces the likelihood of users being exposed to harmful content. Enhancing the overall well-being of their users aligns with the platform’s social responsibility and contributes to building a sustainable and loyal user community.

Limitation and future work

This study has certain inevitable limitations. First, we must admit that we cannot employ the current state-of-the-art learning methods which are proposed every a few weeks, such as GPT, Llama (Touvron et al., 2023) and Gemma (Banks & Warkentin, 2024). Therefore, the novel large pre-trained models are encouraged to be investigated in the field of cyber psychological and behavioral studies. Second, although deep learning model is employed for human behavior prediction, users’ textual content is the main source to encode behavior. Next, the social media users’ integrated behavior, namely basic characters, dynamic behavior and tweet content, can be a unified proxy to be feed into the behavioral encoder. This end-to-end behavior encoder will build a bridge between behavior prediction and deep learning. Third, this study uses specialized datasets specifically tailored to capture indirect aggression behaviors, which may not encompass the diversity found in standard datasets focused on direct aggression. Despite this, our methodology and findings remain valid and significant, as they provide novel insights into the less studied domain of indirect aggression.