1 Introduction
In today's interconnected world, power systems are becoming progressively vulnerable to cyberattacks, which may lead to significant disruptions and threaten the stability of the grid
[1]. A particular type of cyberattack that poses a substantial challenge is the uncertain load redistribution attack
[2]. This attack involves manipulating the load distribution within the system, generating imbalances and potential outages. To address this pressing issue, it is essential to devise effective defense strategies that can mitigate the impact of these attacks and ensure the stability and resilience of the power grid.
In recent years, social media platforms have emerged as a rich source of real-time information, providing invaluable insights that can enhance situational awareness and inform decision-making processes during power outages. By analyzing social media data, particularly the emotional state of the affected population, one can garner crucial insights into the public's perception and reaction to ongoing events, ultimately enabling the development of more targeted and effective defense strategies
[3–5].
Concurrently, advances in natural language processing and machine learning techniques have facilitated the extraction of meaningful information from voluminous social media data. These techniques have been successfully applied to various domains, such as marketing, public health, and disaster management, to glean valuable insights and support decision-making processes. In the context of power systems, integrating social media data analysis with advanced optimization algorithms can significantly enhance the resilience of the grid against cyberattacks and natural disasters.
In this paper, we propose a comprehensive methodology that combines social media data analysis and advanced reinforcement learning techniques to develop defense strategies against uncertain load redistribution cyberattacks in power systems. Our methodology consists of two main stages. In the first stage, we collect and preprocess social media data to extract valuable information about power outages and the emotional state of the affected population. We employ two emotional analysis methods, VADER sentiment analysis and Support Vector Machine (SVM) classifier, to assess the public sentiment during power outages. In the second stage, we utilize the Proximal Policy Optimization (PPO) algorithm to model the defense strategy development and handle the outage uncertainty in an optimal power flow problem. The PPO algorithm is a state-of-the-art reinforcement learning method that offers improved stability and efficiency compared to other policy optimization techniques. By employing a clipped objective function and trust region optimization approach, the PPO algorithm can learn effective defense strategies for managing uncertain load redistribution cyberattacks in power systems while leveraging the valuable information gathered from social media platforms. We test our methodology on an IEEE 6-bus system to demonstrate its effectiveness in mitigating uncertain load redistribution cyberattacks while maintaining the stability of the power grid. The results show that our approach can effectively utilize social media data to inform defense strategies, ensuring a nuanced and targeted response to the unique challenges posed by uncertain load redistribution attacks.
The remainder of this paper is organized as follows. Section 2 reviews the related literature. Section 3 presents our methodology for collecting and preprocessing social media data, as well as the emotional analysis techniques employed to assess public sentiment during power outages. In Section 4, the sentiment analysis is given. In Section 5, we elaborate on the application of the PPO algorithm for defense strategy development and outage uncertainty management in the context of an optimal power flow problem. We then showcase the effectiveness of our approach through a case study on an IEEE 6-bus system in Section 6. Finally, in Section 7, we conclude the paper by summarizing our findings and highlighting the potential of our methodology to address real-world challenges in power systems and enhance grid resilience against cyberattacks.
By integrating social media data analysis and advanced reinforcement learning techniques, our proposed methodology offers a novel approach to developing defense strategies against uncertain load redistribution cyberattacks in power systems. Our case study on the IEEE 6-bus system underscores the potential of our methodology to address real-world challenges in power systems, demonstrating the value of social media data analysis in improving the resilience of power grids against cyberattacks and natural disasters. With the increasing threats to power system security and stability, we believe that our methodology offers a promising avenue for further research and practical applications in the field of power systems and cybersecurity.
2 Literature Review
In this section, we provide an extensive literature review encompassing various domains pertinent to our study, such as data collection and preprocessing from social media, emotional analysis techniques, and the utilization of reinforcement learning algorithms for mitigating uncertain load redistribution cyberattacks in power systems. This comprehensive review will identify research gaps and challenges in the existing body of work, thereby establishing the context and motivation for our proposed methodology.
2.1 Data Collection and Preprocessing in Social Media
The rapid proliferation of social media platforms as indispensable sources of real-time information has triggered an upsurge of research endeavours exploring their application across a multitude of domains, including power systems and disaster management
[6–8]. The sheer convenience and simplicity of sharing information and expressing emotions on these platforms have piqued the interest of researchers and practitioners alike, thereby resulting in a plethora of studies investigating the collection, processing, and analysis of social media data
[9, 10]. Within the context of power systems and disaster management, researchers have examined methodologies to harness social media data to obtain insights into public sentiment, information dissemination patterns, and situational awareness during power outages, natural disasters, and other critical events
[11, 12].
A prominent challenge in the analysis of social media data is the necessity to preprocess the data to eliminate extraneous noise and enhance its quality. Various preprocessing techniques, such as tokenization, stop word removal, stemming, and lemmatization, have been employed to address this challenge and facilitate more accurate analysis
[13]. Additionally, researchers have investigated a range of filtering and sampling strategies to reduce the volume of data while preserving its representativeness and relevance to the problem at hand
[14]. Despite these efforts, there remains a research gap in developing robust and efficient preprocessing techniques capable of handling the diverse and dynamic nature of social media data, particularly in the context of power systems and disaster management.
Another challenge in the collection of social media data is the need to obtain real-time, geographically relevant data directly related to the event of interest. Although numerous studies have proposed methods for geolocating social media data
[15], these methods often suffer from limitations in terms of accuracy and coverage. Additionally, the issue of data sparsity and the uneven distribution of social media users across different geographic regions present further challenges for the collection and analysis of geographically relevant data
[16]. There is a pressing need for more research on effective methods for obtaining real-time, geographically relevant social media data that can provide valuable insights into the dynamics of power systems and disaster management.
2.2 Emotional Analysis in Social Media
Sentiment analysis, or emotional analysis, has been extensively researched in the context of social media data, with various methods proposed to assess the emotional state of users. These methods can be broadly classified into two categories: lexicon-based techniques and machine learning-based approaches
[17].
Lexicon-based techniques, such as the VADER sentiment analysis tool
[18], rely on a pre-defined sentiment lexicon to calculate the sentiment scores of texts. These methods are relatively simple and computationally efficient, making them suitable for large-scale analysis of social media data. However, they are limited by their dependence on a fixed lexicon, which may not capture the nuances of sentiment expression in different contexts or across various social media platforms
[19]. Moreover, lexicon-based techniques may struggle to accurately capture sentiment in the presence of sarcasm, irony, or other complex linguistic constructs
[20].
Machine learning-based approaches, such as SVM and deep learning models, offer a more flexible and data-driven alternative to lexicon-based techniques
[21]. These methods learn to predict sentiment from labeled training data, which can be manually annotated or automatically generated using various strategies
[22]. Machine learning-based approaches have shown promising results in sentiment analysis tasks, particularly when combined with advanced feature extraction techniques such as word embeddings or pre-trained language models. However, these methods are typically more computationally expensive and may require large amounts of labeled data for training, which can be a challenge in certain domains or for specific events. Despite the advancements in sentiment analysis techniques, there remains a research gap in the development of methods that can effectively capture the intricacies of sentiment expression in the context of power systems and disaster management. Traditional lexicon-based and machine learning-based methods may struggle to accurately assess sentiment in the presence of domain-specific jargon, evolving language patterns, and the inherent complexities of online communication. Furthermore, existing studies in sentiment analysis have primarily focused on general-purpose applications, with limited attention given to the challenges and requirements specific to the power systems and disaster management domain.
Recently, some researchers have started exploring the potential of transfer learning and domain adaptation techniques to address the limitations of traditional sentiment analysis methods
[23]. These approaches aim to leverage knowledge learned from a source domain (e.g., general-purpose sentiment analysis) to improve performance in a target domain (e.g., sentiment analysis for power systems and disaster management) by adapting the learning models and features to the specific characteristics of the target domain. Such techniques have shown promise in various sentiment analysis tasks, but their application to the power systems and disaster management domain remains an open research question.
2.3 Social Media's Impact on Critical Infrastructure
This section sheds light on the profound influence of social media on critical infrastructure. Ref [
24] conducts a systematic literature review on the use of social media in disaster recovery, addressing a noted research gap. It analyses the frequency, platforms, and patterns of social media use during post-disaster phases, categorizing findings into seven key aspects of disaster recovery, such as financial support, mental health, and infrastructure services. The comprehensive review aims to guide the disaster risk reduction community and future researchers by offering insights into the current state of knowledge and highlighting areas for further exploration. Ref [
25] utilizes natural language processing and data mining to extract situation awareness information from Twitter messages created during various crises and disasters. Recognizing the real-time nature of Twitter, the study highlights the platform's growing importance in offering rapid communication, especially during emergencies, allowing affected individuals to relay on-the-ground experiences and insights. This approach, distinct due to its focus on high-speed Twitter text streams during emergencies, aims to amplify the perception level of situation awareness, aiding authorities in making better-informed decisions during critical situations. Ref [
26] explores the influence of social media on healthcare services, prompted by the rise in internet accessibility, affordable mobile devices, and the growth of social media platforms. While global studies indicate benefits for both patients and healthcare providers, the Romanian healthcare system grapples with foundational challenges. Through secondary data, the research seeks to highlight concerns about social media's effects on healthcare services, particularly within the Romanian context.
2.4 Reinforcement Learning Algorithms for Mitigating Uncertain Load Redistribution Cyberattacks
Reinforcement learning (RL) is a prominent subfield of machine learning that focuses on training agents to make decisions by interacting with their environment
[27]. Over the past few decades, RL algorithms have been successfully applied to various complex and dynamic domains, including robotics
[28], finance
[29], and healthcare
[30], among others.
In the context of power systems, RL algorithms have been used to address a wide range of problems, such as optimal power flow, demand-side management, and voltage control
[31, 32]. However, the application of RL algorithms to the problem of mitigating uncertain load redistribution cyberattacks in power systems is relatively unexplored. Existing studies in this domain have primarily focused on deterministic load redistribution scenarios and have employed classical optimization methods, such as linear programming or mixed-integer programming, to develop defense strategies
[33, 34]. These classical optimization methods, while effective in certain scenarios, may struggle to cope with the dynamic and uncertain nature of load redistribution cyberattacks. Moreover, they often require precise knowledge of system parameters and rely on predefined models, which may not accurately capture the complex and evolving interactions between the power system and the adversary.
Reinforcement learning algorithms, on the other hand, provide a model-free and adaptive approach to decision-making that can potentially address the challenges associated with uncertain load redistribution cyberattacks. By learning optimal policies through trial-and-error interactions with the environment, RL algorithms can adapt to changing conditions and uncertainties without the need for explicit system models
[35, 36]. Moreover, the use of advanced function approximation techniques, such as deep neural networks, allows RL algorithms to scale to high-dimensional state and action spaces, which are common in power systems
[37]. In summary, the application of reinforcement learning algorithms to mitigate uncertain load redistribution cyberattacks in power systems is a promising and underexplored research direction. By leveraging the model-free, adaptive, and scalable nature of RL algorithms, it may be possible to develop more robust and effective defense strategies against these attacks, thereby enhancing the resilience and security of power systems.
2.5 Research Gaps and Challenges
Through our extensive literature review, we have identified several research gaps and challenges in the domains of social media data collection and preprocessing, emotional analysis, and reinforcement learning algorithms for mitigating uncertain load redistribution cyberattacks:
1. The need for effective methods for obtaining real-time, geographically relevant social media data that can provide valuable insights into the dynamics of power systems and disaster management.
2. The development of sentiment analysis methods that can effectively capture the intricacies of sentiment expression in the context of power systems and disaster management, while addressing the challenges and requirements specific to this domain.
3. The application of reinforcement learning algorithms for mitigating uncertain load redistribution cyberattacks in power systems, with a focus on developing efficient simulation environments, appropriate reward functions, and exploration strategies.
4. The integration of emotional analysis insights from social media data with reinforcement learning-based defense strategies to enhance the resilience and security of power systems against cyberattacks.
In particular, while many studies in the domain of power grid resilience and sentiment analysis employ traditional techniques, our research forges a distinctive path by integrating advanced analytical tools in a novel manner. At the core of our innovative approach lies the seamless fusion of two sentiment analysis tools: VADER, known for its lexical approach to sentiment detection, and SVM, a machine learning technique with proven accuracy in classification tasks. Yet, what truly sets our work apart is the subsequent integration of these sentiment outcomes with the PPO method. This multidimensional strategy not only enhances the resilience of power grids but also brings forth a nuanced understanding of the intricate relationship between social media sentiment and power grid stability. Our belief is that the synergy between these methods can provide a more robust and real-time response mechanism for power grids under duress from cyberattacks. By leveraging the real-time sentiments extracted from social media, our integrated PPO method can make dynamic decisions to bolster the grid's defenses. This innovative fusion, we argue, is pivotal in making power grids smarter and more resilient in the age of digitization and interconnectedness.
3 Data Crawling and Preprocessing
In this section, we provide a detailed description of the process of collecting and preprocessing social media data for our study. We focus on Twitter as our primary data source due to its widespread use, real-time nature, and accessibility during natural disasters or cyberattacks. The following subsections outline the data collection and preprocessing techniques employed to ensure the quality of our dataset. To collect relevant data from Twitter, we utilize the Tweepy library, which is a Python-based wrapper for the Twitter API. This library allows us to query and collect tweets containing specific keywords, hashtags, and user mentions related to power outages. We define a set of relevant search terms, such as "power outage", "blackout", "electricity", "natural disaster" and "cyberattack", among others, to filter and collect tweets that discuss power outages in the aftermath of a natural disaster or cyberattack.
In addition to text-based queries, we also collect metadata associated with each tweet, such as timestamps, geolocation, user information, and engagement metrics (e.g., retweets, likes, and replies). This metadata provides valuable context for our analysis and helps us to better understand the spatiotemporal dynamics of power outages and the affected population's emotional state.
Once we have collected the data, we apply several common preprocessing techniques to remove extraneous noise and improve the quality of our dataset. These techniques include:
a. Data deduplication: We remove duplicate tweets from the dataset to prevent over-representation of certain information. This step is essential as retweets and quoted tweets can create redundancy in the data, which may skew the analysis.
Deduplication Equation:
where represents the original dataset and represents the dataset after deduplication.
b. We standardize the text in the tweets by converting all characters to lowercase, removing special characters, and expanding contractions. This process simplifies the text and facilitates further analysis.
Additionally, we apply text normalization to standardize the text in the dataset:
Normalization Equation:
where represents the original text and represents the normalized text. The function denotes the normalization process, which may include lowercasing, stemming, and lemmatization, among other techniques.
c. Stopword removal: We eliminate common stop words, such as "the", "and" and "is", that do not carry significant meaning and can hinder the efficiency of the sentiment analysis.
d. Tokenization: We break down the text into individual words or tokens, which allows us to more easily analyze the content of each tweet. Tokenization is a crucial step in preparing the data for natural language processing tasks, such as sentiment analysis.
e. Stemming and Lemmatization: We reduce words to their root form using stemming and lemmatization techniques. This process ensures that words with similar meanings are treated as the same, which simplifies the analysis and reduces noise.
f. Feature extraction: We extract relevant features from the cleaned text data and metadata to facilitate our emotional analysis. These features may include n-grams, term frequency-inverse document frequency (TF-IDF) vectors, and sentiment lexicons, among others.
We also use a term TF-IDF vectorizer to transform the text data into a numerical representation: TF-IDF Equation:
where TF represents the term frequency of term in document , and IDF represents the inverse document frequency of term across all documents.
By employing these preprocessing techniques, we effectively remove extraneous noise and improve the quality of our social media dataset. This refined dataset enables us to derive meaningful insights from the data and accurately assess the emotional state of the affected population during power outages. The cleaned and preprocessed data will then be used for emotional analysis in the following section. This comprehensive approach to data collection and preprocessing ensures that our analysis is based on high-quality, relevant data, thus increasing the reliability and validity of our findings.
4 Emotional Analysis
In this section, we explicate two widely adopted methods for emotional analysis: a sophisticated lexicon-based technique and an advanced supervised machine learning approach. Utilizing these methodologies, we accurately evaluate the emotional state of individuals affected by power outages by scrutinizing the content of tweets in our meticulously preprocessed dataset. For our lexicon-based technique, we select the highly regarded VADER (Valence Aware Dictionary and sEntiment Reasoner) sentiment analysis tool. VADER is a rule-based model specifically tailored for the examination of social media text. It takes into account the distinct characteristics of online communication, such as the employment of colloquialisms, emoticons, and informal language. VADER utilizes a comprehensive sentiment lexicon containing over 7, 500 words, each assigned a sentiment score ranging from (most negative) to +4 (most positive). The VADER algorithm calculates the sentiment of a given text by aggregating the sentiment scores of individual words and normalizing the result to a range between (most negative) and +1 (most positive). Moreover, VADER considers the impact of negations, intensifiers, and other linguistic nuances to provide a more precise sentiment score.
Moreover, VADER considers the impact of negations (NEG), intensifiers (INT), and other linguistic nuances to provide a more precise sentiment score:
To implement VADER in our study, we employ the Python library NLTK (Natural Language Toolkit), which encompasses the VADER sentiment analysis module. By applying VADER to our preprocessed dataset, we procure sentiment scores for each tweet, enabling us to assess the overarching emotional state of the affected population.
For our advanced supervised machine learning approach, we opt for the Support Vector Machine (SVM) classifier. SVM is a renowned and extensively utilized algorithm in sentiment analysis due to its capability to manage high-dimensional feature spaces and its robustness against overfitting. To implement the SVM classifier, we initially partition our preprocessed dataset into a training set and a test set. Subsequently, we manually annotate a subset of the training data with sentiment labels (e.g., positive, negative, or neutral) to create a labeled dataset for training the SVM model. The feature extraction techniques applied during the preprocessing stage, such as -grams and TF-IDF vectors, serve as input features for the SVM classifier.
The SVM classifier aims to find the optimal decision boundary (hyperplane) that maximizes the margin between the classes. This is achieved by solving the following optimization problem:
minimize:
subject to:
where:
● : weight vector
● : bias term
● : regularization parameter
● : slack variables
We employ the Python library scikit-learn to implement the SVM classifier, utilizing the linear kernel function and fine-tuning hyperparameters such as the regularization parameter and the kernel parameters for optimal performance. Once the SVM model is trained, we test its performance on the test set, using evaluation metrics such as precision, recall, F1-score, and accuracy to assess the model's efficacy in predicting the sentiment of the tweets.
By amalgamating the outcomes of the VADER sentiment analysis and the SVM classifier, we can achieve a comprehensive understanding of the emotional state of the affected population during power outages. This emotional analysis serves as a foundation for the development of defense strategies in the subsequent section, ensuring a nuanced and targeted response to the unique challenges faced during power outages.
5 Defense Strategy Development and Outage Uncertainty Management
In this section, we present a case study that demonstrates the application of our methodology to mitigating uncertain load redistribution cyberattacks in an IEEE 6-bus system. We utilize the Proximal Policy Optimization (PPO) algorithm to model the defense strategy development and handle the outage uncertainty in the optimal power flow problem.
5.1 Problem Formulation
The IEEE 6-bus system is a widely used benchmark for testing power system analysis and optimization algorithms. It consists of 6 buses, 11 transmission lines, 3 generators, and 3 load points. In the context of uncertain load redistribution caused by cyberattacks, our objective is to develop a defense strategy that maintains the stability of the power grid while minimizing the impact of the cyberattack.
The problem is formulated as an optimal power flow problem with outage uncertainty. The objective function is to minimize the generation cost while satisfying various power system constraints such as power balance, voltage limits, and line flow limits. The uncertainty in load distribution is modeled as random variables, and their impact on the power grid is taken into consideration during the optimization process.
The objective function can be expressed as:
subject to:
where is the generation cost function, is the cost function for demand response programs, and are the lower and upper bounds for the active power generation, and are the lower and upper bounds for the reactive power generation, and are the lower and upper bounds for the voltage magnitude, and are the power balance equations for active and reactive power, and are the active and reactive power losses, and is a weighting factor that determines the trade-off between generation cost and demand response cost.
5.2 PPO Architecture and Training
Proximal Policy Optimization (PPO) is a state-of-the-art reinforcement learning algorithm developed by OpenAI that offers improved stability and efficiency compared to other policy optimization methods. It is an actor-critic algorithm, which means it uses two separate networks: An actor network responsible for selecting actions and a critic network for estimating the value of those actions. PPO addresses the challenges of balancing exploration and exploitation in reinforcement learning by employing a trust region optimization approach.
The PPO algorithm introduces a surrogate objective function that measures the improvement in the expected cumulative reward while limiting the deviation of the new policy from the old policy. This is achieved by using a clipped objective function, which discourages excessively large policy updates that could lead to instability during training. The clipped objective function can be expressed as:
where represents the policy parameters, is the probability ratio of the new policy to the old policy, is the advantage function, is the expectation over time steps , and is the clipping parameter.
During training, the algorithm interacts with the power system by selecting actions based on the current policy, observing the resulting system state and reward, and then updating the policy to improve the expected cumulative reward. The training process continues until the policy converges to an optimal defense strategy that effectively manages the uncertain load redistribution caused by cyberattacks while maintaining the stability of the power grid. To implement the PPO algorithm, we define the following components:
1. State representation (): Includes the power system state variables, such as bus voltages and line flows, as well as the real-time information gathered from social media platforms, such as outage locations and public sentiment.
2. Action space (): Consists of the dispatch of generation resources and the implementation of demand response programs.
3. Reward function (): A function of generation cost, power balance, system constraints, and the satisfaction of the affected population.
where , , , and are weighting factors that determine the relative importance of each term in the reward function, and represents the voltage deviation from the reference values. The PPO algorithm optimizes the policy by maximizing the expected cumulative reward, which can be expressed as , Where is the discount factor, is the time step, and and are the state and action at time step .
5.3 Defense Strategy Optimization
Our case study demonstrates the effectiveness of the PPO algorithm in optimizing defense strategies to mitigate the impact of uncertain load redistribution cyberattacks in the IEEE 6-bus system. The modelling of the particular load redistribution cyberattacks are based on
[38–40]. By leveraging the information gathered from social media platforms and incorporating it into the decision-making process, the PPO-based defense strategy is able to maintain the stability of the power grid and ensure the efficient allocation of resources during the cyberattack.
The PPO algorithm manages the outage uncertainty by adaptively adjusting the dispatch of generation resources based on the real-time information from social media and the observed power system state. The optimized defense strategy takes into consideration the uncertain load distribution and the associated system constraints to minimize the generation cost while maintaining the stability of the power grid.
6 Case studies
6.1 Data Collection
In the challenging world of sentiment analysis, a crucial initial step revolves around the meticulous collection of pertinent data. In our case, the data constituted tweets relevant to a major incident --- The 2015 Ukraine power grid cyberattack.
In this study, we decided to zero in on the major event of the Ukraine power grid cyberattack that occurred in 2015. This cyberattack is historically significant because it marked the first confirmed successful cyberattack on a power grid. As a result of this attack, approximately 230, 000 people experienced a power blackout, revealing the stark vulnerability of critical infrastructure to such threats.
When it came to collecting tweets, the selection of the right timeframe was a critical factor. The aim was to gather data that would reflect the real-time reactions of the public to the cyberattack. Accordingly, we focused on collecting tweets posted during the week surrounding the event. We collected an estimated 1500 tweets related to the attack, offering a robust dataset for our sentiment analysis. To ensure the relevance of the data collected, we filtered the tweets using several key criteria. Primarily, we focused on tweets containing specific keywords and phrases related to the cyberattack. However, we recognized that relevant tweets might not always include this exact phrase. To account for this and to broaden our data scope, we also included tweets featuring relevant hashtags. Twitter's hashtag feature has become a primary means for users to categorize their tweets, making them valuable in filtering data for specific topics. The hashtags used for filtering in our study were "#Ukraine", "#PowerOutage", and "#CyberAttack".
The "#Ukraine" hashtag allowed us to include broader tweets about the prevailing situation in Ukraine during the attack period. Although these tweets might not mention the power grid attack explicitly, they offer a vital understanding of the broader context and sentiment. The inclusion of this hashtag resulted in an additional 700 relevant tweets. The hashtags "#PowerOutage" and "#CyberAttack", on the other hand, helped capture tweets discussing the attack's aftermath.
6.2 Data Preprocessing
Data preprocessing is a vital phase of any data-driven study, even more so in the case of sentiment analysis, where the raw data comes in the form of unstructured text data. In this study, we were dealing with a dataset of approximately 1600 tweets. These tweets were replete with a range of elements - some relevant to our study, others not so much. To ensure a refined, high-quality dataset, we embarked on an extensive preprocessing journey. Before any kind of processing, a cursory look at our raw dataset revealed numerous special characters. Twitter users often use special characters like emoticons, punctuation marks, hashtags, and more to express their thoughts and sentiments more vividly. However, such elements usually do not contribute much to the sentiment analysis process and instead can add noise to the data. In our dataset, around 35% of the tweets contained one or more special characters. These were systematically filtered out. Next, we turned our attention to stop words. Stop words, generally speaking, are words that do not carry much semantic weight. They are the most common words in a language like 'the', 'is', 'at', 'which', 'on', etc. While essential for constructing meaningful sentences, they usually do not contribute to the sentiment of a statement. Therefore, in the interest of refining our dataset, we decided to remove these stop words. Around 70% of the tweets contained stop words. After this step, the average word count per tweet dropped from 18 to 13 words, illustrating the significant reduction of noise in our dataset.
Another aspect that needed addressing in our Twitter data was the presence of URLs. Tweets often contain URLs for various purposes, like referring to news articles, blogs, images, and more. However, for the sentiment analysis, these URLs add little value. From our raw data, we found that nearly 20% of the tweets contained URLs. These URLs were systematically removed, further refining our dataset for sentiment analysis. Finally, we noticed the presence of numbers in our data. Numbers can often be a tricky aspect to deal with in sentiment analysis. In some cases, they might carry some sentiment (for example, 'power out for 12 hours'), but in most other instances, they add little sentiment value. After careful deliberation, we decided to remove numbers from our tweets. The impact was minor but relevant, as approximately 15% of tweets contained numbers.
Through this rigorous preprocessing phase, we were left with a dataset that was leaner and more focused for sentiment analysis. Our dataset was now filtered down to approximately 900 tweets. This reduction was not about quantity but about increasing the quality of the data and ensuring that our sentiment analysis model worked with the most pertinent, meaningful data for accurate and insightful results. The efforts put into this preprocessing phase would pay dividends in the subsequent sentiment analysis phase, where clean and precise data is the key to obtaining reliable results.
6.3 Sentiment Analysis
Sentiment analysis, or opinion mining, is a type of data analysis that interprets and classifies the sentiments expressed in a source of text. In the context of our study, the preprocessed tweets were analyzed for their sentiments regarding the Ukraine power grid cyberattack in 2015. We utilized a pre-trained sentiment analysis model for this purpose. Such models are trained on large datasets and are capable of discerning the subtle nuances of human sentiment expressed in text. Given a piece of text (in this case, a tweet), the model assigns a sentiment score that indicates the polarity (positive, negative, or neutral) and intensity of the sentiment expressed.
Upon running our cleaned dataset of approximately 900 tweets through the sentiment analysis model, we received a sentiment score for each tweet. Analyzing these sentiment scores would give us insight into the public's reaction to the cyberattack. The distribution of sentiment scores was quite revealing. Approximately 56% of the tweets had a negative sentiment, indicating a considerable amount of public discontent and concern about the incident. In contrast, positive sentiment was observed in about 28% of the tweets, possibly reflecting support for Ukraine or admiration for the efforts being made to counteract the attack. The remaining 16% of tweets were neutral, indicating a lack of expressed sentiment or a balanced viewpoint. These statistics merely provide an overview. The true value of sentiment analysis lies in its ability to uncover trends and patterns over time. Thus, we analyzed the changes in sentiment scores as the event unfolded.
During the initial stages of the attack, we observed a spike in negative sentiment, which accounted for about 68% of the tweets at that time. This corresponds with the first public reports of the cyberattack, which described widespread power outages and disruptions. The high proportion of negative sentiment reflects the initial shock and worry expressed by the public.
As the week progressed, we noticed a gradual decrease in negative sentiment and an increase in positive and neutral sentiment. After three days, the distribution of sentiment was roughly equal across negative, positive, and neutral tweets. This shift might be attributed to the growing awareness about the nature of the attack, as well as the public's growing confidence in the measures being taken to restore the power grid and prevent future attacks. Interestingly, we noticed a brief resurgence of negative sentiment (approximately 65% of tweets) when reports revealed that the cyberattack was orchestrated by a foreign entity. This highlights the public's sensitivity to the geopolitical implications of the attack.
In addition to these broad patterns, sentiment analysis also helped uncover more specific public concerns and reactions. For example, the most negative sentiment scores were often associated with tweets discussing the immediate inconvenience caused by the power outage and concerns about personal safety and security. On the other hand, tweets expressing positive sentiment often mentioned the resilience of the Ukrainian people or the swift response by the authorities.
The sentiment analysis was not without its challenges. The model sometimes struggled to accurately interpret sarcasm, which is a common form of expression on social media platforms like Twitter. Furthermore, the sentiment analysis was confined to English tweets, which may have led to a slight bias in the results. Future studies might benefit from incorporating sentiment analysis models capable of handling multiple languages. The success of our sentiment analysis provides a wealth of data that is instrumental in mitigating and defending against cyberattacks on power systems, particularly uncertain load redistribution attacks. When a power grid is hit by a cyberattack, it not only impacts the functionality of the grid but also sparks a public response that can be collected and scrutinized in real-time through social media platforms. By analyzing the sentiment of these responses, we can gain immediate insights into the public's perception of the severity of the attack, the effectiveness of the authorities' response, and their expectations.
Such information is invaluable when formulating defense strategies. For example, a surge in negative sentiment might suggest an increase in public anxiety or frustration, implying that the current mitigation measures may not be effectively allaying public concerns or communicating the actions being taken adequately. In such a scenario, operators can leverage this insight to adjust their response strategies, such as improving communication about the steps being taken to restore the grid and guarantee public safety. Furthermore, a sudden shift in sentiment may act as an early warning system, signaling new developments such as an escalation of the attack or the resolution of a significant issue before it is reported through traditional channels. This early warning can help operators stay a step ahead, swiftly adjust their strategies, and potentially prevent a larger crisis.
We propose a novel application of sentiment analysis insights within the framework of our broader defense strategy against uncertain load redistribution cyberattacks. The sentiment analysis serves as a real-time barometer of the perceived severity of the attack, guiding system operators in their decision-making process. This analysis can be directly used to influence the operation of the IEEE 6-bus system by adjusting the parameters of the model according to the assessed severity of the attack. For example, if the sentiment analysis suggests a highly severe attack, the operator might decide to prioritize the restoration of critical loads, or allocate more resources for countermeasures. This real-time severity assessment, as reflected in public sentiment, allows operators to make dynamic and informed adjustments to the power grid's defense strategies, enhancing the resilience of the grid in the face of an attack. Additionally, continuously monitoring public sentiment during an attack enables operators to adapt strategies in real time as the situation unfolds. This flexibility ensures that defense strategies remain effective, targeting specific needs and priorities based on the ongoing severity of the attack.
6.4 The Resilient Defense Strategy
In this work, we explore a modified IEEE 6-bus test system, strategically equipped with conventional and renewable generation resources as well as battery storage. The system layout and primary characteristics are as follows: Bus 1 remains the conventional generation unit, serving as a PV bus with a conventional power plant. This generator represents the traditional generation assets in our power system, providing a stable source of electricity and ancillary services. Bus 2 is uniquely configured with a wind farm, replacing the traditional generation unit. As an intermittent renewable energy source, the wind farm's power output is highly dependent on real-time wind speed and can be modeled either as a negative load (for a PQ bus representation) or as a dispatchable generator (for a PV bus representation) depending on the level of control assumed over the wind farm operations. Bus 3 is installed with a solar photovoltaic (PV) system, another form of intermittent renewable generation. As with the wind farm, the solar PV system's power output fluctuates based on solar irradiance levels and can be modeled similarly as a PQ or PV bus. Buses 4 and 5 remain unaltered, serving as load buses (PQ buses). They represent demand centers within our power system, consuming electrical power according to the load demand profiles. Bus 6 is distinctively outfitted with a battery energy storage system (BESS) in addition to its load. This BESS is capable of bi-directional power flow, absorbing excess power from the grid during times of low demand or high renewable generation, and releasing stored energy back into the grid during peak demand or low renewable generation periods.
To demonstrate the effectiveness of the PPO method, we compare it with two benchmark methods:
Q-Learning is an off-policy, model-free reinforcement learning algorithm that learns optimal action-selection policy using a Q function; SARSA, short for State-Action-Reward-State-Action, is an on-policy reinforcement learning method. It evaluates the current policy being executed. We use 4 different scenarios to test the algorithm, where the baseline case indicates the attack level () is 10%.
Case 1: baseline case.
Case 2: case 1 with =20%.
Case 3: case 1 with =25%.
Case 4: case 1 without energy storage systems.
In response to the sentiment analysis, we propose a comprehensive, tri-level strategy model that adapts to varying severity levels of cyberattacks and the corresponding public sentiment expressed on social media platforms. The three distinct strategies - 'Critical Response', 'Moderate Action', and 'Passive Monitoring' - allow for a dynamic response tailored to the threat level and public reaction. The 'Critical Response' strategy is triggered under conditions of severe cyberattacks, where the potential impact on the power grid is high, coupled with heightened public concern or anxiety as reflected in social media sentiment. This strategy involves immediate action and robust defensive measures to mitigate the impact and quickly restore normalcy. The 'Moderate Action' strategy is activated in cases of moderate severity cyberattacks. Here, the threat to the power grid is significant but not critical, and the public sentiment shows a moderate level of concern. The response in this scenario involves a balanced approach, leveraging advanced threat countermeasures while maintaining operational stability. The 'Passive Monitoring' strategy is employed in situations where the cyberattacks are of low severity, and the public sentiment is relatively neutral or unconcerned. This strategy focuses on continuous monitoring, assessment, and readiness to escalate the response if the threat level increases.
The presented results from the three tables represent the operation costs and load shedding amounts under three distinct cyberattack response states: Critical Response, Moderate Action, and Passive Monitoring. In the Critical Response state, operation costs ranged from ＄32, 124 to ＄39, 025 across the four cases. Load shedding also varied, with a low of 7.2 MWh in Case 1 and a peak of 9.2 MWh in Case 3. This suggests a heightened vigilance and active countermeasures during cyberattack scenarios which leads to increased operational costs, but effectively minimizes the load shedding to a certain degree.
Table 1 Operation costs and load shedding under critical response state |
| Economic result | Case 1 | Case 2 | Case 3 | Case 4 |
| Operation costs (＄) | 32124 | 37289 | 39025 | 34207 |
| Load shedding (MWh) | 7.2 | 8.4 | 9.2 | 8.0 |
Table 2 Operation costs and load shedding under moderate action state |
| Economic result | Case 1 | Case 2 | Case 3 | Case 4 |
| Operation costs (＄) | 31005 | 36274 | 38120 | 33945 |
| Load shedding (MWh) | 7.6 | 8.8 | 9.7 | 8.4 |
Table 3 Operation costs and load shedding under passive monitoring state |
| Economic result | Case 1 | Case 2 | Case 3 | Case 4 |
| Operation costs (＄) | 30250 | 35140 | 37676 | 33125 |
| Load shedding (MWh) | 7.8 | 9.1 | 10.2 | 8.6 |
The Moderate Action state showed slightly reduced operation costs ranging between ＄31, 005 and ＄38, 120, with corresponding load shedding amounts of 7.6 to 9.7 MWh. This demonstrates that less aggressive cyber defense strategies might lower operational expenses, although this could result in a slight increase in load shedding. This suggests a trade-off scenario where operational costs can be mitigated at the expense of a slightly higher risk of load shedding. Finally the Passive Monitoring state showed the least operation costs, ranging from ＄30, 250 to ＄37, 676. Correspondingly, load shedding was highest in this state, from 7.8 MWh to a peak of 10.2 MWh in Case 3. This indicates that a passive approach to handling cyber threats may result in cost savings but at the potential risk of increased load shedding. These results are in line with the strategic proposal of this study, implying that the level of action and mitigation strategy in response to cyber threats impacts both the operational costs and the load shedding of a power system. This reinforces the necessity of a flexible and context-aware approach to managing cyber threats, allowing for adjustments according to the perceived severity and impact of the attacks. It is therefore vital to perform a thorough cost-benefit analysis for each possible state of action, to understand the potential financial implications and ensure the stability of the power grid.
Figure 1 Load shedding comparison with benchmarks |
Full size|PPT slide
6.5 Comparative Analysis
In the evaluation of the proposed PPO method against Q-Learning and SARSA, the resulting load shedding demonstrates the effectiveness of PPO in mitigating the impact of cyberattacks on an IEEE 6-bus system. From the results, the load shedding under the PPO method is measured at 7.2 MWh. In comparison, Q-Learning, an off-policy reinforcement learning algorithm, resulted in a slightly higher load shedding of 7.8 MWh. This could be attributed to the nature of Q-Learning, which learns an optimal action-selection policy using a Q function. While effective in various scenarios, its off-policy approach might not be as efficient as PPO in the context of minimizing load shedding under cyberattacks. SARSA, an on-policy reinforcement learning method, led to an even higher load shedding of 8.3 MWh. Being an on-policy method, SARSA follows the policy currently in use and tends to be more conservative in its decisions. While this could provide stability in certain situations, it might also lead to more load shedding as a trade-off. Thus, the PPO method, under the context of our study, provides superior performance in minimizing load shedding during cyberattacks on the power system. It further emphasizes the importance of employing a suitable reinforcement learning approach tailored to the intricacies of the power system under consideration. These findings align with our initial assumptions and reinforce the key tenets of our research, demonstrating the potential of PPO as an effective solution for enhancing the resilience of power systems in the face of cyber threats.
In the realm of ensuring power system resilience against cyber threats, our research has underscored the tangible advantages of utilizing the PPO method, especially when juxtaposed against traditional reinforcement learning techniques like Q-Learning and SARSA. One of the primary metrics we employed to gauge effectiveness was the load shedding magnitude, an undeniable indicator of a power system's ability to endure cyberattacks. The PPO method manifested its prowess by restricting the load shedding to a commendable 7.2 MWh. This outcome not only stands as a testament to PPO's efficacy but also its adaptability to the dynamic challenges posed by cyberattacks on an IEEE 6-bus system. In stark contrast, Q-Learning, renowned for its off-policy nature, culminated in a load shedding of 7.8 MWh. While Q-Learning's methodology, which leans on learning an optimal action-selection policy through a Q function, has been efficacious in myriad scenarios, it exhibited certain limitations when faced with the pressing demand of minimizing load shedding amidst cyberattacks. The possible shortfall might stem from its off-policy approach, which, although robust in certain situations, perhaps lacks the nuanced adaptability required in this critical application. Adding another layer to our comparative analysis, we evaluated SARSA, an on-policy reinforcement learning technique. SARSA's results unveiled an even more pronounced load shedding of 8.3 MWh. As an on-policy algorithm, SARSA inherently adheres to the policy that's currently active, ushering in a conservative stance in its decision-making process. While such conservatism might offer stability in specific applications, it also seemed to exact a price in terms of increased load shedding, especially when the system is under cyber duress.
In synthesis, the data-driven insights derived from our study unambiguously position the PPO method as a vanguard in terms of mitigating the repercussions of cyberattacks on power systems. Its ability to astutely balance decision-making, drawing from its inherent methodologies, offers a path forward for power grid managers and researchers alike. The findings from our research not only reaffirm our initial hypotheses but also serve as a clarion call for further exploration and adoption of methods like PPO, which are tailored to navigate the intricate labyrinth of challenges that power systems routinely grapple with.
7 Conclusion
We present a novel approach that integrates social media sentiment analysis with reinforcement learning to enhance power system resilience. We utilize sentiment analysis methods, VADER and SVM, to understand public sentiment during power outages. This insight informs our defense strategies against uncertain load redistribution cyberattacks. To manage outage uncertainty and optimize defense strategies, we employ the PPO, a cutting-edge reinforcement learning algorithm. The effectiveness of this approach is demonstrated on a modified IEEE 6-bus system, highlighting the potential of such integrated strategies in mitigating the impact of cyberattacks. Our methodology underscores the significant value of social media sentiment analysis in power systems security and disaster management. This innovative approach offers promising potential in the field of power systems and cybersecurity. Future work could explore further natural language processing and advanced reinforcement learning techniques for more robust cyber threat management.
{{custom_sec.title}}
{{custom_sec.title}}
{{custom_sec.content}}
PDF(243 KB)
), Saike HE*,1,3(
Table 1 Operation costs and load shedding under critical response state
Figure 1 Load shedding comparison with benchmarks
