<?xml-model href='http://www.tei-c.org/release/xml/tei/custom/schema/relaxng/tei_all.rng' schematypens='http://relaxng.org/ns/structure/1.0'?><TEI xmlns="http://www.tei-c.org/ns/1.0">
	<teiHeader>
		<fileDesc>
			<titleStmt><title level='a'>Data-driven inferences of agency-level risk and response communication on COVID-19 through social media-based interactions</title></titleStmt>
			<publicationStmt>
				<publisher></publisher>
				<date>10/01/2021</date>
			</publicationStmt>
			<sourceDesc>
				<bibl> 
					<idno type="par_id">10318083</idno>
					<idno type="doi">10.5055/jem.0589</idno>
					<title level='j'>Journal of Emergency Management</title>
<idno>1543-5865</idno>
<biblScope unit="volume">19</biblScope>
<biblScope unit="issue">7</biblScope>					

					<author>PhD Candidate Ahmed</author><author>PhD Sadri</author><author>PhD Amini</author>
				</bibl>
			</sourceDesc>
		</fileDesc>
		<profileDesc>
			<abstract><ab><![CDATA[Risk perception and risk averting behaviors of public agencies in the emergence and spread of COVID-19 can be retrieved through online social media (Twitter), and such interactions can be echoed in other information outlets. This study collected time-sensitive online social media data and analyzed patterns of health risk communication of public health and emergency agencies in the emergence and spread of novel coronavirus using data-driven methods. The major focus is toward understanding how policy-making agencies communicate risk and response information through social media during a pandemic and influence community response—ie, timing of lockdown, timing of reopening, etc.—and disease outbreak indicators—ie, number of confirmed cases and number of deaths. Twitter data of six major public organizations (1,000-4,500 tweets per organization) are collected from February 21, 2020 to June 6, 2020. Several machine learning algorithms, including dynamic topic model and sentiment analysis, are applied over time to identify the topic dynamics over the specific timeline of the pandemic. Organizations emphasized on various topics—eg, importance of wearing face mask, home quarantine, understanding the symptoms, social distancing and contact tracing, emerging community transmission, lack of personal protective equipment, COVID-19 testing and medical supplies, effect of tobacco, pandemic stress management, increasing hospitalization rate, upcoming hurricane season, use of convalescent plasma for COVID-19 treatment, maintaining hygiene, and the role of healthcare podcast in different timeline. The findings can benefit emergency management, policymakers, and public health agencies to identify targeted information dissemination policies for public with diverse needs based on how local, federal, and international agencies reacted to COVID-19.]]></ab></abstract>
		</profileDesc>
	</teiHeader>
	<text><body xmlns="http://www.tei-c.org/ns/1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink">
<div xmlns="http://www.tei-c.org/ns/1.0"><head>INTRODUCTION</head><p>No person or place is resistant from disasters or losses resulting from an extreme event. Infectious disease, in addition to natural hazards, wildfires, acts of terrorism, civil instability, or financial disasters may all have far-reaching effects for the country and its communities. <ref type="bibr">1</ref> One way to reduce the impact of disasters on the nation and its communities is by investing in resilience enhancement similar to infrastructure resilience. <ref type="bibr">2</ref> Building disaster resilience culture and practice, however, is not simple or inexpensive. Bottom-up initiatives (communities' dedication to improving their resilience) are important as local conditions differ widely throughout the world. Communicating risks, connecting community networks, <ref type="bibr">3</ref> and promoting resilience culture can aid local communities in making progress to increase their resilience. Throughout this regard, community coalitions of elected representatives from the public and private sectors, with ties and funding from federal and state governments, and with input from local residents, are becoming very relevant. Such coalitions DOI:10.5055/jem.0589 may be charged to determine the exposure and vulnerability of the community to hazard, to inform and communicate risk, and to analyze and extend the capacity of the community to manage such risk. A truly robust coalition will have a clear leadership and governance system at its heart, and people with sufficient time, expertise, and commitment required to establish and sustain relationships among all the community stakeholders. <ref type="bibr">4</ref> Public agencies engagement in risk communication can lead to more effective decision-making and enhanced public feedback to the regulatory process. The primary goal of this study is to mine and analyze large-scale time-sensitive perishable crowdsourced and social media data (rich spatiotemporal data) and reveal interaction patterns of health-risk communication and community responses of major public agencies in the emergence and spread of novel coronavirus (nCov) using data-driven methods and network science theories. The specific aims are twofold: (1) to document how public agencies interact and communicate health risk information through their online social networks during a major disease outbreak and (2) to examine how online social networks influence the current COVID-19 pandemic situation in terms of the change in daily number of cases and deaths in the United States. To achieve the goal and aims, this study has utilized time sensitive social media interactions of six major public agencies: World Health Organization (WHO), Center for Disease Control and Prevention (CDC), Federal Emergency Management Agency (FEMA), American Red Cross, New York City (NYC) Department of Health and Mental Hygiene, and Mount Sinai Health System, NYC with crowd-sourced information on COVID- 19.  This study has four theoretical and methodological contributions as compared with the literature:</p><p>(1) It has developed methods to advance our understanding of how public agencies influence online, while communicating health risks and interacting in their respective communities as the disease continues to spread.</p><p>(2) It informed the literature on how information is exchanged among people who are socially connected online and exposed to health risk in such outbreaks of disease.</p><p>(3) This study used novel machine learning models to quantify the influence on such communication behavior.</p><p>(4) It captured the variability in risk communication strategies and risk averting behaviors adopted based on temporal correlations of risk and disease contagion.</p><p>The findings from this research will be useful to public health and emergency management agencies for tailoring effective information dissemination policies for diverse user groups based on their social network characteristics, activities, and interactions in response to similar public health hazards. The methodologies and implications of this research can be transferred in designing effective intervention policies to other natural and man-made disaster contexts, in which public health risks become major concerns.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>LITERATURE REVIEW</head><p>The transmutability of the nCov disease in 2019 was tracked in Zhejiang, China, accounting for transmissions from imported cases. While Zhejiang is one of the worst 21 affected provinces, an interruption in the transmission of the disease, ie, an instant reproduction number &lt; 1, was observed in early/mid-February 2020 following an early social-distancing response to the outbreak. <ref type="bibr">5</ref> Zhang et al. developed data-driven, susceptible-exposed-infectious-quarantine-recovered models to simulate the COVID-19 outbreak with measures of social distancing and epicenter lockdown. Population migration data combined with officially recorded data were used to estimate model parameters and then to measure the daily exported infected individuals by estimating the daily infected ratio and the daily susceptible population size. <ref type="bibr">6</ref> In requires steps ranging from increasing social distancing to city-wide quarantine. Whether these steps (isolation, quarantine, and social distancing) will be appropriate to monitor 2019-nCoV depends on resolving some unanswered issues. <ref type="bibr">7</ref> Many countries only seek to achieve social distancing and hygiene measures when widespread transmission is apparent. It gives the virus a number of weeks to propagate with a higher basic reproductive number than if it had been in place before transmission was observed or widespread. Hence, preventive, low cost, enhanced hygiene, and social distance in the sense of imminent population transmission of the nCoV COVID-19 should be considered. <ref type="bibr">8</ref> Social media is considered as an accessible and widely used platform for efficient crisis communication in recent literature. <ref type="bibr">[9]</ref><ref type="bibr">[10]</ref><ref type="bibr">[11]</ref> Traditional media is primarily intended for one-way communication, whereas social media allows two-way communication; hence, social media platforms are quite different and provide a wider range of data. <ref type="bibr">12,</ref><ref type="bibr">13</ref> Sadri et al. explored the critical role of social media during crisis by facilitating communication and information dissemination to both evacuee and nonevacuee during Hurricane Sandy. <ref type="bibr">14</ref> Zhang et al. envisage intelligent public disaster information and alert based on social media, which has three functions: (1) effectively and efficiently collecting disaster situational awareness information <ref type="bibr">15</ref> ; (2) promoting self-organized assistance activities; and (3) enabling emergency management agencies to hear from the public. The results of this analysis highlight the importance of such research fields: (1) a fine-grained social media catastrophe ontology with semantic interoperability, (2) trend knowledge network pattern and emerging prominent users, (3) fine-grained societal impact assessment due to infrastructure failures, and (4) best practices for social media use during disasters. <ref type="bibr">16</ref> Austin et al. examine how audiences seek social and traditional media information, and what factors influence media usage during crises. Using the model of social-mediated crisis communication, a review of crisis information and sources reveals that audiences use social media for insider information and check-in with family/friends during crises, and use traditional medias, such as television, radio, newspaper, etc., for increasing public awareness. Convenience, interaction, and personal feedback encourage the use of conventional and social media; usage of both discourages overload. Humor and beliefs toward social media uses discourage the use of social media, while legitimacy promotes the use of conventional media. The findings stressed the importance of the influence of third parties in crisis communication and the need to use traditional and social media in crisis response. <ref type="bibr">17</ref> Freimuth et al. describe the design, implementation, and assessment of a risk communication simulation during the first hours of a pandemic. The simulation design was focused on the communication of crisis and emergency risks principles upheld by the CDC, as well as the author's collective experience. Several local health district risk communicators in Georgia responded to a scenario where after returning from an international conference, every community in the state had teenagers infected with avian flu. The evaluation revealed that under the time pressures of a realistic and stressful event, local risk communicators had much greater difficulty following the principles of risk communication than they did in a tabletop workout. Strengths and weaknesses of local risk communicators' performance are identified in addition to the lessons learned on designing and implementing a simulation for risk management. <ref type="bibr">18</ref> Palen et al. <ref type="bibr">19</ref> studied the rapid growth of social media in a number of disaster situations, exploring issues, such as citizen engagement, communityoriented computing, distributed problem solving, and digital volunteerism as modes of sociotechnical innovation, as well as issues of situational knowledge and truthfulness as opportunities and challenges emerging from the social media data deluge. This article also discusses the study that looks at integrating social media technologies and data into current emergency response work. Reflecting on the decade-old area of science, the authors warned of the danger that all "crisis" encounters can fail unintentionally without differentiation, which appears to happen because social media networks cross-cut all emergency situations. nontechnological influences on cultural sociobehavioral scales greatly affect the usage of social media itself.</p><p>Misinformation spreading in social media is also becoming an evolving concern. Monahan et al. mentioned that mass media play an important but often misunderstood role in the events of a catastrophe. Research has repeatedly shown that disaster-related media reporting appears to be riddled with disinformation and promotes misconceptions about race, social status, aggression, and crime. Studies have found that powerful media campaigns can support prevention efforts, strengthen early warning systems, facilitate orderly, prompt evacuation procedures, and help bring communities together in times of upheaval. Authors review research on the relationship between media and disaster to highlight the many ways media can positively impact disaster preparation and recovery, while also highlighting the many issues associated with disaster reporting. Future directions for media-disaster research are being discussed along with ways in which media staff and emergency response professionals can handle the media-disaster relationship more efficiently before, during, and after emergency incidents. <ref type="bibr">20</ref> Battur et al. detected twitter bot, which is a software that sends fake tweets automatically to users. Detecting bots is necessary to identify the fake users and to protect the genuine users from misinformation and malicious intents. This study proposes an approach to detect the twitter bots using several machine learning algorithms, such as Decision Tree, Multinomial Na&#239;ve Bayes, Random Forest, and Bag of Words. The algorithm with highest accuracy (Bag of Words) is used to test real time data. <ref type="bibr">21</ref> Asr et al. stated that misinformation detection at the level of full news articles is a text classification problem, and reliably labeled data in this domain are rare. Previous work relied on news headlines, microblogs, tweets, and articles collected from so-called "reputable" and "suspicious" websites and labeled accordingly. Authors leveraged fact-checking websites to collect individually labeled news articles with regard to the veracity of their content. <ref type="bibr">22</ref> Huang et al. proposed a systematic meta-analysis looked at 38 studies involving real responses to hurricane warnings and 11 studies with expected responses to hypothetical hurricane performed since 1991. <ref type="bibr">23</ref> </p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>METHODOLOGY</head><p>The focus of this study is to examine how public agencies' risk assessments, risk averting behaviors, and crisis communication patterns in online social network influence the pandemic situation using a datadriven solution that leverages Twitter data. Several machine learning algorithms have been used in this study to reveal different agencies' interaction patterns. First, a sentiment analysis over time for major public organizations had performed. Then, dynamic topic model (DTM) (based on Topic Model theory) was applied to identify the topics over the same time horizon that causes the change of sentiments.</p><p>Sentiment analysis is a method of natural language processing (NLP) task at many levels of granularity. Starting from a document level classification task, it has been deployed at the sentence level and more recently at the phrase level. <ref type="bibr">24</ref> It is well recognized that twitter user-generated content with rich sentiment information should be utilized for many applications such as search engines and other information systems. While tweet level sentiment analysis results indeed provide very useful information, the overall or general sentiment tendency toward topics is more appealing in some scenarios. For example, different stakeholders, eg, advertising companies, are curious about how others feel about Apple's new product, "iPhone11," and it will offer great convenience for them if major opinions are collected from massive tweets. <ref type="bibr">25</ref> To derive the sentiment scores, a text sentiment analysis model named VADER (valence aware dictionary for sentiment reasoning) <ref type="bibr">26</ref> is used, which considers both the polarity (positive/negative) and the force (strength) of emotion. It calculates the positive, negative, and neutral sentiment score for each string (set of words) and reports the compound (compute by normalization) score directly. The sentimental study of VADER is based on a dictionary that maps lexical characteristics to emotion intensities, which are referred to as sentiment ratings. A text's sentiment score can be calculated by including the strength of each word in the text. <ref type="bibr">27</ref> Vol. 19, No. 7</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Special Issue on COVID-19</head><p>A topic model is a statistical model to explore the abstract "topics" that happen in an assembly of information in machine learning and NLP. This was initially explored by David Blei according to the most common topic model named Latent Dirichlet Allocation (LDA). The idea behind LDA is that the set of texts reveal numerous topics. In topic models, first, the algorithm selects a topic, and then sample a set of words from the given topic. Clusters of similar terms are the themes or topics created by topic modeling technology. A topic model is a recurrently performed text-mining tool for the discovery of hidden semantic structures in a text body. Topic models can help us to organize and provide insights into understanding large collections of unstructured text bodies. <ref type="bibr">28</ref> The DTM includes a group of probabilistic time series model, which is used to observe the time evolution of topics in huge document collections. This group of models was proposed by David Blei and John Lafferty and is an extension to LDA that can handle chronological documents. In LDA, both the order and the words appear in a document, whereas words are still assumed to be interchangeable, but in a DTM, the order of the documents plays a key role. The method is to use state-space models to represent the topics on the natural parameters of the multinomial distributions. To perform approximate posterior inference over the latent topics, variation in approximations based on Kalman filters and nonparametric wavelet regression are developed. In addition to providing sequential, quantitative, and predictive models, DTM provides a qualitative window into the contents of a large document collection.</p><p>It is assumed that the data are divided by time intervals in a DTM, for example, by month. It is modeled as each slice's documents with a K component subject model, where slice t-related topics evolve from slice t-1-related topics. Let &#946; t denotes the V-vector of natural parameters for topic k in slice t for a K-component model with V terms. A multinomial distribution is usually represented by its mean parameterization. If we denote the mean parameter of a V-dimensional multinomial by &#960;, the mapping &#946; i = log (&#960; i /&#960; V ) of the ith element of the natural parameter is given. Dirichlet distributions are used in typical language modeling applications to model uncertainty over word distributions. The Dirichlet, however, is not conducive to sequential modeling. Alternatively, we chain the natural parameters of each &#946; t subject into a state-space model that evolves with Gaussian noise; the following equation shows the simplest version of such a model.</p><p>In LDA, the document-specific topic proportions &#952; are drawn from a Dirichlet distribution. In the DTM, a logistic normal is used with mean &#945; to express uncertainty over proportions. The sequential structure between models is again captured with a simple dynamic model, which is expressed by the following equation:</p><p>For simplicity, it did not model the dynamics of topic correlation, as it was done for static models by Blei and Lafferty. <ref type="bibr">29</ref> By chaining together topics and topic proportion distributions, it has sequentially tied a collection of topic models.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>DATA SOURCE AND DATA COLLECTION</head><p>Existing traditional datasets have limited capacity to adequately capture user risk communication strategies and analyze user concerns with such details and coverage. As such, social media datasets, enriched with user activity information, will be useful to capture user sentiments and concerns in real time and help early detection of the people exposed to healthrisks in the vulnerable communities. Twitter, in particular, provides unique features to release their data through their application programing interface (API) and make it publicly available which could then be combined with other complementary information, eg, crowdsourced data, in particular over the timeline of COVID-19 crisis. For this study, Twitter-search APIs is used to collect and store public agencies' crisis interactions through social media outlets. NLP and machine learning techniques are adopted here to extract user concerns, response, and needs over time. However, Worldometer data (crowdsource data) is used for extracting the daily number of cases and deaths due to COVID-19 for the United States as it is one of the most reliable and trustworthy sources. <ref type="bibr">30</ref> The main reason Vol. 19, No. 7</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Special Issue on COVID-19</head><p>for considering these data is to compare the prominent discussions on Twitter along with the changes in number of cases and deaths due to COVID-19.</p><p>The goal of this study is to reveal some specific agencies' perspective during COVID-19 that exists in social media interactions. To achieve this goal, the first key data source we considered is the Twitter data. We were particularly interested in the tweets generated from major public health agencies (WHO, CDC, and NYC Department of Health), disaster management organizations (FEMA and American Red Cross The following flowchart (Figure <ref type="figure">1</ref>) is showing the methodology of the data analytics for this study.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>ANALYSES AND RESULTS</head><p>To reveal the interaction patterns of the public organizations, several machine learning algorithms, including sentiment analysis and topic frequency, are applied over time. The graphical representations and </p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>WHO interactions</head><p>To understand the dynamics of the crisis communication pattern, 15 optimum number of topics were identified from static topic model analysis of WHO tweets. In Figure <ref type="figure">A1</ref>, the topic frequencies are represented along with the time (all the dates are in the year of 2020). Furthermore, in Figure <ref type="figure">A2</ref>, average sentiment scores of tweets are illustrated over the same timestamp. WHO emphasized on community transmission and the need of vaccine in March; shortage of healthcare workers was discussed (Figure <ref type="figure">A1</ref>) at the beginning of April; shortage of personal protective equipment (PPE), need of staying home, and importance of wearing masks became prominent from late April and in May 2020. However, requirement of government support got notified at the beginning of May, the number of cases (COVID-19 in countries topic) found increasing after 1.5 months' time interval. In Figure <ref type="figure">2</ref>, the word probabilities over the timeline of each topic are visualized from which the change in social distancing phrase to physical distancing is observed in topic 10.</p><p>Several other topics, such as health concerns, importance of physical activities, responses from different countries, social measures, and use of tobacco, received attention also from WHO in twitter.</p><p>By comparing Figures <ref type="figure">A1</ref> and<ref type="figure">A2</ref> (rolling mean or 1-day running average), we observe that the tweet sentiment went to most negative, while WHO emphasized on alarming number of cases of COVID-19, lack of peoples' responses, shortage of PPE, and negative impact of using tobacco. Then, positive sentiments have found when WHO discussed about the need of government support, responses from people, development of vaccine, importance of home quarantine, and use of tobacco to develop the vaccine. Hence, the use of tobacco considered as negative topic in the beginning, but later it turned into positive topic.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>CDC interactions</head><p>From CDC tweets, eight optimum number of topics are plotted over the three-and half-month timeline in Figure <ref type="figure">A3</ref>. CDC put importance on following their guideline consistently, increase in number of cases in whole March, hospitalization rates at the end of April which also found frequent after 1 month, pandemic stress in May rather than from the beginning, prevention of spread during the mid-April, contact tracing at the beginning of April, and risk of older people in March and May. In Figure <ref type="figure">3</ref>, the word probabilities over the time of each topic are presented.</p><p>After comparing Figures <ref type="figure">A3</ref> and<ref type="figure">A4</ref>, positive spikes in tweet sentiment have found when CDC put emphasize on following their guideline from the beginning and mostly on May, importance of contact tracing at the beginning of April, reducing the spread of the virus as well as the pandemic stress. However, negative sentiments are observed when elderly people at risk; increased number of cases and hospitalization rates became more frequent in twitter.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>FEMA interactions</head><p>From around 2,000 tweets of FEMA, 10 optimum number of topics word probabilities over the timeline are visualized in Figure <ref type="figure">4</ref>.</p><p>FEMA discussed about the importance of critical information in April, need of COVID-19 responses in mid-March and also after 1 month (mid-April), spreading of the virus at the end of April, and need of supplies and food in April and May. Hurricane season also got attention of FEMA during the pandemic. However, FEMA emphasized on the importance of COVID-19 testing from April to May, which is not mentioned by WHO and CDC. Also, lack of medical supply got notified from end of the March, April, and May, which show after each month, medical supply in the United States is needed.</p><p>By interpreting Figures <ref type="figure">A5</ref> and<ref type="figure">A6</ref> together, the extreme negative sentiments are observed when the need of medical supplies, lack of food, and the emergence of upcoming hurricane season along with the increased spreading of the virus are discussed. Then, positive sentiments have found while the importance of COVID-19 testing, availability of responses, fund, and spreading of critical information got more importance.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>American Red Cross interactions</head><p>American Red Cross tweets showed eight optimum topics which are plotted over time in Figure <ref type="figure">5</ref>.</p><p>America Red Cross emphasized on taking care of sick people due to COVID-19 from the mid-March and showed willingness to provide more help to the COVID-19-affected people in May frequently, blood donation and the eligibility requirements to donate blood from the US Food and Drug Administration (FDA) in March. However, they also focused on the volunteer support consistently, importance of social distancing, and COVID-19 testing at the end of May. The most unique topic they mentioned is the importance of convalescent plasma donation to cure the COVID-19-infected patients throughout the timeline.</p><p>By comparing Figures <ref type="figure">A7</ref> and<ref type="figure">A8</ref>, spikes in positive sentiments are found when American Red Cross put importance of taking care of sick people due to COVID-19, exhibited willingness to provide more help to the affected people, and discussed the importance of social distancing to slow down the spread of nCoV and the effectiveness of convalescent plasma to cure the COVID-19-infected patients. On the other hand, negative sentiments have observed while the eligibility requirements to donate blood from the FDA and the lack of volunteer support topics became more frequent in Twitter.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>New York City Department of Health and Mental Hygiene interactions</head><p>From the NYC Department of Health tweets, eight optimum number of topics are visualized over the three-and half-month timeline in Figure <ref type="figure">6</ref>.</p><p>They focused on pandemic stress reduction and the spreading of the nCoV consistently throughout the timeline due to the outbreak and were concerned about the health risk and COVID-19 responses at the beginning of March. However, they also emphasized on staying home and symptoms of COVID-19 at the end of March and provided health tips consistently on twitter during the mentioned timeline.</p><p>After comparing Figures <ref type="figure">A9</ref> and<ref type="figure">A10</ref>, positive sentiments of tweets have been found when NYC Department of Health emphasized on the importance of staying home to slow down the spread of coronavirus, specified the symptoms of the disease, provided the health tips for fighting the pandemic, and was concerned about the COVID-19 responses. However, negative sentiments are observed when the spread of nCoV, concerns about health risk, and stress induced due to the outbreak discussed more recurrently in twitter.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Mount Sinai Health System interactions</head><p>From more than 2,000 tweets of Mount Sinai Health System, 11 optimum number of topics frequency are visualized in Figure <ref type="figure">A11</ref>. Mount Sinai discussed about the need of frontline support from April, medical care and a surge of cancer patients in March, and the importance of contact tracing at the end of February. They also emphasized on inflammatory bowel disease (IBD) antibodies consistently during the timeline, workers' health and patient care from April, symptoms and hygiene aspects of COVID-19 such as hand washing with soap for at least 20 seconds in March, and the importance of social distancing in March and April. The most unique topic discussed by Mount Sinai is the role of healthcare podcast to share health information. However, some of the hashtags such as #mountsinaistrong and #fitforthefrontlines also became popular. <ref type="figure">A11</ref> and<ref type="figure">A12</ref> together, the negative sentiments are observed when the surge of cancer patients is reported, and the lack of social distancing as well as patient care was emphasized. Discussion on need of hygiene practice also resulted in negative sentiment in twitter. Then, positive sentiments are found while importance of frontline support, workers' health, IBD antibodies, and the role of healthcare podcast to share health information became more prominent. In Figure <ref type="figure">7</ref>, the word probabilities over the time of each topic are replicated. In order to identify the specific interaction patters along with the sentiments that might influence the changes (increase and decrease) in the number of cases and deaths in the United States, the topic dynamics and sentiments over time are represented along with the number of cases and deaths of COVID-19 from Worldometer data <ref type="bibr">31</ref> in Figures <ref type="figure">8</ref> and<ref type="figure">9</ref>.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>By interpreting Figures</head><p>According to these figures, it can be observed that the document's topic frequency is strengthened at the peak of COVID-19 cases. Furthermore, "willingness to help" is at its highest value after approximately 1 month after the peak of the cases. This may show societal desire to help each other after understanding the consequences of the COVID-19 pandemic. Another interesting observation is the peak in COVID-19 testing's document's topic frequency and the peaks in the numbers of reported cases.</p><p>From both Figures <ref type="figure">8</ref> and<ref type="figure">9</ref>, it is obvious that the number of cases and deaths started to increase rapidly from March 20, 2020 and hit the first pick (around 40,000 cases and 2,500 deaths per day) around April 10, 2020. In this time duration, WHO discussed more about the lack of social measures, responses, requirement of adequate number of nurses, and importance of wearing face masks; CDC emphasized on need of contact tracing, failure of preventing the spread, and suggested to follow their guidelines; FEMA mentioned about shortage of medical supplies, responses, and the importance of critical information dissemination.</p><p>However, American Red Cross stated about the care of sick people due to COVID-19, volunteer support, blood donation, and the requirement from FDA to donate blood; NYC Department of Health emphasized on health risks, responses, providing health tips, symptoms of COVID-19, importance of home quarantine, and Mount Sinai Health System focused on patient care, hygiene activities, contact tracing, and healthcare podcast during the mentioned timeline. These figures also represent the similar fluctuating patterns in all these organizations over time as well as the number of COVID-19 new cases and deaths. We conclude that society's sentiment in response to the verified news has a short-term component, which is highly volatile and changes with daily news and updates about an infectious disease, in this case, COVID-19-related news.</p><p>Then, the number of cases and deaths remained nearly constant from April 11, 2020 to May 10, 2020. In this time, importance of social distancing, increase in funding, response and support from government, essence of pandemic stress management, new measures to prevent the spread, organizations' willingness to help more, frontline support, and the requirement of PPE have started becoming into the focus of the discussion. The probable influence of emphasizing on these issues is obvious from the last portion (May 11, 2020 to June 6, 2020) of the graphs (Figures <ref type="figure">8</ref> and<ref type="figure">9</ref>), where both the number of cases and deaths decreased. In this timeline, increased response from people and government, awareness about the virus spreading, lowering hospitalization rates, following health guidance, understanding the risk of elderly people, increased medical supply, importance of convalescent plasma donation, medical care, social distancing, working together, pandemic stress management programs, and focusing on workers' health issues also might have influenced human behavior to reduce the number of cases and deaths resulting in flattening the curve. The text-based infographics showed in previous sections have revealed different interaction patterns along with the sentiments from the four major public agencies during the pandemic. These interactions were generated from most frequently appeared words in different topics from these entities which are listed later in different timeline. The entire time is divided into four subperiod based on the lockdown (mid-March 2020) and reopening (beginning of the May 2020) time in the United States.</p><p>After analyzing all the results and infographics, the noteworthy findings from each organization are listed as follow: This study identified specific topics in Twitter that could have a probable influence on the change in the number of cases and deaths across the United States. Findings could help policy makers to take better preparation for the next phases of the ongoing pandemic as well as in future pandemic situations.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>CONCLUSIONS AND FUTURE RECOMMENDATIONS</head><p>Social media is considered as an effective information dissemination platform in recent past. The influence of social media on human behavior and decision making is remarkable. Although the spread of misinformation and information overload challenges the effectiveness of the information propagation through social media, <ref type="bibr">32</ref> it plays a vital role in crisis communication. Hence, exploring the major public agencies interactions over Twitter and interpreting the results along with crowdsourced data were the main focus of this study. From the collected tweets (around 11,000 tweets) of the six major public organizations, it was obvious that WHO remained most active in social media during the first three and half month of COVID-19 pandemic. Topic frequency distribution and sentiment analysis over time helped us to identify the positive and negative influence of different interaction patterns over the time.</p><p>The results showed that the importance of wearing face mask was ignored by most of the organizations, although WHO discussed about it in April, which may have been too late. The change of social distancing terminology to physical distancing over the time indicated the increase in understanding and awareness about COVID-19. However, shortage of nurses, medical supplies, contact tracing, and social distancing were the probable influential factors behind the sudden increase in both the number of cases and deaths in the United States. The importance of COVID-19 testing was only emphasized by FEMA along with the emergence of upcoming hurricane season. CDC focused on pandemic stress management and monitoring the hospitalization rates. American Red Cross put importance on blood donation, the eligibility requirement of FDA, and the treatment of COVID-19-infected patients with convalescent plasma. New York Department of Health and Mental Hygiene emphasized on the pandemic stress management and provided health tips to fight against COVID-19. Mount Sinai Health System focused about the frontline support, IBD antibodies, and the role of healthcare podcast for health information sharing.</p><p>The major contribution of this study is the demonstration of prominent discussions from different agencies through social media communication to a larger audience in a major crisis and how they may correlate with community response and societal outcome indicators, ie, growth in number of cases, lockdown activities, among others. Although such outcomes and response depend on several factors, the findings of this research could help the policy makers, government officials, and emergency management organizations to better prepare for the next phases of the ongoing COVID-19 as well as future pandemic. For example, this study identifies specific topics that could potentially influenced the increase and decrease in the number of cases around the globe, ie, information contagion. Decision makers as well as the people in the vulnerable areas could benefit from such information and data-driven approach to revisit social media communication practices to make the strenuous pandemic situation better.</p><p>One limitation of this study is in the dataset used being limited to 3,200 historical tweets per agencies based on Twitter Search API limitations. Future studies should utilize more advanced API to conduct more rigorous analysis of how users who follow these agencies reacted to a given information. However, all the machine learning and NLP models (DTM and sentiment analysis) used in this study are probabilistic model; future studies may consider interpretable machine learning models, ie, decision trees, <ref type="bibr">33</ref> neural network, <ref type="bibr">34</ref> and random forest, <ref type="bibr">35</ref> to predict the future trend of discussions in social media. Another emerging issue in social media is the spread of misinformation, <ref type="bibr">36</ref> which itself became a pandemic, where defining the misinformation <ref type="bibr">37</ref>  foremost task. The spread of misinformation also depends on the social network characteristics (followee-follower and user mention), <ref type="bibr">14,</ref><ref type="bibr">38,</ref><ref type="bibr">39</ref> which needs to be considered to track the misinformation propagation. However, the identification of the good and bad actors in social media can also contribute to tackle the misinformation problem.     </p></div></body>
		</text>
</TEI>
