- Award ID(s):
- 1750038
- PAR ID:
- 10413466
- Date Published:
- Journal Name:
- 2022 IEEE 30th International Requirements Engineering Conference Workshops (REW)
- Page Range / eLocation ID:
- 130 to 135
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
Phishing and disinformation are popular social engineering attacks with attackers invariably applying influence cues in texts to make them more appealing to users. We introduce Lumen, a learning-based framework that exposes influence cues in text: (i) persuasion, (ii) framing, (iii) emotion, (iv) objectivity/subjectivity, (v) guilt/blame, and (vi) use of emphasis. Lumen was trained with a newly developed dataset of 3K texts comprised of disinformation, phishing, hyperpartisan news, and mainstream news. Evaluation of Lumen in comparison to other learning models showed that Lumen and LSTM presented the best F1-micro score, but Lumen yielded better interpretability. Our results highlight the promise of ML to expose influence cues in text, toward the goal of application in automatic labeling tools to improve the accuracy of human-based detection and reduce the likelihood of users falling for deceptive online content.more » « less
-
The advanced capabilities of Large Language Models (LLMs) have made them invaluable across various applications, from conversational agents and content creation to data analysis, research, and innovation. However, their effectiveness and accessibility also render them susceptible to abuse for generating malicious content, including phishing attacks. This study explores the potential of using four popular commercially available LLMs, i.e., ChatGPT (GPT 3.5 Turbo), GPT 4, Claude, and Bard, to generate functional phishing attacks using a series of malicious prompts. We discover that these LLMs can generate both phishing websites and emails that can convincingly imitate well-known brands and also deploy a range of evasive tactics that are used to elude detection mechanisms employed by anti-phishing systems. These attacks can be generated using unmodified or "vanilla" versions of these LLMs without requiring any prior adversarial exploits such as jailbreaking. We evaluate the performance of the LLMs towards generating these attacks and find that they can also be utilized to create malicious prompts that, in turn, can be fed back to the model to generate phishing scams - thus massively reducing the prompt-engineering effort required by attackers to scale these threats. As a countermeasure, we build a BERT-based automated detection tool that can be used for the early detection of malicious prompts to prevent LLMs from generating phishing content. Our model is transferable across all four commercial LLMs, attaining an average accuracy of 96% for phishing website prompts and 94% for phishing email prompts. We also disclose the vulnerabilities to the concerned LLMs, with Google acknowledging it as a severe issue. Our detection model is available for use at Hugging Face, as well as a ChatGPT Actions plugin.more » « less
-
Phishing websites remain a persistent security threat. Thus far, machine learning approaches appear to have the best potential as defenses. But, there are two main concerns with existing machine learning approaches for phishing detection. The first is the large number of training features used and the lack of validating arguments for these feature choices. The second concern is the type of datasets used in the literature that are inadvertently biased with respect to the features based on the website URL or content. To address these concerns, we put forward the intuition that the domain name of phishing websites is the tell-tale sign of phishing and holds the key to successful phishing detection. Accordingly, we design features that model the relationships, visual as well as statistical, of the domain name to the key elements of a phishing website, which are used to snare the end-users. The main value of our feature design is that, to bypass detection, an attacker will find it very difficult to tamper with the visual content of the phishing website without arousing the suspicion of the end user. Our feature set ensures that there is minimal or no bias with respect to a dataset. Our learning model trains with only seven features and achieves a true positive rate of 98% and a classification accuracy of 97%, on sample dataset. Compared to the state-of-the-art work, our per data instance classification is 4 times faster for legitimate websites and 10 times faster for phishing websites. Importantly, we demonstrate the shortcomings of using features based on URLs as they are likely to be biased towards specific datasets. We show the robustness of our learning algorithm by testing on unknown live phishing URLs and achieve a high detection accuracy of 99.7%.more » « less
-
Abstract Initial research on using crowdsourcing as a collaborative method for helping individuals identify phishing messages has shown promising results. However, the vast majority of crowdsourcing research has focussed on crowdsourced system components broadly and understanding individuals' motivation in contributing to crowdsourced systems. Little research has examined the features of crowdsourced systems that influence whether individuals utilise this information, particularly in the context of warnings for phishing emails. Thus, the present study examined four features related to warnings derived from a mock crowdsourced anti‐phishing warning system that 438 participants were provided to aid in their evaluation of a series of email messages: the number of times an email message was reported as being potentially suspicious, the source of the reports, the accuracy rate of the warnings (based on reports) and the disclosure of the accuracy rate. The results showed that crowdsourcing features work together to encourage warning acceptance and reduce anxiety. Accuracy rate demonstrated the most prominent effects on outcomes related to judgement accuracy, adherence to warning recommendations and anxiety with system use. The results are discussed regarding implications for organisations considering the design and implementation of crowdsourced phishing warning systems that facilitate accurate recommendations.
-
Background Inhibitory control training and working memory training are 2 cognitive interventions that have been considered for alcohol use disorder (
AUD ). Existing studies have typically relied on small samples that preclude the evaluation of small effects. Crowdsourcing is a sampling method that can address these limitations by effectively and efficiently recruiting large samples with varying health histories. This study tested the feasibility and acceptability of delivering cognitive training interventions via crowdsourcing.Methods Participants with
AUD were recruited from the crowdsourcing website Amazon Mechanical Turk (mT urk) (ClinicalTrials.gov;NCT 03438539). Following completion of a baseline survey, participants were randomized to an inhibitory control, working memory, or control training condition. Participants were asked to complete training tasks daily over a 2‐week period. Follow‐up assessments evaluating acceptability measures and alcohol and soda consumption were completed immediately following and 2 weeks after training.Results Response rates were satisfactory over the 2‐week intervention period (65% of training tasks completed), and performance on training tasks was consistent with expected effects. A majority of participants indicated that they were satisfied with the study procedures (94.6%), would participate again (97.4%), and would consider incorporating the training task in their daily life (81.1%). Modest reductions in alcohol consumption were observed (e.g., 0.5 drinking day/wk), primarily in the inhibitory control group, and these effects were selective to alcohol use and did not extend to soda consumption.
Conclusions These findings demonstrate the feasibility and acceptability of utilizing crowdsourcing methods for interventions development. Such a demonstration helps establish the crowdsourcing setting for future large sample studies testing novel interventions for
AUD and other substance use disorders.