Search for: All records

In addressing various risks on social media, the HCI community has advocated for teen-centered risk detection technologies over platform-based, parent-centered features. However, their real-world viability remains underexplored by secondary stakeholders beyond the family unit. Therefore, we present an evaluation of a teencentered social media risk detection dashboard through online interviews with 33 online safety experts. While experts praised our dashboard’s clear design for teen agency, their feedback revealed five primary tensions in implementing and sustaining such technology: objective vs. context-dependent risk definition, informing risks vs. meaningful intervention, teen empowerment vs. motivation, need for data vs. data privacy, and independence vs. sustainability. These findings motivate us to rethink “teen-centered” and a shift from a “fail fast” to a “mature safely” paradigm for youth safety technology innovation.We offer design implications for addressing these tensions before system deployment with teens and strategies for aligning secondary stakeholders’ interests to deploy and sustain such technologies in the broader ecosystem of youth online safety. 

Although social media use is ubiquitous among adolescents, research often emphasizes benefits and risks without considering the design-based affordances that may influence these experiences. Affordances enable or constrain user behavior and thus provide a valuable lens for promoting positive online engagement. To this end, we developed an affordance-centered framework and systematically coded 66 empirical studies on U.S. adolescents published in the past decade to map how platform features relate to specific user activities, the affordances they instantiate, and associated benefits and risks. For instance, Discoverability affordance, driven by algorithmic recommendations, supported Cognitive engagement by facilitating learning while heightening exposure to misinformation. Visibility affordance enabled Identity exploration through ephemeral sharing, lowering performance anxiety but raising post-disclosure regret when content reached hostile viewers. By clarifying how specific affordances facilitate adolescent experiences, our work contributes a design-grounded framework for future research that advances both theoretical understanding and the development of youth-centered interventions. 

BackgroundThis study aims to detect self-harm or suicide (SH-S) ideation language used by youth (aged 13-21 y) in their private Instagram (Meta) conversations. While automated mental health tools have shown promise, there remains a gap in understanding how nuanced youth language around SH-S can be effectively identified. ObjectiveOur work aimed to develop interpretable models that go beyond binary classification to recognize the spectrum of SH-S expressions. MethodsWe analyzed a dataset of Instagram private conversations donated by youth. A range of traditional machine learning models (support vector machine, random forest, Naive Bayes, and extreme gradient boosting) and transformer-based architectures (Bidirectional Encoder Representations from Transformers and Distilled Bidirectional Encoder Representations from Transformers) were trained and evaluated. In addition to raw text, we incorporated contextual, psycholinguistic (linguistic injury word count), sentiment (Valence Aware Dictionary and Sentiment Reasoner), and lexical (term frequency–inverse document frequency) features to improve detection accuracy. We further explored how increasing conversational context—from message-level to subconversation level—affected model performance. ResultsDistilled Bidirectional Encoder Representations from Transformers demonstrated a good performance in identifying the presence of SH-S behaviors within individual messages, achieving an accuracy of 99%. However, when tasked with a more fine-grained classification—differentiating among “self” (personal accounts of SH-S), “other” (references to SH-S experiences involving others), and “hyperbole” (sarcastic, humorous, or exaggerated mentions not indicative of genuine risk)—the model’s accuracy declined to 89%. Notably, by expanding the input window to include a broader conversational context, the model’s performance on these granular categories improved to 91%, highlighting the importance of contextual understanding when distinguishing between subtle variations in SH-S discourse. ConclusionsOur findings underscore the importance of designing SH-S automatic detection systems sensitive to the dynamic language of youth and social media. Contextual and sentiment-aware models improve detection and provide a nuanced understanding of SH-S risk expression. This research lays the foundation for developing inclusive and ethically grounded interventions, while also calling for future work to validate these models across platforms and populations. 

Youth, while tech-savvy and highly active on social media, are still vulnerable to online privacy and security risks. Therefore, it is critical to understand how they negotiate and manage social connections versus protecting themselves in online contexts. In this work, we conducted a thematic analysis of 1,318 private conversations on Instagram from 149 youth aged 13-21 to understand the digital privacy and security topics they discussed, if and how they engaged in risky privacy behaviors, and how they balanced the benefits and risks (i.e., privacy calculus) of making these decisions. Overall, youth were forthcoming when broaching a wide range of topics on digital privacy and security, ranging from password management and account access challenges to shared experiences of being victims of privacy risks. However, they also openly engaged in risky behaviors, such as sharing personal account information with peers and even perpetrating privacy and security risks against others. Nonetheless, we found many of these behaviors could be explained by the unique ''privacy calculus'' of youth, where they often prioritized social benefits over potential risks; for instance, youth often shared account credentials with peers to foster social connection and affirmation. As such, we provide a nuanced understanding of youth decision-making regarding digital security and privacy, highlighting both positive behaviors, tensions, and points of concern. We encourage future research to continue to challenge the potentially untrue narratives regarding youth and their digital privacy and security to unpack the nuance of their privacy calculus that may differ from that of adults. 

Academic research is largely an adult endeavor that creates systemic power imbalances when studying teen-centered topics, such as adolescent online safety. To rectify this problem, we engaged seven teens as co-researchers through a year-and-a-half-long Youth Advisory Board (YAB) program to critically assess our research processes, lead online safety solutions, and to reflect on their experiences participating in a YAB. Teens pushed back on standard research practices such as parental consent, sought decision-making power in study documentation, design, and execution, and gave more meaningful feedback on research protocols when more deeply involved in the research. For safety interventions, teens proposed both incremental changes for social media platforms (e.g., advanced privacy settings) and more disruptive changes (e.g., decentralized social media platforms) that enhance individual control, digital resilience, and equity. For the YAB, teens highlighted challenges, such as losing momentum over time, lack of collaborative opportunities, and competing interests, fueling frustrations and rifts in engagement. Our research underscores the value of involving teens as co-partners in shaping online safety research. Finally, we provide design implications for social media safety interventions that strengthen teens' agency and actionable guidelines for developing future long-term programs to ensure meaningful contributions to online safety research. 

We analyzed 1,596 sub-conversations within 451 direct message (DM) conversations from 67 teens (ages 13-17) who engaged in private discussions about body image on Instagram. Our findings show that teens often receive support when sharing struggles with negative body image, participate in criticism when engaging in body-shaming, and are met with appreciation when promoting positive body image. Additionally, these types of disclosures and responses varied based on whether the conversations were one-on-one or group-based. We found that sharing struggles and receiving support most often occurred in one-on-one conversations, while body shaming and negative interactions often occurred in group settings. A key insight of the study is that private social media settings can significantly influence how teens discuss and respond to body image issues. Based on these findings, we propose design guidelines for social media platforms that can foster positive interactions around body image, ultimately creating a healthier and more supportive online environment for teens struggling with body image concerns. 

Ensuring the online safety of youth has motivated research towards the development of machine learning (ML) methods capable of accurately detecting social media risks after-the-fact. However, for these detection models to be effective, they must proactively identify high-risk scenarios (e.g., sexual solicitations, cyberbullying) to mitigate harm. This `real-time' responsiveness is a recognized challenge within the risk detection literature. Therefore, this paper presents a novel two-level framework that first uses reinforcement learning to identify conversation stop points to prioritize messages for evaluation. Then, we optimize state-of-the-art deep learning models to accurately categorize risk priority (low, high). We apply this framework to a time-based simulation using a rich dataset of 23K private conversations with over 7 million messages donated by 194 youth (ages 13-21). We conducted an experiment comparing our new approach to a traditional conversation-level baseline. We found that the timeliness of conversations significantly improved from over 2 hours to approximately 16 minutes with only a slight reduction in accuracy (0.88 to 0.84). This study advances real-time detection approaches for social media data and provides a benchmark for future training reinforcement learning that prioritizes the timeliness of classifying high-risk conversations. 

On social media, teens must manage their interpersonal boundaries not only with other people, but also with the algorithms embedded in these platforms. In this context, we engaged seven teens in an Asynchronous Remote Community (ARC) as part of a multi-year Youth Advisory Board (YAB) to discuss how they navigate, cope, and co-design for improved boundary management. Teens had preconceived notions of different platforms and navigated boundaries based on specific goals; yet, they struggled when platforms lacked the granular controls needed to meet their needs. Teens enjoyed the personalization afforded by algorithms, but they felt violated when algorithms pushed unwanted content. Teens designed features for enhanced control over their discoverability and for real-time risk detection to avoid boundary turbulence. We provide design guidelines for improved social media boundary management for youth and pinpoint educational opportunities to enhance teens’ understanding and use of social media privacy settings and algorithms. 

The SIGCHI and Social Computing research communities have been at the forefront of online safety efforts for youth, ranging from understanding the serious risks youth face online to developing evidence-based interventions for risk protection. Yet, to bring these efforts to bear, we must partner with practitioners, such as industry stakeholders who know how to bring such technologies to market, and youth service providers who work directly with youth. Therefore, we interviewed 33 stakeholders in the space of youth online safety, including industry professionals (n=12), youth service providers (n=11), and researchers (n=10) to understand where their visions toward working together to protect youth online converged and surfaced tensions, as well as how we might reconcile conflicting viewpoints to move forward as one community with synergistic expertise on how to change the current sociotechnical landscape for youth online safety. Overall, we found that non-partisan leadership is necessary to chart actionable, equitable goals to facilitate collaboration between stakeholders, combat feelings of isolation, and foster trust between the stakeholder groups. Based on these findings, we recommend the use of open-innovation methods with their inherent transparency, federated governance models, and clear but inclusive leadership structures to promote collaboration between youth online safety stakeholders. We propose the creation of an open-innovation organization that unifies the diverse voices in youth online safety to develop open-standards and evidence-based design patterns that centralize otherwise fragmented efforts that have fallen short of the goal of effective technological solutions that keep youth safe online.