More Like this

1. Much Ado About Gender: Current Practices and Future Recommendations for Appropriate Gender-Aware Information Access

   https://doi.org/10.1145/3576840.3578316  

   Pinney, Christine; Raj, Amifa; Hanna, Alex; Ekstrand, Michael D. (March 2023, CHIIR '23: Proceedings of the 2023 Conference on Human Information Interaction and Retrieval)

   Information access research (and development) sometimes makes use of gender, whether to report on the demographics of participants in a user study, as inputs to personalized results or recommendations, or to make systems gender-fair, amongst other purposes. This work makes a variety of assumptions about gender, however, that are not necessarily aligned with current understandings of what gender is, how it should be encoded, and how a gender variable should be ethically used. In this work, we present a systematic review of papers on information retrieval and recommender systems that mention gender in order to document how gender is currently being used in this field. We find that most papers mentioning gender do not use an explicit gender variable, but most of those that do either focus on contextualizing results of model performance, personalizing a system based on assumptions of user gender, or auditing a model’s behavior for fairness or other privacy-related issues. Moreover, most of the papers we review rely on a binary notion of gender, even if they acknowledge that gender cannot be split into two categories. We connect these findings with scholarship on gender theory and recent work on gender in human-computer interaction and natural language processing. We conclude by making recommendations for ethical and well-grounded use of gender in building and researching information access systems. 

   more » « less
2. 6th KidRec Workshop: Information Retrieval Systems for Children in the COVID-19 Era

   https://doi.org/10.1145/3501712.3536387  

   Pera, Maria Soledad; Landoni, Monica; Murgia, Emiliana; Huibers, Theo; Fails, Jerry Alan (June 2022, Interaction Design and Children)

   The role that technology plays in supporting children at school and at home is more prominent than ever before due to the global COVID-19 pandemic. This has prompted us to focus the 6th International and Interdisciplinary Perspectives on Children \& Recommender and Information Retrieval Systems (KidRec) workshop on what the lasting changes will be to the design and development of child information retrieval systems. After two years, are information retrieval systems used more in and out of the classroom? Are they more interactive, more or less personalized? What is the impact on the research and business community? Are there long-term and unexpected changes on the design, ethics, and algorithms? The primary goal of our workshop continues to be to build community by bringing together researchers, practitioners, and other stakeholders from various backgrounds and disciplines to understand and advance information retrieval systems for children. 

   more » « less
3. Distributionally-Informed Recommender System Evaluation

   https://doi.org/10.1145/3613455  

   Ekstrand, Michael D.; Carterette, Ben; Diaz, Fernando (August 2023, ACM Transactions on Recommender Systems)

   Current practice for evaluating recommender systems typically focuses on point estimates of user-oriented effectiveness metrics or business metrics, sometimes combined with additional metrics for considerations such as diversity and novelty. In this paper, we argue for the need for researchers and practitioners to attend more closely to various distributions that arise from a recommender system (or other information access system) and the sources of uncertainty that lead to these distributions. One immediate implication of our argument is that both researchers and practitioners must report and examine more thoroughly the distribution of utility between and within different stakeholder groups. However, distributions of various forms arise in many more aspects of the recommender systems experimental process, and distributional thinking has substantial ramifications for how we design, evaluate, and present recommender systems evaluation and research results. Leveraging and emphasizing distributions in the evaluation of recommender systems is a necessary step to ensure that the systems provide appropriate and equitably-distributed benefit to the people they affect. 

   more » « less
4. Deconstructing Conceptions of Rigor in Computer Science Education

   https://doi.org/10.1145/3776542  

   Everson, Jayne; Kivuva, F Megumi; Sherif, Eman; Oleson, Alannah; Ko, Amy (November 2025, ACM Transactions on Computing Education)

   ObjectivesIn calls for excellent and equitable Computer Science (CS) education, the wordrigoroften appears, but it often goes undefined. The goal of this work is to understand how CS teachers, instructors, and students conceive of rigor. Research Questions:1) What do CS instructors think rigor is? and 2) What do students think rigor is? Methods:Using the principles of phenomenological research, we conducted a semi-structured interview study with 10 post-secondary CS students, 10 secondary CS teachers, and 9 post-secondary CS instructors, to understand their conceptions of rigor. Results:Analysis showed that no participants had the same understanding of rigor. We found that participants had abstractPrinciples of Rigorwhich included: Precision, Systematic Thought Process, Depth of Understanding, and Challenge. They also had concreteObservations of Rigorthat included Time and Effort, Intrinsic Drive, Productive Failure, Struggle, Outcomes, and Gatekeeping. Participants also sharedConditions for Rigorwhich included Expectations, Standards, Community Support, and Resources. Implications:Our data supports prior work that educators are using different definitions of rigor. This implies that each educator holds different expectations for students, without necessarily communicating these expectations to their students. In the best case, this might confuse students; in the worst case, it reinforces hegemonic norms which can lead to gatekeeping which prevents students from fully participating in the CS field. Based on these insights, we argue that to commit to the idea of quality CS learning, the community must discard the use of this concept of rigor to justify student learning and re-imagine alternate benchmarks. 

   more » « less
5. Recommending With, Not For: Co-Designing Recommender Systems for Social Good

   https://doi.org/10.1145/3759261  

   Ekstrand, Michael D; Razi, Afsaneh; Sarcevic, Aleksandra; Pera, Maria Soledad; Burke, Robin; Wright, Katherine Landau (August 2025, ACM Transactions on Recommender Systems)

   Recommender systems are usually designed by engineers, researchers, designers, and other members of development teams. These systems are then evaluated based on goals set by the aforementioned teams and other business units of the platforms operating the recommender systems. This design approach emphasizes the designers’ vision for how the system can best serve the interests of users, providers, businesses, and other stakeholders. Although designers may be well-informed about user needs through user experience and market research, they are still the arbiters of the system’s design and evaluation, with other stakeholders’ interests less emphasized in user-centered design and evaluation. When extended to recommender systems for social good, this approach results in systems that reflect the social objectives as envisioned by the designers and evaluated as the designers understand them. Instead, social goals and operationalizations should be developed through participatory and democratic processes that are accountable to their stakeholders. We argue that recommender systems aimed at improving social good should be designedbyandwith, not justfor, the people who will experience their benefits and harms. That is, they should be designed in collaboration with their users, creators, and other stakeholders as full co-designers, not only as user study participants. 

   more » « less