More Like this

1. "Procurement As Policy: Administrative Process for Machine Learning."

   https://doi.org/10.2139/ssrn.3464203  

   Mulligan, Deirdre K. ( February 2020 , Berkeley technology law journal)

   null (Ed.)

   At every level of government, officials contract for technical systems that employ machine learning-systems that perform tasks without using explicit instructions, relying on patterns and inference instead. These systems frequently displace discretion previously exercised by policymakers or individual front-end government employees with an opaque logic that bears no resemblance to the reasoning processes of agency personnel. However, because agencies acquire these systems through government procurement processes, they and the public have little input into-or even knowledge about-their design or how well that design aligns with public goals and values. This Article explains the ways that the decisions about goals, values, risk, and certainty, along with the elimination of case-by-case discretion, inherent in machine-learning system design create policies-not just once when they are designed, but over time as they adapt and change. When the adoption of these systems is governed by procurement, the policies they embed receive little or no agency or outside expertise beyond that provided by the vendor. Design decisions are left to private third-party developers. There is no public participation, no reasoned deliberation, and no factual record, which abdicates Government responsibility for policymaking. This Article then argues for a move from a procurement mindset to policymaking mindset. When policy decisions are made through system design, processes suitable for substantive administrative determinations should be used: processes that foster deliberation reflecting both technocratic demands for reason and rationality informed by expertise, and democratic demands for public participation and political accountability. Specifically, the Article proposes administrative law as the framework to guide the adoption of machine learning governance, describing specific ways that the policy choices embedded in machine learning system design fail the prohibition against arbitrary and capricious agency actions absent a reasoned decision-making process that both enlists the expertise necessary for reasoned deliberation about, and justification for, such choices, and makes visible the political choices being made. Finally, this Article sketches models for machine-learning adoption processes that satisfy the prohibition against arbitrary and capricious agency actions. It explores processes by which agencies might garner technical expertise and overcome problems of system opacity, satisfying administrative law's technocratic demand for reasoned expert deliberation. It further proposes both institutional and engineering design solutions to the challenge of policymaking opacity, offering process paradigms to ensure the "political visibility" required for public input and political oversight. In doing so, it also proposes the importance of using "contestable design"-design that exposes value-laden features and parameters and provides for iterative human involvement in system evolution and deployment. Together, these institutional and design approaches further both administrative law's technocratic and democratic mandates. 

   more » « less
2. Cogenerative Dialogues and Development of the Culturally Relevant Pedagogical Guidelines for Computational Thinking and Computer Science.

   Sirrakos, G. ( January 2023 , American Association for the Advancement of Curriculum Studies Annual Meeting)

   In this proposal, we will share some initial findings about how teacher and student engagement in cogenerative dialogues influenced the development of the Culturally Relevant Pedagogical Guidelines for Computational Thinking and Computer Science (CRPG-CSCT). The CRPG-CSCT’s purpose is to provide computer science teachers with tools to enhance their instruction by accurately reflecting students’ diverse cultural resources in the classroom. Additionally, the CRPG-CSCT will provide guidance to non-computer science teachers on how to facilitate the integration of computational thinking skills to a broad spectrum of classes in the arts, humanities, sciences, social sciences, and mathematics. Our initial findings shared here are part of a larger NSF-funded research project (Award No. 2122367) which aims to better understand the barriers to entry and challenges for success faced by underrepresented secondary school students in computer science, through direct engagement with the students themselves. Throughout the 2022-23 academic year, the researchers have been working with a small team of secondary school teachers, students, and instructional designers, as well as university faculty in computer science, secondary education, and sociology to develop the CRPG-CSCT. The CRPG-CSCT is rooted in the tenets of culturally relevant pedagogy (Ladson-Billings, 1995) and borrows from Muhammad’s (2020) work in Cultivating Genius: An Equity Framework for Culturally and Historically Responsive Literacy. The CRPG-CCT is being developed over six day-long workshops held throughout the academic year. At the time of this submission, five of the six workshops had been completed. Each workshop utilized cogenerative dialogues (cogens) as the primary tool for organizing and sustaining participants’ engagement. Through cogens, participants more deeply learn about students’ cultural capital and the value of utilizing that capital within the classroom (Roth, Lawless, & Tobin, 2000). The success of cogens relies on following specific protocols (Emdin, 2016), such as listening attentively, ensuring there are equal opportunities for all participants to share, and affirming the experiences of other participants. The goal of a cogen is to reach a collective decision, based on the dialogue, that will positively impact students by explicitly addressing barriers to their engagement in the classroom. During each workshop, one member of the research team and one undergraduate research assistant observed the interactions among cogen participants and documented these in the form of ethnographic field notes. Another undergraduate research assistant took detailed notes during the workshop to record the content of small and large group discussions, presentations, and questions/responses throughout the workshops. A grounded theory approach was used to analyze the field notes. Additionally, at the conclusion of each workshop, participants completed a Cogen Feedback Survey (CFS) to gather additional information. The CFS were analyzed through open thematic coding, memos, and code frequencies. Our preliminary results demonstrate high levels of engagement from teacher and student participants during the workshops. Students identified that the cogen structure allowed them to participate comfortably, openly, and honestly. Further, students described feeling valued and heard. Students’ ideas and experiences were frequently affirmed, which served as an important step toward dismantling traditional teacher-student boundaries that might otherwise prevent them from sharing freely. Another result from the use of cogens was the shared experience of participants comprehending views from the other group’s perspective in the classroom. Students appreciated the opportunity to learn from teachers about their struggles in keeping students engaged. Teachers appreciated the opportunity to better understand students’ schooling experiences and how these may affirm or deny aspects of their identity. Finally, all participants shared meaningful suggestions and strategies for future workshops and for the collective betterment of the group. Initial findings shared here are important for several reasons. First, our findings suggest that cogens are an effective approach for fostering participants’ commitment to creating the conditions for students’ success in the classroom. Within the context of the workshops, cogens provided teachers, students, and faculty with opportunities to engage in authentic conversations for addressing the recruitment and retention problems in computer science for underrepresented students. These conversations often resulted in the development of tangible pedagogical approaches, examples, metaphors, and other strategies to directly address the recruitment and retention of underrepresented students in computer science. Finally, while we are still developing the CRPG-CSCT, cogens provided us with the opportunity to ensure the voices of teachers and students are well represented in and central to the document. 

   more » « less
3. Ethical Mediation in UX Practice

   https://doi.org/10.1145/3290605.3300408  

   Gray, Colin M. ; Chivukula, Shruthi Sai ( May 2019 , Proceedings of the 2019 CHI Conference on Human Factors in Computing Systems - CHI '19)

   HCI scholars have become increasingly interested in describing the complex nature of UX practice. In parallel, HCI and STS scholars have sought to describe the ethical and value-laden relationship between designers and design outcomes. However, little research describes the ethical engagement of UX practitioners as a form of design complexity, including the multiple mediating factors that impact ethical awareness and decision-making. In this paper, we use a practice-led approach to describe ethical complexity, presenting three varied cases of UX practitioners based onin situ observations and interviews. In each case, we describe salient factors relating to ethical mediation, including organizational practices, self-driven ethical principles, and unique characteristics of specific projects the practitioner is engaged in. Using the concept of mediation from activity theory, we provide a rich account of practitioners' ethical decision making. We propose future work on ethical awareness and design education based on the concept of ethical mediation. 

   more » « less
4. Activating social change together: A qualitative synthesis of collaborative change research, evaluation and design literature

   https://doi.org/http:dx.doi.org/10.5130/ijcre.v12i1.6693  

   Busch, Melida D ; Jean-Baptiste, Elizabeth ; Person, Pamela F ; Vaughn, Lisa M. ( December 2019 , Gateways)

   Researchers, evaluators and designers from an array of academic disciplines and industry sectors are turning to participatory approaches as they seek to understand and address complex social problems. We refer to participatory approaches that collaboratively engage/ partner with stakeholders in knowledge creation/problem solving for action/social change outcomes as collaborative change research, evaluation and design (CCRED). We further frame CCRED practitioners by their desire to move beyond knowledge creation for its own sake to implementation of new knowledge as a tool for social change. In March and May of 2018, we conducted a literature search of multiple discipline-specific databases seeking collaborative, change-oriented scholarly publications. The search was limited to include peerreviewed journal articles, with English language abstracts available, published in the last five years. The search resulted in 526 citations, 236 of which met inclusion criteria. Though the search was limited to English abstracts, all major geographic regions (North America, Europe, Latin America/Caribbean, APAC, Africa and the Middle East) were represented within the results, although many articles did not state a specific region. Of those identified, most studies were located in North America, with the Middle East having only one identified study. We followed a qualitative thematic synthesis process to examine the abstracts of peer-reviewed articles to identify practices that transcend individual disciplines, sectors and contexts to achieve collaborative change. We surveyed the terminology used to describe CCRED, setting, content/topic of study, type of collaboration, and related benefits/outcomes in order to discern the words used to designate collaboration, the frameworks, tools and methods employed, and the presence of action, evaluation or outcomes. Forty-three percent of the reviewed articles fell broadly within the social sciences, followed by 26 percent in education and 25 percent in health/medicine. In terms of participants and/ or collaborators in the articles reviewed, the vast majority of the 236 articles (86%) described participants, that is, those who the research was about or from whom data was collected. In contrast to participants, partners/collaborators (n=32; 14%) were individuals or groups who participated in the design or implementation of the collaborative change effort described. In terms of the goal for collaboration and/or for doing the work, the most frequently used terminology related to some aspect of engagement and empowerment. Common descriptors for the work itself were ‘social change’ (n=74; 31%), ‘action’ (n=33; 14%), ‘collaborative or participatory research/practice’ (n=13; 6%), ‘transformation’ (n=13; 6%) and ‘community engagement’ (n=10; 4%). Of the 236 articles that mentioned a specific framework or approach, the three most common were some variation of Participatory Action Research (n=30; 50%), Action Research (n=40; 16.9%) or Community-Based Participatory Research (n=17; 7.2%). Approximately a third of the 236 articles did not mention a specific method or tool in the abstract. The most commonly cited method/tool (n=30; 12.7%) was some variation of an arts-based method followed by interviews (n=18; 7.6%), case study (n=16; 6.7%), or an ethnographic-related method (n=14; 5.9%). While some articles implied action or change, only 14 of the 236 articles (6%) stated a specific action or outcome. Most often, the changes described were: the creation or modification of a model, method, process, framework or protocol (n=9; 4%), quality improvement, policy change and social change (n=8; 3%), or modifications to education/training methods and materials (n=5; 2%). The infrequent use of collaboration as a descriptor of partner engagement, coupled with few reported findings of measurable change, raises questions about the nature of CCRED. It appears that conducting CCRED is as complex an undertaking as the problems that the work is attempting to address. 

   more » « less
5. WIP Toward a Free-Body Diagram Mobile Application

   Sloboda, A. R. ; Wodin-Schwartz, S. ; Lechasseur, K. ; deWinter, J. ( June 2023 , ASEE annual conference exposition proceedings)

   No skill is more important for a student of mechanics than the ability to draw a complete and accurate free-body diagram (FBD). A good FBD facilitates proper accounting of forces when writing the balances that lead to governing equations in statics, solid mechanics, and dynamics. Because this skill is essential, educational approaches that improve the ability of students to draw correct FBDs are critical for maximizing the potential of the next generation of engineers. Traditionally, learning to draw FBDs involves classroom instruction followed by homework practice consisting of problems drawn from a textbook. Homework as practice does not serve all students well, because it does not scaffold the process of drawing FBDs in terms of distinct tasks (e.g., isolating the body, considering support reactions) nor does it offer immediate feedback, which students often need to avoid falling into the same error repeatedly. To address these shortcomings, we embarked on the design, implementation, and testing of a mobile application (app) that offers an alternative venue for FBD practice. The app provides students with asynchronous opportunities for training, varied tasks that target specific FBD issues, and several levels of immediate feedback. We hypothesize that the gamified environment and puzzle-based gameplay will improve student skill and self-efficacy in drawing FBDs, particularly for women, who may feel less confident in their spatial skills. Data collected to describe student experiences may also provide additional insight into how to improve FBD instruction generally. In this paper, we detail the process for designing and implementing the app and provide initial data regarding student impressions and use. The app was piloted in Fall 2022 in a large Introduction to Statics course as a non-graded study activity; all students except one (n=97) participated in an evaluation of its design features and user experiences. Approximately half (54%) of students indicated they had played half or more of the available games. When commenting about how the FBD app did, or did not, help their learning, 49% of respondents appreciated that the app allowed additional opportunities for practice. Students used these opportunities to further develop several skills, such as visualizing the system and setting up accurate diagrams, which strengthened their confidence and reviewed key concepts. While describing the value of practicing through the app, 21% of students called out how the app provided feedback. They specifically mentioned the positive experiences of receiving feedback that is immediate, that explains boundary connections, and that deepens learning after mistakes are made. These and other findings from the pilot study are discussed with corresponding next steps for development. 

   more » « less