skip to main content


Title: Cataloging Algorithmic Decision Making in the U.S. Government
Government use of algorithmic decision-making (ADM) systems is widespread and diverse, and holding these increasingly high-impact, often opaque government algorithms accountable presents a number of challenges. Some European governments have launched registries of ADM systems used in public services, and some transparency initiatives exist for algorithms in specific areas of the United States government; however, the U.S. lacks an overarching registry that catalogs algorithms in use for public-service delivery throughout the government. This paper conducts an inductive thematic analysis of over 700 government ADM systems cataloged by the Algorithm Tips database in an effort to describe the various ways government algorithms might be understood and inform downstream uses of such an algorithmic catalog. We describe the challenge of government algorithm accountability, the Algorithm Tips database and method for conducting a thematic analysis, and the themes of topics and issues, levels of sophistication, interfaces, and utilities of U.S. government algorithms that emerge. Through these themes, we contribute several different descriptions of government algorithm use across the U.S. and at federal, state, and local levels which can inform stakeholders such as journalists, members of civil society, or government policymakers  more » « less
Award ID(s):
1845460
NSF-PAR ID:
10386490
Author(s) / Creator(s):
; ; ;
Date Published:
Journal Name:
Computation + Journalism Symposium
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Winkler, Inga T. (Ed.)
    Access to clean and safe water is essential for human health and well-being, but recent and substantial increases in the cost of water for residential customers in the United States endanger the health of those who cannot afford to pay. This study identifies pathways through which unaffordable water bills may influence the behaviors and health of vulnerable people. We interviewed a sample of low-income residential water customers who were experiencing water bill hardship in Boston and Chelsea, Massachusetts, U.S. between October 2018 and December 2019. We conducted a thematic content analysis of interview transcripts. Results showed that some participants improvised ways to pay their water bills, and some confronted obstacles that made it simply impossible to pay at times. Behavioral responses to coping with high water bills were influenced by household earning potential, self-reported health status, caretaking responsibilities, and accessibility of utility assistance programs. Consequences of unaffordable household bills included reduced access to other necessities, debt accumulation, risk of water shutoff, housing insecurity, and public humiliation. Reported health-related impacts of water bill hardship were food insecurity, underutilization of healthcare and medications, and decline in mental health. Comprehensive reforms at all levels of government are needed to make water affordable for all low-income households. Federal investments in water infrastructure, state oversight of affordability and human rights, as well as municipal tiered water pricing and comprehensive assistance policies for low-income households are needed to address the growing water affordability crisis and to mitigate harm to the well-being of vulnerable residents and communities in the United States. 
    more » « less
  2. null (Ed.)
    Cost and schedule overruns are common in the procurement of large-scale defense acquisition programs. Current work focuses on identifying the root causes of cost growth and schedule delays in the defense acquisition programs. There is need for a mix of quantitative and qualitative analysis of cost and schedule overruns which takes into account program factor such as, technology maturity, design maturity, initial acquisition time, and program complexity. Such analysis requires an easy to access database for program-specific data about how an acquisition programs’ technical and financial characteristics vary over the time. To fulfill this need, the objective of this paper is twofold: (i) to develop a database of major US defense weapons programs which includes details of the technical and financial characteristics and how they vary over time, and (ii) to test various hypotheses about the interdependence of such characteristics using the collected data. To achieve the objective, we use a mixed-method analysis on schedule and cost growth data available in the U.S. Government Accountability Office's (GAO's) defense acquisitions annual assessments during the period 2003-2017. We extracted both analytical and textual data from original reports into Excel files and further created an easy to access database accessible from a Python environment. The analysis reveals that technology immaturity is the major driver of cost and schedule growth during the early stages of the acquisition programs while technical inefficiencies drive cost overruns and schedule delays during the later stages. Further, we find that the acquisition programs with longer initial length do not necessarily have higher greater cost growth. The dataset and the results provide a useful starting point for the research community for modeling cost and schedule overruns, and for practitioners to inform their systems acquisition processes. 
    more » « less
  3. While resilience in emergency management has been studied at the macro- (government) and micro-levels (individual field responder), little is known for resilience of incident management teams (IMTs). To investigate challenges and resilience factors of IMTs, this paper documents thematic analysis of 10 interviews with emergency personnel who responded to Hurricane Harvey in 2017. Themes emerged in four categories: goals, challenges, resilience factors, and technical tools of IMTs. Given similar goals but unique challenges during Harvey, IMTs sought to establish and maintain a common operating picture to make sense of evolving situations and make decisions adaptively. Various technical tools were used providing different functionalities, but a need for technology to reduce cognitive load was indicated. Findings of this study will inform the development of more resilient IMTs in future disasters. 
    more » « less
  4. Data too sensitive to be "open" for analysis and re-purposing typically remains "closed" as proprietary information. This dichotomy undermines efforts to make algorithmic systems more fair, transparent, and accountable. Access to proprietary data in particular is needed by government agencies to enforce policy, researchers to evaluate methods, and the public to hold agencies accountable; all of these needs must be met while preserving individual privacy and firm competitiveness. In this paper, we describe an integrated legal-technical approach provided by a third-party public-private data trust designed to balance these competing interests. Basic membership allows firms and agencies to enable low-risk access to data for compliance reporting and core methods research, while modular data sharing agreements support a wide array of projects and use cases. Unless specifically stated otherwise in an agreement, all data access is initially provided to end users through customized synthetic datasets that offer a) strong privacy guarantees, b) removal of signals that could expose competitive advantage, and c) removal of biases that could reinforce discriminatory policies, all while maintaining fidelity to the original data. We find that using synthetic data in conjunction with strong legal protections over raw data strikes a balance between transparency, proprietorship, privacy, and research objectives. This legal-technical framework can form the basis for data trusts in a variety of contexts. 
    more » « less
  5. We conducted an observational study to document student misconceptions about cybersecurity using thematic analysis of 25 think-aloud interviews. By understanding patterns in student misconceptions, we provide a basis for developing rigorous evidence-based recommendations for improving teaching and assessment methods in cybersecurity and inform future research. This study is the first to explore student cognition and reasoning about cybersecurity. We interviewed students from three diverse institutions. During these interviews, students grappled with security scenarios designed to probe their understanding of cybersecurity, especially adversarial thinking. We analyzed student statements using a structured qualitative method, novice-led paired thematic analysis, to document patterns in student misconceptions and problematic reasoning that transcend institutions, scenarios, or demographics. Themes generated from this analysis describe a taxonomy of misconceptions but not their causes or remedies. Four themes emerged: overgeneralizations, conflated concepts, biases, and incorrect assumptions. Together, these themes reveal that students generally failed to grasp the complexity and subtlety of possible vulnerabilities, threats, risks, and mitigations, suggesting a need for instructional methods that engage students in reasoning about complex scenarios with an adversarial mindset. These findings can guide teachers’ attention during instruction and inform the development of cybersecurity assessment tools that enable cross-institutional assessments that measure the effectiveness of pedagogies. 
    more » « less