Search for: All records

Systems such as “311” enable residents of a community to report on their environments and to request non-emergency municipal services. While such systems provide an important link between community and government, resident-generated data suffer from reporting bias, with some subpopulations reporting at lower rates than others. Our research focuses on defining the under-reporting of heating and hot water problems to New York City’s 311 system and developing methods to estimate under-reporting. First, we estimate non-reporting by fitting a latent variable model which estimates both the probability of an underlying heating problem conditional on building characteristics, and the probability of reporting a problem conditional on population characteristics. Second, we analyze “less-than-expected” reporting: buildings with fewer 311 calls than expected as compared to similarly-sized buildings with similar estimated problem durations. Together, these analyses determine neighborhoods and neighborhood-level socioeconomic characteristics that are predictive of under-reporting of heating and hot water problems. Our approaches can aid government agencies wishing to use resident-generated data to assist in constructing fair public policies. 

The field of fair machine learning aims to ensure that decisions guided by algorithms are equitable. Over the last decade, several formal, mathematical definitions of fairness have gained prominence. Here we first assemble and categorize these definitions into two broad families: (1) those that constrain the effects of decisions on disparities; and (2) those that constrain the effects of legally protected characteristics, like race and gender, on decisions. We then show, analytically and empirically, that both families of definitions typically result in strongly Pareto dominated decision policies. For example, in the case of college admissions, adhering to popular formal conceptions of fairness would simultaneously result in lower student-body diversity and a less academically prepared class, relative to what one could achieve by explicitly tailoring admissions policies to achieve desired outcomes. In this sense, requiring that these fairness definitions hold can, perversely, harm the very groups they were designed to protect. In contrast to axiomatic notions of fairness, we argue that the equitable design of algorithms requires grappling with their context-specific consequences, akin to the equitable design of policy. We conclude by listing several open challenges in fair machine learning and offering strategies to ensure algorithms are better aligned with policy goals. 

With an increased focus on incorporating fairness in machine learning models, it becomes imperative not only to assess and mitigate bias at each stage of the machine learning pipeline but also to understand the downstream impacts of bias across stages. Here we consider a general, but realistic, scenario in which a predictive model is learned from (potentially biased) training data, and model predictions are assessed post-hoc for fairness by some auditing method. We provide a theoretical analysis of how a specific form of data bias, differential sampling bias, propagates from the data stage to the prediction stage. Unlike prior work, we evaluate the downstream impacts of data biases quantitatively rather than qualitatively and prove theoretical guarantees for detection. Under reasonable assumptions, we quantify how the amount of bias in the model predictions varies as a function of the amount of differential sampling bias in the data, and at what point this bias becomes provably detectable by the auditor. Through experiments on two criminal justice datasets– the well-known COMPAS dataset and historical data from NYPD’s stop and frisk policy– we demonstrate that the theoretical results hold in practice even when our assumptions are relaxed. 

Abstract Social distancing remains an effective nonpharmaceutical behavioral interventions to limit the spread of COVID-19 and other airborne diseases, but monitoring and enforcement create nontrivial challenges. Several jurisdictions have turned to “311” resident complaint platforms to engage the public in reporting social distancing non-compliance, but differences in sensitivity to social distancing behaviors can lead to a mis-allocation of resources and increased health risks for vulnerable communities. Using hourly visit data to designated establishments and more than 71,000 social distancing complaints in New York City during the first wave of the pandemic, we develop a method, derived from the Weber-Fechner law, to quantify neighborhood sensitivity and assess how tolerance to social distancing infractions and complaint reporting behaviors vary with neighborhood characteristics. We find that sensitivity to non-compliance is lower in minority and low-income neighborhoods, as well as in lower density areas, resulting in fewer reported complaints than expected given measured levels of overcrowding. 

Field studies in many domains have found evidence of decision fatigue, a phenomenon describing how decision quality can be impaired by the act of making previous decisions. Debate remains, however, over posited psychological mechanisms underlying decision fatigue, and the size of effects in high-stakes settings. We examine an extensive set of pretrial arraignments in a large, urban court system to investigate how judicial release and bail decisions are influenced by the time an arraignment occurs. We find that release rates decline modestly in the hours before lunch and before dinner, and these declines persist after statistically adjusting for an extensive set of observed covariates. However, we find no evidence that arraignment time affects pretrial release rates in the remainder of each decision-making session. Moreover, we find that release rates remain unchanged after a meal break even though judges have the opportunity to replenish their mental and physical resources by resting and eating. In a complementary analysis, we find that the rate at which judges concur with prosecutorial bail requests does not appear to be influenced by either arraignment time or a meal break. Taken together, our results imply that to the extent that decision fatigue plays a role in pretrial release judgments, effects are small and inconsistent with previous explanations implicating psychological depletion processes.