More Like this

1. Counterfactuals for the Future

   https://doi.org/10.1609/aaai.v37i12.26655  

   Bynum, Lucius E. ; Loftus, Joshua R. ; Stoyanovich, Julia ( June 2023 , Proceedings of the AAAI Conference on Artificial Intelligence)

   Counterfactuals are often described as 'retrospective,' focusing on hypothetical alternatives to a realized past. This description relates to an often implicit assumption about the structure and stability of exogenous variables in the system being modeled --- an assumption that is reasonable in many settings where counterfactuals are used. In this work, we consider cases where we might reasonably make a different assumption about exogenous variables; namely, that the exogenous noise terms of each unit do exhibit some unit-specific structure and/or stability. This leads us to a different use of counterfactuals --- a forward-looking rather than retrospective counterfactual. We introduce "counterfactual treatment choice," a type of treatment choice problem that motivates using forward-looking counterfactuals. We then explore how mismatches between interventional versus forward-looking counterfactual approaches to treatment choice, consistent with different assumptions about exogenous noise, can lead to counterintuitive results. 

   more » « less
2. Counterfactual Sensitivity and Robustness

   https://doi.org/10.3982/ECTA17232  

   Christensen, Timothy ; Connault, Benjamin ( January 2023 , Econometrica)

   We propose a framework for analyzing the sensitivity of counterfactuals to parametric assumptions about the distribution of latent variables in structural models. In particular, we derive bounds on counterfactuals as the distribution of latent variables spans nonparametric neighborhoods of a given parametric specification while other “structural” features of the model are maintained. Our approach recasts the infinite‐dimensional problem of optimizing the counterfactual with respect to the distribution of latent variables (subject to model constraints) as a finite‐dimensional convex program. We also develop an MPEC version of our method to further simplify computation in models with endogenous parameters (e.g., value functions) defined by equilibrium constraints. We propose plug‐in estimators of the bounds and two methods for inference. We also show that our bounds converge to the sharp nonparametric bounds on counterfactuals as the neighborhood size becomes large. To illustrate the broad applicability of our procedure, we present empirical applications to matching models with transferable utility and dynamic discrete choice models. 

   more » « less
3. Consistent Counterfactuals for Deep Models

   Emily Black ; Zifan Wang ; Matt Fredrikson ; Anupam Datta ( May 2021 , Ninth International Conference on Learning Representations)

   Counterfactual examples are one of the most commonly-cited methods for explaining the predictions of machine learning models in key areas such as finance and medical diagnosis. Counterfactuals are often discussed under the assumption that the model on which they will be used is static, but in deployment models may be periodically retrained or fine-tuned. This paper studies the consistency of model prediction on counterfactual examples in deep networks under small changes to initial training conditions, such as weight initialization and leave-one-out variations in data, as often occurs during model deployment. We demonstrate experimentally that counterfactual examples for deep models are often inconsistent across such small changes, and that increasing the cost of the counterfactual, a stability-enhancing mitigation suggested by prior work in the context of simpler models, is not a reliable heuristic in deep networks. Rather, our analysis shows that a model's local Lipschitz continuity around the counterfactual is key to its consistency across related models. To this end, we propose Stable Neighbor Search as a way to generate more consistent counterfactual explanations, and illustrate the effectiveness of this approach on several benchmark datasets. 

   more » « less
4. Selective Ensembles for Consistent Predictions

   Emily Black, Klas Leino ( May 2021 , Ninth International Conference on Learning Representations)

   Counterfactual examples are one of the most commonly-cited methods for explaining the predictions of machine learning models in key areas such as finance and medical diagnosis. Counterfactuals are often discussed under the assumption that the model on which they will be used is static, but in deployment models may be periodically retrained or fine-tuned. This paper studies the consistency of model prediction on counterfactual examples in deep networks under small changes to initial training conditions, such as weight initialization and leave-one-out variations in data, as often occurs during model deployment. We demonstrate experimentally that counterfactual examples for deep models are often inconsistent across such small changes, and that increasing the cost of the counterfactual, a stability-enhancing mitigation suggested by prior work in the context of simpler models, is not a reliable heuristic in deep networks. Rather, our analysis shows that a model's local Lipschitz continuity around the counterfactual is key to its consistency across related models. To this end, we propose Stable Neighbor Search as a way to generate more consistent counterfactual explanations, and illustrate the effectiveness of this approach on several benchmark datasets. 

   more » « less
5. Being nice by choice: The effect of counterfactual reasoning on children's social evaluations

   https://doi.org/10.1111/desc.13394  

   Wong, Alyson ; Cordes, Sara ; Harris, Paul L. ; Chernyak, Nadia ( April 2023 , Developmental Science)

   The ability to engage in counterfactual thinking (reason about what elsecouldhave happened) is critical to learning, agency, and social evaluation. However, not much is known about how individual differences in counterfactual reasoning may play a role in children's social evaluations. In the current study, we investigate how prompting children to engage in counterfactual thinking about positive moral actions impacts children's social evaluations. Eighty‐seven 4‐8‐year‐olds were introduced to a character who engaged in a positive moral action (shared a sticker with a friend) and asked about whatelsethe character could have done with the sticker (counterfactual simulation). Children were asked to generate either a high number of counterfactuals (five alternative actions) or a low number of counterfactuals (one alternative action). Children were then asked a series of social evaluation questions contrasting that character with one who did not have a choice and had no alternatives (was told to give away the sticker to his friend). Results show that children who generatedselfishcounterfactuals were more likely to positively evaluate the character with choice than children who did not generate selfish counterfactuals, suggesting that generating counterfactuals most distant from the chosen action (prosociality) leads children to view prosocial actions more positively. We also found age‐related changes: as children got older, regardless of the type of counterfactuals generated, they were more likely to evaluate the character with choice more positively. These results highlight the importance of counterfactual reasoning in the development of moral evaluations.

   Older children were more likely to endorse agents whochooseto share over those who do not have a choice.

   Children who were prompted to generate more counterfactuals were more likely to allocate resources to characters with choice.

   Children who generated selfish counterfactuals more positively evaluated agents with choice.

   Comparable to theories suggesting children punish willful transgressors more than accidental transgressors, we propose children also consider free will when making positive moral evaluations.

    

   more » « less