Despite its tremendous success, deep learning suffers from a significant challenge of robust out-of-distribution (OOD) predictions when the test distribution is different from the training distribution, especially due to its inclination to learn spurious patterns and shortcuts to solve the task [Jo and Bengio, 2017, Geirhos et al., 2020, Poliak et al., 2018, D'Amour et al., 2020]. Invariant Risk Minimization and similar methods [Arjovsky et al., 2019, Bellot and van der Schaar, 2020, Krueger et al., 2021] propose to solve this by learning representations that are invariant across multiple environments but can be insufficient for OOD generalization without additional assumptions Ahuja et al. [2021]. Recent works have increasingly used causal language to formally define and learn non-spurious representations [Wang andJordan, 2021, Veitch et al., 2021] in order to be robust in OOD tasks. Veitch et al. [2021], Mouli and Ribeiro [2022] define counterfactual invariance to spurious features as a requirement for robust OOD predictors.
A simple way of (approximately) achieving counterfactualinvariant predictors is via counterfactual data augmentations (CDA) [Lu et al., 2020, Kaushik et al., 2019, Sauer and Geiger, 2021], where one augments the training data with inputs generated from different spurious features. This enables a predictor to learn to be invariant to these spurious features. Lu et al. [2020], Zmigrod et al. [2019], Maudslay et al. [2019] use counterfactual data augmentation to mitigate gender biases in natural language models, for example by counterfactually modifying the gendered words in the text. Kaushik et al. [2019Kaushik et al. [ , 2020]], Teney et al. [2020] use human annotators to generate counterfactual examples by making minimal changes to a given text, although this approach may not achieve the desired robustness due to lack of diversity in augmented examples [Joshi and He, 2021]. Von Kügelgen et al. [2021] uses self-supervision and data augmentation to provably disentangle content from style in vision tasks. Other works have used pretrained models to counterfactually augment smaller datasets [Hasan andTalbert, 2021, Liu et al., 2021]. While these works propose varied ways of performing counterfactual data augmentations, the general principle remains the same: To obtain representations that are either disentangled or counterfactually-invariant to spurious features (Definition 1).
In this work, we show how counterfactual data augmentations may not achieve the desired counterfactual invariance to spurious associations if these augmentations are performed by a context-guessing machine (Definition 2). We define a context-guessing machine as an abstract machine (ML model, human annotator or algorithm) that infers the most-likely context of a given input x before performing counterfactual modifications. We show that performing counterfactual changes with the most-likely context rather than considering all possible contexts can result in a representation that is not counterfactually invariant (Theorem 1). Our analysis suggests that one must be careful while design-Figure 1: SCM over the observed input X and the corresponding label Y . X is obtained from two variables C and Z with only C affecting the label Y and Z is spurious. U X and U Y denote the background noise variables in the SCM. The associational task is to predict Y from X. Since Y depends only on C, we wish to learn a representation Γ : X → R d of the input X that is counterfactually-invariant to Z.
ing counterfactual data augmentation methods (e.g., eliciting counterfactual examples from human annotators) to avoid the bias introduced from guessing a particular context for the given example.
We begin with a brief discussion of structural causal models and the definition of counterfactual variables which will be helpful in defining counterfactual-invariant representations.
A structural causal model (SCM) [Pearl, 2009, Chapter 7] describes the causal relationships between all the relevant variables and encodes the assumptions on how the observed data is generated. An SCM consists of two sets of variables: (a) endogenous variables, those that have a causal definition of how they are obtained from other variables, and (b) exogenous variables, those that are not described by the given causal model, but affect the endogenous variables. For example, consider the SCM given below
where {X, Y } are observed endogenous variables and {U X , U Y , Z, C} are unobserved exogenous variables. A (given) distribution over the exogenous variables P (U X , U Y , Z, C) entails a distribution over the endogenous variables P (X, Y ). This SCM can be represented as a causal graph as shown in Figure 1. A typical learning task is to predict Y from X; note that the task is associational, there is no causal link from X to Y .
question given a particular observation. For example, observing X = x, the counterfactual question can be "what would be the value of X had Z = z?". We express this question using the counterfactual random variable X(Z = z)|X = x.
Given the complete structural causal model, the counterfactual variable X(Z = z)|X = x can be computed as follows [Pearl, 2009, Chapter 7]:
1. (Abduction.) Compute P (U|X = x) where U is the set of all exogenous variables.
2. (Action.) Perform the intervention do(Z = z) in the given SCM.
3. (Prediction.) Compute the distribution of X in the modified SCM using the modified distribution for the exogenous variables.
We can write distribution of the counterfactual variable X(Z = z)|X = x formally as:
(1)
The integral denotes the three steps of computing the counterfactuals: (i) abduction step to obtain the distribution of the exogenous variables P (U|X = x), (ii) making the desired intervention do(Z = z), and (iii) computing the endogenous variable X under intervention using the abducted distribution. An important point to remember while computing counterfactuals is that they need not only deal with individual realizations, i.e., given X = x, there can be a population of individuals given by the distribution P (U|X = x). The do-operation is then applied to all these individuals.
Counterfactual-invariant representations. Now we are ready to describe counterfactual-invariant representation defined in Mouli and Ribeiro [2022], Veitch et al. [2021]. Veitch et al. [2021] define the counterfactual variable using the potential outcomes notation X(z ), i.e., what would X be had Z = z leaving all else fixed. While we use the notation of Mouli and Ribeiro [2022], the definitions are equivalent when supp(Z te ) = supp(Z), as assumed throughout this work.
Definition 1 (Counterfactual-invariant representations [Mouli and Ribeiro, 2022]). Given any SCM with at least two variables X and Z, a representation
almost everywhere, ∀z ∈ supp(Z), ∀x ∈ supp(X), where supp(A) is the support of random variable A.
The counterfactual variable X(Z = z |X = x) in the RHS of Equation ( 2) is as defined in Equation (1).
In the next section, we look at an example of counterfactual data augmentation in the context of classifying text reviews and showcase a scenario when it does not lead to robust classifiers.
In this section, we consider an example NLP task of predicting the helpfulness of a product review while being counterfactually-invariant to the sentiment of the review which is spurious for this task [Veitch et al., 2021].
Structural causal model for review classifiation. We begin with the structural causal model that generates the text X and the helpfulness label Y (associated causal graph in Figure 1). In this example, Z denotes the sentiment of the reviewer about the product (like or dislike), C denotes the content describing the product, U X is the type of reviewer, crudely categorized as straightforward (U X = 1) or sarcastic (U X = -1), and U Y is label noise (assumed to be zero). Table 1 concretely defines how these variables affect the text X and its helpfulness label Y . Since it is not feasible to describe all possible text inputs X, we use placeholders [ • • • ] to describe the type of text, while the actual content of the review may vary across the dataset. For example, [good 1 , positive tone] represents a particular review with good quality content and written in a positive tone. Further assume that P (U X = 1) = 0.9, i.e., straightforward reviewers are a lot more likely than sarcastic ones, P (Z = like) = 0.5, and P (U Y = 0) = 1.
A straightforward individual's sentiment affects the text in the usual way (e.g., Z = like =⇒ X has a positive tone). On the other hand, the effect of a sarcastic individual's sentiment on X is more complicated: Z = dislike =⇒ X has a positive tone, and Z = like =⇒ X has a neutral tone. Now, it is clear from Table 1 (also from Figure 1) that the sentiment Z is spurious for the label Y which only depends on C. However, a classifier may not learn this invariance to Z automatically from training data, especially if all the possible values of X are not seen during training. Thus, our goal is to obtain a representation that is counterfactually-invariant to the sentiment Z which will allow us to build OOD robust classifiers. That is, we want to augment the original dataset with counterfactual data with respect to the sentiment Z in order to obtain the counterfactual-invariant representation.
Counterfactual data augmentation (CDA). Given a text input x = [good 1 , positive tone], Definition 1 enforces the invariance considering all possible contexts (Z = like, Z = dislike), thus considering sarcastic individuals as well as straightforward ones. Considering both contexts, CDA augments [good 1 , negative tone] and [good 1 , neutral tone], thus enforcing the following invariance over the representation
Next we show that a bias may be introduced if the counterfactual data augmentation algorithm (e.g., using humansin-the-loop) does not consider all the possible contexts, but instead guesses the most-likely context. We will denote this type of augmentation machine as a contextguessing machine. As before, consider the text input x = [good 1 , positive tone]. A context-guessing machine infers the most-likely context (i.e., maximum a posteriori estimate) of the text as Z = like due to the positive tone, thus indirectly only considering straightforward individuals with U X = 1. The counterfactually augmented example is x = [good 1 , negative tone] with the same label Y = helpful and enforces the following invariance on Γ cda : Γ cda ([good 1 , positive tone]) = Γ cda ([good 1 , negative tone]). But, clearly this is not enough for counterfactual-invariance as Γ cda ([good 1 , neutral tone]) can be arbitrarily different. Thus, a classifier that uses the representation Γ cda is not guaranteed to be robust to OOD changes to sentiment Z.
In this section, we analyze the counterfactual data augmentations performed by a context-guessing machine more formally. We begin with a definition of a context-guessing machine, an abstract machine (e.g., ML model, human annotator or algorithm) that guesses the most-likely context for the given input x, i.e., most-likely instantiation of a parent of x. Throughout this section, our definitions consider a single parent Z of X, but they can be easily extended for multiple parent (context) variables.
Definition 2 (Context-guessing machine). Given any SCM with Z → X, a context-guessing machine assumes the context of x to be z MAP (x) = argmax z P (Z = z|X = x), which is the maximum a posteriori (MAP) estimate of Z given X = x.
In Definition 3, we define counterfactual data augmentation with such a context-guessing machine which works as follows: (a) Given X = x, the context is inferred to be Z = z MAP (x), and (b) a counterfactual example is generated conditioned on the inferred context while preserving the label. 1). X and Y denote the observed text review and the corresponding helpfulness label respectively. The associational task is to predict Y from X. Input X is obtained as a function of two unobserved variables C and Z. C denotes the actual content describing the product and directly affects the helpfulness label Y . We show two possible values for C denoted [good 1 ] and [poor 1 ] for simplicity representing particular good and poor quality contents respectively. Z denotes the sentiment of the reviewer about the product. U X denotes the different types of reviewers, straightforward (U X = 1) or sarcastic (U X = -1). For a straightforward individual, Z = like implies that X has a positive tone, whereas for a sarcastic individual, Z = dislike implies that X has a positive tone. Finally, since Y depends only on C and not on the sentiment Z, we wish to learn a representation Γ : X → R d of the input X that is counterfactually-invariant to Z using counterfactual DA. However, a context-guessing machine infers the most likely context, for example, that positive tone is from a straightforward reviewer who likes the product, and does CDA under this context. This does not result in counterfactual invariance as the alternative context for the positive tone-a sarcastic reviewer who dislikes the product-is not considered.
Definition 3 (Guess-CDA). Counterfactual data augmentation derived from a context-guessing machine is defined as follows: For every (x, y) in the training data D, an augmented example is (x , y) where x ∼ P (X(Z = z)|X = x, Z = z MAP (x)) for z ∈ supp(Z).
The variable
Recall that the definition of counterfactual variables in Equation (1) involved abducted distributions over the set of exogenous U given the evidence.
The distribution of counterfactually-augmented examples in Definition 3 is given by
where the abducted distribution P (U X = u|X = x, Z = z MAP (x)) marks the only difference from Equation (1). Next, we explicitly define the invariance imposed on a representation trained over the counterfactually-augmented data of Definition 3.
Definition 4 (Guess-CDA-invariance). Given any SCM with Z → X, the invariance imposed on a representation Γ cda : X → R d , d ≥ 1 of X by the counterfactual data augmentation from a context-guessing machine in Definition 3 is
almost everywhere, ∀z ∈ supp(Z), ∀x ∈ supp(X), where z MAP (x) = argmax z P (Z = z|X = x) is the maximum a posteriori (MAP) estimate of Z given X = x and supp(A) is the support of random variable A.
The support of the counterfactual variable in the RHS of Equation ( 4) can be different than the support of that in Equation (2) as illustrated by the example in Section 3. Our next theorem formalizes this notion and states that the invariance imposed by Definition 4 on Γ cda is weaker than the desired invariance of Definition 1. Hence, when performing CDA with a context-guessing machine, we are not guaranteed to obtain a counterfactually-invariant representation.
Theorem 1 (Γ cda of Definition 4 is not counterfactually-invariant). Given any SCM with Z → X, let Γ cda denote the representation defined in Definition 4 obtained via counterfactual data augmentation from a context-guessing machine. Then, in general, Γ cda is not counterfactual-invariant according to Definition 1.
We prove the theorem in Appendix A.1 in two steps. (a) First, we show that the invariance restriction imposed over Γ cda in Definition 4 is never stronger than that imposed over Γ cf in Equation ( 2) by comparing the supports of the RHS in Equations ( 2) and (4). That is, we show z supp(X(Z = z)|X = x, Z = z MAP (x)) ⊆ z supp(X(Z = z)|X = x). (b) Then, we show a linear SCM example where the invariance restriction of Definition 4 is strictly weaker than that of Definition 1. In this simple example, Definition 1 forces Γ cf to be a constant function, whereas Definition 4 allows Γ cda to take two different values based on the input x.
The solution to the challenge described above is relatively simple. We just need to avoid guessing the most likely context. The support (or at least all likely contexts Z) must be present in the data augmentation procedure. That means, for instance, giving context suggestions to human annotators either by sampling from P (Z|X) or by considering all the likely contexts based on P (Z|X).
Counterfactual invariance to spurious features is a desired property for OOD robustness of predictors. A general way of approximately achieving counterfactual invariance is via counterfactual data augmentations. In this work, we studied counterfactual data augmentations performed by a contextguessing machine and showed that a representation trained on the resultant augmented data may not be counterfactualinvariant. Our analysis suggests that one must be careful while designing counterfactual data augmentation methods (e.g., eliciting counterfactual examples from human annotators) to avoid the bias introduced from guessing a particular context for the given example.