The conditional moment problem is a powerful formulation for describing structural causal parameters in terms of observables, a prominent example being instrumental variable regression. We introduce a very general class of estimators called the variational method of moments (VMM), motivated by a variational minimax reformulation of optimally weighted generalized method of moments for finite sets of moments. VMM controls infinitely for many moments characterized by flexible function classes such as neural nets and kernel methods, while provably maintaining statistical efficiency unlike existing related minimax estimators. We also develop inference algorithms and demonstrate the empirical strengths of VMM estimation and inference in experiments.
]]>For many problems in fields such as economics, sociology, or epidemiology, we seek to use observational data to estimate structural parameters, which often describe some causal relationship. A common framework that unifies many such problems is the conditional moment problem, which assumes that the parameter of interest θ 0 is the unique element of some parameter space Θ such that
where X ∈ X denotes the observed data, Z ∈ Z is a random variable that is measurable with respect to X, and ρ : X → R m is a vector-valued function indexed by Θ. Note that Equation ( 1) is an identity of random variables, not of numbers; that is, it holds almost surely with respect to the random Z.
That Z is measurable with respect to X is without loss of generality, since given any X, Z we may define X = ( X, Z) as the observed data; thus ρ may potentially depend on all data. Note also that at this point we let Θ be general; for example, it may be finite dimensional or it may be a class of functions.
Example 1 Perhaps, the most common example of a conditional moment problem is the instrumental variable regression problem (see e.g., Angrist & Pischke, 2008, and citations therein), where we seek to estimate the causal effect of some treatment T on an outcome Y, where the observed relationship between T and Y may be confounded by some unobserved variables, but we have an instrumental variable Z that affects T but only affects Y via its effect on T. Given some regression function g parameterized by θ ∈ Θ, the value θ 0 corresponding the true regression function is assumed to be the unique solution to
This is an example of Equation ( 1) with X = (T, Y, Z) and ρ(X; θ) = Yg(T; θ). More intricate variants of this, for example, include Berry et al. (1995), which incorporate discrete choice and is widely used to formulate structural demand parameters in industrial organizations. Example 2 A second example is instrumental quantile regression (Chernozhukov et al., 2007;Horowitz & Lee, 2007). Here, again assume a treatment T, outcome Y, and instrumental variable Z, but now we seek to estimate the causal effect of T on the p th quantile of Y, for some 0 < p < 1. In this case, given a quantile regression function g parameterized by θ ∈ Θ, the value θ 0 corresponding to the true quantile regression function is assumed to be the unique solution to
This is an example of Equation ( 1) with X = (T, Y, Z) and ρ(X; θ) = 1{Y ≤ g(T; θ)} -p.
Example 3 A third example of a problem is estimating the stationary state density ratio between two policies in offline reinforcement learning (Bennett et al., 2021;Kallus & Uehara, 2022;Liu et al., 2018). Consider a Markov decision process given by an unknown transition kernel p(S ′ | S, A) describing the distribution of next state S ′ when action A is taken in previous state S. Suppose Then, for example, if Θ satisfies ∫ d(s; θ)dμ(s) = 1 ∀θ ∈ Θ for some fixed measure μ, and there exists some θ 0 ∈ Θ such that d(S; θ 0 ) ∝ d(S), then this conditional moment restriction will identify θ 0 . Note that although d(S; θ 0 ) ≠ d(S) in general, estimates of θ 0 are still of interest, as they could be used to estimate d(S) in downstream tasks, for example by dividing by a plug-in estimate of E[d(S; θ 0 )] using estimates of θ 0 and E [since d(S) is known to satisfy the normalization constraint E[d(S)] = 1.] This is an example of Equation ( 1) with X = (S, A, S ′ ), Z = S ′ , and ρ(X; θ) = d(S; θ 0 )π e (A | S)π -1 b (A | S) -d(S ′ ; θ 0 ).
The classic approach to the conditional moment problem is to reduce it to a system of k marginal moments, E[F(Z)ρ(X; θ 0 )] = 0, where F : Z 7 ! R k×m is a chosen matrix-valued function. Then, we can apply the optimally weighted generalized method of moments (OWGMM; Hansen, 1982), which we present in detail in Section 2.1. Since this marginal moment formulation is implied by Equation (1) but not necessarily vice versa, this requires us to we find a sufficiently rich F(Z) such that the marginal moment problem still identifies θ 0 , that is, it is still the unique solution in Θ. Moreover, even if this identifies θ 0 and even though OWGMM is efficient in the model implied by E[F(Z)ρ(X; θ 0 )] = 0, the result may not be efficient in the model implied by Equation (1).
There are a few general approaches to deal with this. There are classic nonparametric approaches that are sieve-based and simply grow k, the output dimension of F(Z), with n by including additional functions from a basis for L 2 such as power series (Chamberlain, 1987). There are also classic nonparametric approaches that directly estimate some special identifying F * (Z) that also induces an efficient OWGMM (Newey, 1990(Newey, , 1993)). For example, in Example 1 with g(T; θ) = θ ⊤ T, we have F * (Z) = E[T | Z], which can be nonparametrically estimated and plugged into OWGMM. Furthermore, there are approaches that used sieve-based methods to simultaneously estimate E[ρ(X; θ) | Z] for every θ ∈ Θ, and pick θ to minimize some weighted empirical norm of these estimated conditional expectations (Ai & Chen, 2003;Chen & Pouzo, 2009, 2012;Newey & Powell, 2003).
A recent line of work instead focuses on tackling this with machine-learning-based approaches (Bennett et al., 2019;Dikkala et al., 2020;Hartford et al., 2017;Kallus et al., 2021;Lewis & Syrgkanis, 2018;Muandet et al., 2019;Singh et al., 2019;Uehara et al., 2021). These approaches are varied, with some solving the general problem in Equation (1) and others solving the more specific instrumental variable regression problem or other specific problems, with approaches based on deep learning, kernel methods, or both. Most of these are based on an adversarial/minimax/ saddle-point approach (Bennett et al., 2019;Dikkala et al., 2020;Kallus et al., 2021;Lewis & Syrgkanis, 2018;Muandet et al., 2019;Uehara et al., 2021).
Currently, there is a disconnect between these two lines of approaches. On the one hand, the more classical approaches are well motivated by efficiency theory when we impose certain smoothness assumptions. This is in contrast with the recent machine-learning based approaches; while some provide consistency guarantees (Bennett et al., 2019) and even rates (Dikkala et al., 2020;Kallus et al., 2021;Singh et al., 2019;Uehara et al., 2021), none of these approaches are shown to be semiparametrically efficient for Equation (1) or can facilitate inference on θ 0 . On the other hand, however, the more recent line of work leverages modern machine learning approaches, which are commonly believed to have superior practical properties. For example, they have been empirically observed to be more stable, have easier parameter tuning, or be better able to adapt to the low-dimensional latent structure of complex data. Although our experiments do indeed seem to support this thesis, especially in more challenging settings, we emphasise that the point of this paper is not to demonstrate that modern machine learning-based approaches are superior to classical ones. Rather, we observe that for various reasons there is significant, growing interest in machine-learning based approaches to these problems within the community, and therefore extending this line of work to be semiparametrically efficient and to perform inference is of great importance.
In this paper, we study a general class of minimax approaches, which we call the variational method of moments (VMM). This generalizes the method of Bennett et al. (2019), who presented an estimator for instrumental variable regression using adversarial training of neural networks. Their proposal was motivated by a variational reformulation of OWGMM, aiming to combine the efficiency of more classical approaches with the flexibility of machine learning methods. This style of estimator has since been applied to a variety of other conditional moment problems including policy learning from observational data (Bennett & Kallus, 2020) and estimating stationary state density ratios (Bennett et al., 2021). However, this past work did not provide a general formulation of VMM and a detailed theoretical analysis. And, although they are motivated by efficiency considerations, it is not immediately clear that this actually leads to efficient estimators.
We present a unified theory for a general class of VMM estimators. In particular, for some specific versions of these estimators based on either deep learning or kernel methods, we provide appropriate assumptions under which these methods are consistent, asymptotically normal, and semiparametrically efficient. In addition, we provide inference algorithms for these estimators, which can be used to construct confidence intervals for the estimated parameters. These inference algorithms are based on the same kind of variational reformulation as the estimation algorithms themselves, again with varieties based on both kernel methods and deep learning. Finally, we provide a detailed series of experiments that demonstrate that these VMM algorithms obtain very good finite-sample estimation performance and that the corresponding inference algorithms produce high quality confidence intervals.
The rest of this paper is structured as follows: in Section 2, we define the VMM estimator and provide motivation for it by interpreting OWGMM as a specific case thereof; in Section 3, we provide our theory for kernel VMM estimators, which are a specific instance of VMM estimators based on kernel methods; in Section 4, we provide our theory for neural VMM estimators, which are an alternative instance of VMM based on deep learning methods; in Section 5, we present our inference theory, with proposed kernel-and neural net-based algorithms; in Section 7, we provide a detailed empirical evaluation of our proposed estimation and inference methods; and in Section 8 we provide a detailed discussion of past work on solving conditional moment problems and how these approaches relate to our VMM estimators.
Bennett and Kallus
We use uppercase letters such as X to denote random variables and lowercase ones to denote nonrandom quantities. The set of positive integers is N, and for any n ∈ N we use [n] to refer to the set {1, . . . , n}. We denote by ‖ • ‖ Lp the usual L p functional norm, defined as ‖f
where the probability measure is implicit from context.
We now define the class of VMM estimators. We consider data consisting of n independent and identically distributed observations of X, namely, X 1 , . . . , X n ∼ P, where P denotes the data distribution. Let some sequence of function classes F n be given, such that each f ∈ F n has signature f : Z → R m . Let a 'prior estimate' θn ∈ Θ be given. In general, this may be any data-driven choice from Θ and need not necessarily be consistent for θ 0 ; in the theory that follows we will elaborate on what conditions θn needs to satisfy for our respective results. Furthermore, let R n : F n → [0, ∞] be some optional regularizer, which measures the complexity of f ∈ F n . Then, we define the VMM estimate
) corresponding to these choices as follows:
where E n is an empirical average over the n data points.
In Section 3, we study the instantiation of this with F n being a reproducing kernel Hilbert space (RKHS). In Section 4, we study the instantiation with F n being a class of neural networks.
Before proceeding to study these new machine-learning-based instantiations of the VMM estimator with flexible choices for F n , we discuss a very simple instantiation that recovers OWGMM, which provides motivation and interpretation for each of the terms in Equation (2).
First, we present the classic OWGMM method. Given F(Z) = (f 1 (Z), . . . , f k (Z)), we obtain the marginal moment conditions E[f i (Z) ⊤ ρ(X; θ 0 )] = 0 ∀i ∈ [k]. Let a 'prior estimate' θn be given and define the matrix Γ as
Then, the OWGMM estimate
Given certain regularity conditions and assuming the choice of functions f 1 , . . . , f k are sufficient such that the corresponding k moment conditions uniquely identify θ 0 , standard GMM theory says that θn is consistent for θ 0 . Furthermore, if the prior estimate θn is consistent for θ 0 , then this estimator is efficient with respect to the model defined by these k moment conditions (Hansen, 1982). OWGMM generalizes the method of moments, which solves
When there are many moments, we cannot make all of them zero due to finite-sample noise and instead we seek to make them near zero. But, it is not clear which moments are more important; for example, there may be duplicate or near-duplicate moments. The key to OWGMM's efficiency is to optimally combine the k objectives of making each moment near zero into a single objective function. To get a consistent prior estimate, we can for example let θn itself be a OWGMM with any fixed prior estimate, leading to the two-step GMM estimator. This can be repeated, leading to the multi-step GMM estimator.
Unfortunately, estimators of this kind have many limitations. For one, in practice, it is difficult or impossible to verify that any such set of functions f 1 , . . . , f k are sufficient for identification. In addition, while such an estimator is efficient with respect to the model imposed by these k moment conditions, ideally we would like to be efficient with respect to the model given by Equation (1); that is, we would wish to be efficient with respect to the model given by all moment conditions of the form E[f (Z) ⊤ ρ(X; θ 0 )] = 0 for square integrable f. Finally, in the case that k were very large and growing with n, as would be required to (at least approximately) alleviate the prior two concerns, the corresponding sieve-based estimator would require impractical tuning to select which basis of L 2 to use and to choose k as a function of n. As will be seen below, such an approach may be seen as equivalent to estimating the optimal instruments over a linear sieve, but unlike our variational approach that we propose below it is unclear how to appropriately regularize this sieve estimation and take advantage of modern machine learning advances on non-parametric function approximation.
One motivation for our VMM class of estimators, Equation ( 2), is that it recovers OWGMM with its efficient weighting.
The following result simply appeals to the optimization structures of Equations ( 2) and (3) and generalizes Bennett et al. (2019, Lemma 1). We include its proof as it is short and instructive.
where the second equality is a reformulation of the rotated Euclidean norm (see Online Supplementary Material, Lemma 15 for the general Hilbert-space version). The conclusion follows by noting
Through the lens of Lemma 1, we can understand each term of Equation ( 2) as follows. The first term pushes θ to make
, appropriately weights the relative importance of making each of these near zero. Finally, varying F n and/or R n (f ) with n allows us to control the richness of moments that we consider, in analogy to sieve-based methods that grow the dimension of the space span({f 1 , . . . , f k }) but admitting more flexible machine-learning approaches. This motivation is similar to Bennett and Kallus (2020); Bennett et al. (2019Bennett et al. ( , 2021)), but these did not study the problem in generality or establish properties such as asymptotic normality or efficiency.
First, we consider a class of VMM estimators where for every n we have F n = F , where F = m i=1 F i and each F i is a RKHS of functions Z → R given by a symmetric positive definite kernel K i : Z × Z → R, and regularization is performed using the RKHS norm of F , which we denote by
We will call these estimators kernel VMM estimators, which we concretely define according to
and α n is some non-negative sequence of regularization coefficients. Explicitly, this fits into our general VMM definition with F n = F for every n, and R n (f ) = α n ‖f ‖ 2 . Before we provide our main theory for kernel VMM estimators, we provide a convenient reformulation of Equation ( 4). Let H be the dual space of F (that is, the space of all bounded linear functionals of the form F 7 ! R) and for each θ ∈ Θ define the element ̅ h n (θ) ∈ H according to
Furthermore, define the linear operator C n : H → H according to
where φ : H → F maps any element in H to its Riesz representer in F such that h(f ) = 〈φ(h), f 〉.
The kernel VMM estimator defined in Equation ( 4) is equivalent to
where I is the identity operator Ih = h.
We note that comparing this result to Equation (3), this is a clear infinite-dimensional generalization of the OWGMM objective, where the matrix Γ defined there is replaced with a linear operator, and the inversion is performed using Tikhonov regularization. Note that this re-framing of our kernel VMM estimator also shows a connection to the continuum GMM estimators considered by Carrasco and Florens (2000). However, our estimator does not strictly fit within their framework. We discuss this in more detail in Section 8.
We first provide some sufficient assumptions in order to ensure that our kernel VMM estimator is consistent; that is, θK-VMM n → θ 0 in probability. Before we present these assumptions, we define the conditional covariance function of the moment problem
(5)
For our first assumption, we require each F i to be universally approximating with a smooth kernel. Recall for this definition that, a function is C ∞ -smooth if it is n-times continuously differentiable for every positive integer n. In addition, we recall that a kernel is universal if the corresponding RKHS is dense in the space of continuous real-valued functions on Z under the supremum norm (Sriperumbudur et al., 2011). Note that all of the properties of the following assumption hold, for example, for the commonly used Gaussian kernel.
Assumption 1 (Universal RKHS). For each i ∈ [m], K i is C ∞ -smooth in both arguments and F i is universal.
Next, we require a basic regularity condition on the observed data distribution. This together with Assumption 1 ensure that F is well-behaved with respect to ρ and satisfies some nice properties in terms of boundedness and metric entropy, as formalized by Online Supplementary Material, Lemma 16 in the Appendix.
Assumption 2 (Regularity). Z is a bounded subset of R dz for some positive integer d z .
Next, we require that the set of possible functions {ρ( • ; θ) : θ ∈ Θ} satisfies some basic boundedness, smoothness, and complexity properties. A simple example satisfying the below is for Θ to be a compact set in some finite-dimensional Euclidean space, and for ρ(x; θ) to be equi-Lipschitz continuous in θ for every x. Other examples that easily satisfy the second part of the below include {θ( • ; θ) : θ ∈ Θ} having finite Vapnik-Chervonenkis dimension (see e.g., Kosorok, 2007, Theorem 8.19 and Corollary 9.5), or be a bounded-norm subset of an RKHS (see Online Supplementary Material, Lemma 17 in the Appendix for details). This assumption ensures that consistent estimation of θ 0 is possible, even though inversion of the conditional moment operator could be ill-posed.
Assumption 3 (Moment Class Complexity). sup x∈X ,θ∈θ |ρ(x; θ)| < ∞, and ρ(X; θ) is Lipschitz continuous in θ under the L 1 norm. Also, for each i ∈ [m], the function set {ρ i ( • ; θ) : θ ∈ Θ} is P-Donsker.
We also assume that the prior estimate θn is well-behaved, meaning that it converges sufficiently fast to some limit in probability. This limit need not be θ 0 for our consistency results. This will be used to ensure the convergence of the linear operator C n defined above to some limiting operator C.
Assumption 4 (Convergent Prior Estimate). The prior estimate θn has a limit θ in probability, and satisfies ‖ρ i (X; θn ) -ρ i (X; θ)‖ 2 = O p (n -p ) for every i ∈ [m] and some 0 < p ≤ 1/2.
Finally, we assume a nonsingular covariance with bounded inverse moments.
Assumption 5 (Non-Degenerate Moments). For each θ ∈ { θ, θ 0 }, we have that V(Z; θ) is invertible almost surely, and also that ‖σ min (Z; θ) -1 ‖ ∞ < ∞, where σ min (Z; θ) denotes the minimum eigenvalue of V(Z; θ).
Assumption 5 is slightly subtle and is used to ensure that the objective J n defined above converges to a well-behaved limiting objective J that is uniquely minimized by θ 0 , which is central to our consistency proof. In the absence of this assumption, it is possible that the limiting objective may diverge. We note that in the case of m = 1, the second part of the assumption is equivalent to requiring that ‖V(Z; θ) -1 ‖ ∞ , ‖V(Z; θ 0 ) -1 ‖ ∞ < ∞, and in the case that the prior estimate θn is consistent we only need this condition to hold at θ 0 = θ. In general, it can be viewed in terms of certain moments defined in terms of the data distribution and ρ being bounded.
With these assumptions, we are prepared to state our consistency result.
Theorem 1 (Consistency). Let Assumptions 1-5 be given and suppose the regularization coefficient satisfies α n = o(1) and α n = ω(n -p ), where p is the constant referenced in Assumption 4. Then, for any θn that satisfies J n ( θn ) = inf θ∈Θ J n (θ) + o p (1), we have θn → θ 0 in probability.
Comparing this result to the corresponding consistency result given by Bennett et al. (2019, Theorem 2), we note that this result does not rely on any specific identification assumptions beyond Equation (1). Conversely, Bennett et al. (2019) assume that the class F of neural nets that they take a supremum over is sufficient to uniquely identify θ 0 , which is a questionable assumption since this class is assumed to be fixed and not growing with n. Therefore, we argue that our VMM consistency here is given under much more reasonable assumptions.
Next, we make some observations about how this result compares with consistency results in the literature that tackles nonlinearities using sieves. First, note that Assumption 3 is weaker than the corresponding assumptions in Ai and Chen (2003) and Newey and Powell (2003), who assume that ρ(x; θ) is point-wise Hölder-continuous and Θ is compact. Instead, we require the more general assumption of continuity in the L 1 -norm, along with a Donsker condition. Conversely, Chen and Pouzo (2009) and Chen and Pouzo (2012) similarly allow for non-smooth ρ, but they consider the setting where Θ can be non-compact, introducing ill-posedness issues that they tackle in their work. Rather, we specifically consider metrics on Θ under which ill-posedness is not an issue, given our Donsker assumption on {ρ( • ; θ) : θ ∈ Θ} and L 1 -continuity. Furthermore, we note that assumptions similar to Assumptions 2 and 5 are standard in these past works, and Assumptions 1 and 4 are straightforward technical conditions related to implementation choices for our method.
We now present our theory for the asymptotic normality of kernel VMM estimates. Here, we consider the special case where Θ is a compact subset of R b for some positive integer b. We note that in this case, as discussed above, Assumption 3 follows under very simple additional conditions; e.g., ρ(x; θ) being equi-Lipschitz continuous in θ for every x ∈ X . Under this setting, we will characterize the asymptotic distribution of �� n √ ( θn -θ 0 ). First, we require that ρ(X; θ) satisfies the following differentiability condition.
Assumption 6 (ρ Differentiable in Absolute Mean). For each i ∈ [m], there exists some vector-valued function D i (X; θ) ∈ R b indexed by θ, and some neighbourhood Θ 0 of θ 0 , such that, for every θ ∈ Θ 0 , we have lim
In other words, D i is a gradient-like function such that the first-order Taylor error decays to zero at a o(‖θ ′ -θ‖) rate under the L 1 norm. For example, in the case that ρ i (x; θ) is continuously differentiable in θ within some neighbourhood of θ 0 for all x ∈ X , then Assumption 6 trivially follows from Taylor's theorem. Furthermore, it is easy to see that for any x, θ where ρ i (x; θ) is differentiable w.r.t. θ, we must have D i (x; θ) = ∇ρ i (x; θ). However, the above is more general and allows for situation where ρ i (x; θ) is non-differentiable at some values of x and θ. In particular, the following lemma allows us to establish this assumption under more general conditions. Lemma 3 Suppose there exist ϕ i : X → R indexed by Θ, and 'gradient-like' and 'Hessian-like' functions ρ ′ i and ρ ′′ i such that: (1) ρ i (X; θ) is twice differentiable in θ with gradient ρ ′ i (X; θ) and Hessian ρ ′′ i (X; θ) whenever ϕ(X; θ)
for some L ϕ (x) such that the probability density of the random variable L ϕ (X) -1 ϕ(X; θ) is bounded within some neighbourhood of zero. Then, we have that Assumption 6 holds with
This lemma allows us to establish Assumption 6 for a range of problems where ρ(X; θ) has some points of non-smoothness. Intuitively, the boundedness condition on ρ ′′ allows us to bound the first-order Taylor error whenever ρ is smooth, and the Lipschitz and bounded density assumptions on ϕ near ϕ = 0 prevents non-smoothness from impacting the first-order Taylor expansion, up to an additional o(‖θ ′ -θ‖) factor.
Next, we let D(X; θ) ∈ R m×b denote the Jacobian-like function given by concatenating
where H is the dual space of F , as above. We also define the analogue of the gradient of the objective
and note that in the case that ρ(X
In addition, we define linear operators C : H → H and C 0 :
where θ is the probability limit of θn as specified by Assumption 4, and φ is defined as in the definition of C n above. Given these definitions, we can now specify our additional assumptions and the asymptotic normality result.
This next additional assumption is a regularity condition on D(X; θ), which extends the properties of ρ(X; θ) specified in Assumption 3 to D(X; θ) j for each j ∈ [b].
Assumption 7 (Gradient Complexity). Let Θ 0 be the neighbourhood of θ 0 from Assumption 6.
Next, we assume a certain non-degeneracy in the parametrization of the problem, locally near θ 0 .
Assumption 8 is needed to ensure that the limiting asymptotic variance is finite and that the matrix Ω defined in the theorem statement below is invertible. It can be interpreted as the assumption that the parametrization of Θ is non-degenerate, since it requires that the functions E[ρ ′ i (X; θ 0 ) | Z] are linearly independent. Note that this assumption is somewhat lax, since if it were violated, it is likely possible we could re-parameterize the problem with a lower-dimensional Θ in order to avoid this issue.
Finally, we will need to introduce a couple of important definitions. We say that an estimator θn for θ 0 is asymptotically linear if θn = E n [ψ(X)] + o p (n -1/2 ), for some ψ satisfying E[ψ] = θ 0 . n addition, we say that such an estimator is asymptotically normal if �� n √ ( θn -θ 0 ) converges in distribution to a mean-zero Gaussian random variable, with some fixed covariance matrix.
With these additional assumptions and definitions, we are prepared to present our asymptotic normality result.
Theorem 2 (Asymptotic Normality). Let Assumptions 1-8 be given, and suppose the regularization coefficient satisfies α n = o(1) and α n = ω(n -p ), where p is the constant defined in Assumption 4. Then, for any θn that satisfies
is asymptotically linear and asymptotically normal, with covariance matrix Ω -1 ΔΩ -1 , where Δ and Ω are defined according to
Note that this theorem requires an approximate first-order optimality condition, namely, that the gradient-like element
), which is stronger than the approximate optimality condition in Theorem 1. Although this condition may be difficult to interpret or verify in general, the following lemma provides some sufficient conditions.
Lemma 4 (Sufficient Conditions for Approximate First-Order Optimality). Suppose that either (1) θn ∈ arg min θ J n (θ); or (2) ρ(x; θ) is twice continuously differentiable in θ for every x ∈ X , and
Then, given the other conditions of Theorem 2, we have
Comparing this result to comparable results in the literature leveraging more classical nonparametric approaches, we note that our differentiability condition in Assumption 6 is weaker than the point-wise differentiability of ρ(X; θ) assumed by Ai and Chen (2003), but stronger than Chen and Pouzo (2009) who only require differentiability of E[ρ(X; θ) | Z]. We also note that, these two works further allow for non-parametric nuisance functions in addition to the asymptotically normal parametric component. Furthermore, we note that Assumption 8 is a standard condition in all of these works.
Next, we address the question of efficiency of these kernel VMM estimators. In order to present this theory, we first need to introduce the notions of regularity and semiparametric efficiency; we refer the reader to Van der Vaart (2000) for precise definitions. Roughly speaking, we say that an estimator θn is regular with respect to some model of distributions if it is sufficiently well behaved such that its asymptotic behaviour is invariant to small perturbations [of size O p (n -1/2 )] to the data-generating distribution that remain inside the model. In addition, we say that θn is semiparametrically efficient with respect to a model of distributions if it is regular and achieves the minimum asymptotic variance among all regular estimators (with respect to that model).
Given the complex form of the limiting covariance in Theorem 2 in terms of linear operators and inner products on H, it is not immediately clear how large this covariance is and whether it is efficient under any conditions. Fortunately, the following theorem, which holds under no additional assumptions, justifies efficiency in the case that our prior estimate for θ 0 is consistent.
Theorem 3 (Efficiency). Let the assumptions of Theorem 2 be given with θ = θ 0 , and let θn be any estimator that satisfies the conditions of Theorem 2. Then, θn is semiparametrically efficient with respect to the model given by Equation ( 1) and �� n √ ( θn -θ 0 ) is asymptotically normal with asymptotic covariance matrix Ω -1 0 , where Ω 0 is defined according to
This theorem immediately implies that such a kernel VMM estimator is not only efficient with respect to the class of all kernel VMM estimators, but that it achieves the semiparametric efficiency bound for solving Equation (1). This is a very strong result, which ensures that these kernel VMM estimators inherit the efficiency properties that OWGMM estimators possess for standard moment problems, as was hoped.
Comparing against the efficiency results of the continuum GMM estimators of Carrasco and Florens (2000), which is the most similar approach to kernel VMM, ours is stronger. Specifically, they only justified that their estimator is efficient compared with other estimators in their class of continuum GMM estimators, while we have proven efficiency relative to all possible regular estimators. That is, we achieve the same semiparametric efficiency as, e.g., Ai and Chen (2003) and Chen and Pouzo (2009), who use fundamentally different sieve-based approaches.
Finally, we note that although this limiting covariance matrix has a somewhat complicated form, this form has a variational interpretation similar to the Kernel VMM estimator itself. We discuss this interpretation and how to use it to estimate the efficient asymptotic variance in Section 5.
Finally, we address some implementation considerations for kernel VMM estimators.
Firstly, we note that the above theory does not provide any guidance on how to actually construct a prior estimate θn that has the required properties described in Assumption 4. In order to address this issue, we now present a concrete method for constructing such a θn , which allows us to avoid explicitly assuming Assumption 4. Let us use the terminology that θn is a 0-step kernel VMM estimate if θn is chosen as some arbitrary fixed value, which does not depend on the observed data. Then, for any integer k > 0, we say that θn is a k-step kernel VMM estimate if θn is computed by approximately solving Equation (4) according to J n ( θn ) = inf θ∈Θ J n (θ) + o p (1/n), with θn chosen as a (k -1)-step kernel VMM estimate. In other words, θn is a k-step kernel VMM estimate if it is computed by iteratively approximately solving Equation ( 4) k times, with θn chosen as the previous iterate solution, starting from some arbitrary constant value. This scheme is analogous to that of the k-step GMM estimator (Hansen et al., 1996). Given this definition, we have the following lemma:
Lemma 5 Suppose that θn is a k-step kernel VMM estimate for some k > 0. Then, given all assumptions of Theorem 2 except for Assumption 4, it follows that θn satisfies the conditions of Assumption 4 with p = 1/2, and θ = θ 0 .
Therefore, as long as we construct θn as a k-step kernel VMM estimator as described above for some k > 1, we are assured that Assumption 4 will be met with p = 1/2 and θ = θ 0 . Given this and Theorem 3, we immediately have the following corollary for k-step kernel VMM estimators.
Corollary 1 Suppose that θn is calculated as a k-step kernel VMM estimate for some k > 1.
Then given Assumptions 1-3 and 5-8, and assuming that the regularization coefficient satisfies α n = o(1) and α n = ω(n -1/2 ), it follows that θn is semiparametrically efficient for θ 0 .
This corollary ensures that, given our regularity assumptions about F and the conditional moment problem itself, we can construct a specific k-step kernel VMM estimator that is semiparametrically efficient. The above also provides a valid specific choice of the regularization coefficient α n that does not depend on unknown parameters.
Secondly, we address the fact that the cost function described in Equation ( 4) is given by a supremum over the infinite F and provide a closed-form for the objective. By appealing to the representer theorem, and the factorization of F into the direct sum of m RKHSs, we can establish the following lemma.
Then, the cost function J n (θ) being minimized by Equation ( 4) is equivalent to
In other words, the kernel VMM estimator can be computed by minimizing a simple closedform cost function, which is given by a particular convex quadratic form on the terms of the form
In the special case of instrumental variable regression, where we are fitting the regression function within an RKHS ball, we can not only find a closed-form solution for the cost function J n (θ) to be minimized, but for the kernel VMM estimator itself. Specifically, we provide the following lemma, which follows by applying the representer theorem again.
Lemma 7 Consider the instrumental variable regression problem, where m = 1, ρ(X; θ) = Y -θ(T), F is the RKHS with kernel K f , and Θ is a ball of the RKHS with kernel K g with radius r and centred at zero. In addition, let Y denote the vector of outcomes (Y 1 , . . . , Y n ), let L f and L g denote the kernel Gram matrices of K f and K g on the data Z 1 , . . . , Z n and T 1 , . . . , T n , respectively, and define the n × n matrices Q(θ) and M according to
Then, we have
where
for some λ n ≥ 0 which depends implicitly on r, K f , K g , θn , and the observed data.
The term λ n enters into the above equation via Lagrangian duality, since minimizing J n (θ) over the RKHS ball with radius r is mathematically equivalent to minimizing J n (θ) + λ n ‖θ‖ 2 over the entire RKHS, for some implicitly defined λ n ≥ 0. In practice, however, when performing IV regression according to Lemma 7 we could freely select λ n as a hyperparameter instead of r. Superficially, the form of this estimator is similar to that of other recently proposed kernel-based estimators for IV regression (Muandet et al., 2019;Singh et al., 2019). However, unlike those estimators, ours incorporates optimal weighting using the prior estimate θn .
We now consider a different class of VMM estimators, where the sequence of function classes F n is given by a class of neural networks with growing depth and width. We will refer to estimators in this class as neural VMM (N-VMM). Most generally, we will define the class of N-VMM estimators according to
where R n (f ) is some regularizer. In this section, we analyze K-VMM for different choices of R n .
For simplicity, we will restrict our theoretical analysis to the case where F n is a fully connected neural network with ReLU activations and a common width in all layers, which allows us to use the universal approximation result of Yarotsky (2017, Theorem 1). Specifically, we fix a network architecture with D n hidden layers, each with W n neurons, with the final fully connected layer connecting to the m outputs. Then, the class F n is given by varying the weights on this network. We note that this choice is made for simplicity of exposition, but similar bounds could be given for different kinds of architectures, using other universal approximation results as in e.g., Yarotsky (2017Yarotsky ( , 2018)).
First, we consider the case where we regularize using some RKHS norm. Specifically, let F K be a product of m RKHSs satisfying Assumption 1, and
where
, and K k is the kernel Gram matrix on the data Z 1 , . . . , Z n using the kernel for the k th dimension of F K . Then, we will consider estimators of the form
We note that if we were to replace F n with F K in equation ( 7), then this equation would be equivalent to equation ( 4), since by the representer theorem regularizing by ‖f ‖ F K gives the same supremum over f as regularizing by ‖f ‖ n,K . Given this and the known universal approximation properties of neural networks, it may be hoped that if we grow the class F n sufficiently fast, then the objective we are minimizing over θ in equation ( 7) is approximately equal to that of equation (4) in a uniform sense over θ ∈ Θ. This, then, would hopefully imply that this neural VMM estimator is able to achieve the same desirable properties, in terms of consistency, asymptotic normality, and efficiency, as our kernel VMM estimators.
In order to formalize the above intuition, we first require the following assumption, which allows us to account for the rate of growth of the kernel Gram matrix inverses K -1 k in the results we give below.
Assumption 9 (Inverse Kernel Growth). There exists some deterministic positive sequence
In addition, we require the following assumption on the rate of growth on the width W n and depth D n of F n , in order to ensure that we can approximate equation (4) sufficiently well.
Assumption 10 (Neural Network Size). There exist constants q ≥ 0, 0 < a < 1/2 and a sequence
Finally, in our results and discussion below, we will define J n (θ) to be the loss in θ minimized by θNK-VMM n , and J * n (θ) to be the corresponding oracle loss if we were to replace F n with F K .
Lemma 8 Let Assumptions 1-3, 9, and 10 be given. Then, we have
This lemma follows by applying recent results on the size of a neural network required to uniformly approximate all functions of a given Sobolev norm (Yarotsky, 2017), and also older results that show that, under the conditions of Assumption 1, any RKHS ball has bounded Sobolev norm for any Sobolev space using more than d z /2 derivatives (Cucker & Smale, 2002).
Given this, we can immediately state the following theorem, which ensures that the theoretical results of our kernel VMM estimators carry over to our neural VMM estimators with kernel regularization.
Theorem 4 Let the assumptions of Theorem 1 and Assumptions 9 and 10 be given. In addition, let θn be any sequence that satisfies J n ( θn ) = inf θ∈Θ J n (θ) + o p (n -q ), where q is the constant referenced in Assumption 10. Then, in the case that these assumptions hold with q = 0, we have θn → θ 0 in probability. Furthermore, suppose in addition that the assumptions of Theorem 2 hold, and the above assumptions are strengthened to hold with q = 1. Then, we have that �� n √ ( θn -θ 0 ) converges in distribution to a mean-zero Gaussian random variable, with covariance as given by Theorem 2.
Finally, assume that in addition θ = θ 0 . Then the asymptotic variance of θn is given by Theorem 3, and the estimator is semiparametrically efficient.
The proof of this theorem follows immediately from Lemma 8, since this Lemma and the Theorem's conditions ensure that J * n ( θn
). Therefore, we can directly apply Theorems 1-3 to obtain these three results.
An immediate observation given this theorem is that, if we define k-step estimators as in Section 3.4, then by applying an identical argument we can construct efficient neural VMM estimators without having to explicitly make Assumption 4.
Motivated by our theory above using kernel-based regularizers, we now provide some discussion of general neural VMM estimators of the form given by Equation ( 6) for other choices of R n (f ), and in particular we discuss how these estimators may be justified.
First, consider the case where the kernel Gram matrices K i for i ∈ [m] are approximately equal to σ i I, where σ i is some scalar and I is the identity matrix. For example, this is the case if we use a Gaussian kernel with very small length scale parameter. In this case, we may reasonably approximate
That is, we could justify instead regularizing using some (possibly weighted) Frobenius norm of the matrix given by the values of the vector-valued f at the n data points. This form of regularization is much more attractive than that given by ‖f ‖ n,K , since it does not involve the computation of inverse kernel Gram matrices, and it more naturally fits into estimators for Equation ( 6) given by some form of alternating stochastic gradient descent. We also note that this form of regularization, based on the Frobenius norm of f, is similar to that used by Dikkala et al. (2020), although with some important differences; their proposed estimators do not include the -(1/4)E n [(f (Z) ⊤ ρ(X; θn )) 2 term motivated by efficiency theory, and they only present theory on bounding the risk of their learned function given by θn , not on the consistency or semiparametric efficiency of the estimated θn . We discuss this comparison in more detail in Section 8. Alternatively, we may heuristically justify leaving out the R n (f ) term altogether, under the argument that neural network function classes naturally impose some smoothness constraints, and therefore optimizing over F n is morally similar to optimizing over F K with some norm constraint. This intuition can be made more concrete by noting that there is a rich literature showing equivalence between optimizing loss functions over neural network function classes, and optimizing the same loss over some norm-bounded RKHS class whose kernel is implicitly defined by the neural network architecture [see e.g., Shankar et al. (2020) and citations therein]. However, we leave more specific non-heuristic claims on the performance of our neural VMM algorithms with R n (f ) = 0 to future work.
Regardless of the choice of the regularization term R n , the question remains of how to actually solve Equation ( 6). Past work (Bennett & Kallus, 2020;Bennett et al., 2019) has solved this problem using the Optimistic Adam (OAdam) algorithm, which is a form of alternating stochastic gradient descent (that is, alternating between first-order gradient steps minimizing the game objective with respect to θ and maximizing the game objective with respect to f) that has been designed to have good properties for solving minimax problems (Daskalakis et al., 2017). These past works have proposed to do this by continuously updating θn ; that is, at each iteration of alternating stochastic gradient descent they set θn as the previous iterate solution.
Alternatively, there is a rich recent literature on other, potentially more efficient, methods for solving smooth game optimization problems such as Equation (4). For example, see Fiez et al. (2020); Gidel et al. (2019); Lin et al. (2020aLin et al. ( , 2020b)); Loizou et al. (2020); Thekumparampil et al. (2019), and references therein. Some or all of the approaches suggested in these recent works may lead to successful neural VMM implementations. However, we leave this more empirical investigation to future work, and in our experiments we focus on approaches based on OAdam with continuously updated θn , as discussed above.
So far, we have developed both theory and algorithms for kernel and neural VMM estimators, providing conditions under which such estimators are consistent, asymptotically normal, and/or efficient. We now extend our efficient estimation theory to efficient inferential theory, focusing on the case of Θ ⊆ R b . Now, suppose we want to construct confidence intervals for ψ( θn ), for some ψ : R b 7 ! R. This is a very general kind of quantity to consider, since, for example, if were interested in ( θn ) i for some i ∈ [b], we could define ψ(θ) = θ i . By the delta method, if θn were an efficient estimate then the asymptotic variance of ψ( θn ) would be ∇ψ(θ 0 ) ⊤ Ω -1 0 ∇ψ(θ 0 ), where Ω -1 0 is the efficient covariance matrix defined in Theorem 3. Therefore, this suggests that we could construct asymptotically calibrated Wald confidence intervals by estimating β⊤ n Ω -1 0 βn , for some data-driven βn . In particular, if ∇ψ(θ 0 ) were known, which would be the case if ψ were linear, then we could do this with βn = ∇ψ(θ 0 ). Otherwise, we could do this using βn = ∇ψ( θn ), where θn is some consistent estimate of θ 0 (such as a VMM estimate), which would be consistent for ∇ψ(θ 0 ) given Assumption 7.
In this section, we provide consistent algorithms for estimating β ⊤ Ω -1 0 β for arbitrary β ∈ R b , with analogous kernel and neural varieties of our algorithms. These consistent variance estimators can then immediately be used with the delta method, as discussed above, to construct asymptotically calibrated Wald confidence intervals for our efficient VMM estimators.
Our algorithms, which are presented in the next subsections, are motivated by the following key lemma.
Lemma 9 Let Ω 0 be defined as in Theorem 3, let the conditions of Theorem 3 hold, and let ∇ρ(X; θ) ∈ R m×b denote the Jacobian of ρ(X; θ) with respect to θ. Then, for any vector β ∈ R b , we have
The first part of this lemma follows by applying a similar variational reformulation argument as in the proof of Lemma 1, and the second part follows by applying a similar argument again on the γ ⊤ Ω 0 γ term, given the definition of Ω 0 from Theorem 3. More details are given in the Appendix.
We note that the right hand side of Lemma 9 has a very similar structure to the game objective of our VMM algorithms. Given this, the previous argument suggests that the asymptotic variance of any such ψ( θn ) could be estimated using approaches similar to our kernel and neural VMM estimation algorithms presented previously. In the remainder of this section, we build on this intuition and present kernel-and neural-based algorithms for inference.
First, we present an inference algorithm along the lines of our kernel VMM estimator. This algorithm is summarized by the following theorem.
Theorem 5 Let the conditions of Theorem 3 be given, and let θn be any corresponding efficient estimate of θ 0 . In addition, let L and Q(θ) be defined as in Lemma 6, and define D(θ) ∈ R (n•m)×b and Ω n ∈ R b×b according to
where α n is any sequence satisfying the assumptions of Theorem 3. Then Ω n → Ω 0 in probability.
We note that an immediate corollary of this theorem is that, for any continuously differentiable ψ and an efficient θn such as our VMM estimators, ∇ψ( θn ) ⊤ Ω - n ∇ψ( θn ) is consistent for the asymptotic variance of ψ( θn ), which is an efficient estimate of ψ(θ 0 ), where Ω - n denotes the pseudo-inverse of Ω n . This follows trivially by the continuous mapping theorem and Slutsky's theorem, since by assumption Ω 0 is invertible. An advantage of this algorithm is that it allows easy estimation of the entire covariance matrix Ω -1 0 , from which the asymptotic variance of any singledimensional function of θn can instantly be estimated without applying any additional variational algorithms.
Our neural inference algorithm is similar in nature to our neural VMM estimator and is given by the following smooth game:
where R n is a regularizer for f and F n is a sequence of neural net classes. Then, given Lemma 9, we expect v n (β) to be a reasonable estimator for β ⊤ Ω -1 0 β. Furthermore, following the argument presented at the beginning of Section 5, we expect v n ( βn ) to be a reasonable estimator for the (efficient) asymptotic variance of ψ( θn ) if βn is consistent for ∇ψ(θ 0 ). We note that, unlike for our neural VMM estimator, we do not provide any theoretical guarantees for this algorithm, due to some additional technical complications; unlike the game objective being solved by neural VMM, the space being minimized over for γ is unbounded, which complicates the technical argument by universal approximation we used for neural VMM. We leave this theoretical question to future work. However, we note that in our inference experiments in Section 7 this method seems to work well.
Unlike our kernel inference algorithm, this approach has the disadvantage that it requires solving a separate optimization problem for every given scalar parameter ψ. In practice, though, this may be alleviated by the practical strengths of neural methods, as discussed previously. In addition, as with our neural VMM algorithm, we may regularize for example by using a kernel-based norm or the Frobenius norm of {f (Z 1 ), . . . , f (Z n )}, or we may omit this regularization term entirely.
Next, let us provide some concrete examples of our theory, in order to demonstrate how the assumptions for our consistency and asymptotic normality theory may be satisfied. For each example, we do not discuss Assumptions 1, 4, 9, and 10 explicitly, as these govern design choices for the algorithm that can be generically satisfied given the other assumptions.
First, let us consider a specific nonparametric instrumental regression example, which instantiates Example 1 from Section 1. Specifically, we will consider conditions under which our consistency result Theorem 1 applies. Let us consider the data generating process Y = g(T; θ 0 ) + ϵ, where
for some fixed λ > 0. Let us also suppose that g(T; θ) is L(T)-Lipschitz continuous in θ, where E[L(T) 2 ] < ∞, that sup θ∈Θ ‖g(T; θ)‖ ∞ < ∞, and that G = {g( • ; θ) : θ ∈ Θ} is a Donsker class. As one example, these conditions would be satisfied if G were given by the class of all monotonic functions on T such that ‖g(X; θ)‖ ∞ ≤ b for some fixed b < ∞, with the norm on Θ given by ‖θ ′ -θ‖ = ‖g(T; θ ′ ) -g(T; θ)‖ ∞ . As a second example, G could be a norm-bounded RKHS satisfying the conditions of Assumption 1, with the norm on Θ given by the corresponding RKHS norm.
First, given the conditions on Z in this example, Assumption 2 is trivial. Second, given the conditions on the regression class G, along with the assumption that ‖Y‖ ∞ < ∞, Assumption 3 trivially follows by applying Lemma 9.14 in Kosorok (2007). Finally, assuming that the prior estimate θn comes from some arbitrary consistent methodology, then Assumption 5 only needs to hold for θ = θ 0 . In this case, this is ensured under the above condition on the conditional variance of ϵ, since
Given this, we have consistency via Theorem 1.
Next, let us consider a specific nonparametric instrumental regression example, which instantiates Example 2 from Section 1. Again, we will consider conditions under which consistency holds. Let us consider the data generating process Y = g(T; θ 0 ) + ϵ, where Prob(ϵ ≤ 0 | Z) = p for almost everywhere Z, and ρ(X; θ) = 1{Y ≤ g(X; θ)} -p. Again, we assume that Z ∈ R dz , and ‖Z‖ ∞ < ∞. In this case, we will assume that G = {g( • ; θ) : θ ∈ Θ} is some regression class that is Donsker under the supremum norm ‖θ ′ -θ‖ = ‖g(X; θ ′ ) -g(X; θ)‖ ∞ , and that Y has bounded density.
Again, given the conditions on Z in this example, Assumption 2 is trivial, and the Donsker part of Assumption 3 follows from the fact that G is Donsker by Lemma 9.14 of Kosorok (2007). Also, we have E[|ρ(X; θ ′ ) -ρ(X; θ)|] = Prob( min (g(X; θ ′ ), g(X; θ)) ≤ Y ≤ max (g(X; θ ′ ), g(X; θ)). Now, since by assumption Y has bounded density, it easily follows that there exists some constant L such that Prob( min (g(X; θ ′ ), g(X; θ)) ≤ Y ≤ max (g(X; θ ′ ), g(X; θ)) ≤ L‖g(X; θ ′ ) -g(X; θ)‖ ∞ , which gives us the required Lipschitz continuity under L 1 norm. Also, the required boundedness is trivial since |ρ(X; θ)| ∈ { -p, 1 -p}, so we have Assumption 3. Finally, we have
surely, and therefore ‖V(Z; θ 0 ) -1 ‖ ∞ ≤ (pp 2 ) -1 < ∞, which gives us Assumption 5, again as long as the prior estimate θn is consistent. Therefore, again we have consistency via Theorem 1.
Next, we will consider a parametric expectile (including mean) regression example (Newey & Powell, 1987;Sobotka et al., 2013), where we can establish both consistency, asymptotic normality, and efficiency. For this example, we will assume that the data generating process is again given by Y = g(T; θ 0 ) + ϵ, where ϵ instead satisfies pE[ϵ1{ϵ
for some p ∈ (0, 1). For p = 0.5, we get the usual mean regression. Here, we let ρ(X; θ) = w(X; θ)(Y -g(T; θ)), where w(X; θ) = p1{Y ≥ g(T; θ)} + (1 -p)1{Y < g(T; θ)}, and the goal is to find the unique θ 0 ∈ Θ such that E[ρ(X; θ 0 ) | Z] = 0. Note that this problem that can be seen as a mid-point between standard instrumental variable regression and instrumented quantile regression. Similar to the above example, let us suppose that Z ∈ R d z , ‖Z‖ ∞ < ∞, ‖Y‖ ∞ < ∞, and V[ϵ | Z] ≥ λ almost surely, for some fixed λ > 0. For this example, we will further assume that T ∈ R dt , ‖T‖ ∞ < ∞, the regression class is given by g(t; θ) = θ ⊤ t, where Θ = {θ ∈ R dt : ‖θ‖ 2 ≤ b} for some b < ∞, and that the matrix
Again, given the conditions on Z in this example, Assumption 2 is trivial. Similarly, given the boundedness of Θ and T, and the fact that ‖w(X; θ)‖ ∞ ≤ 1, along with Lemma 9.14 of Kosorok (2007), we easily have that Assumption 3 holds. In addition, we have V(Z; θ 0 ) ≥ min (p, 1 -p) 2 V[ϵ | Z], and so Assumption 5 follows from our minimum conditional variance assumption as in the previous example. Therefore, we can establish consistency via Theorem 1.
Next, under the additional assumption that Y and T both have bounded probability density, then so does Yg(T; θ) for every θ ∈ Θ. Therefore, we can apply Lemma 3 with ϕ(X; θ) = Yg(T; θ) in order to establish Assumption 6 with D(X; θ) = T, which we note trivially satisfies the conditions of Assumption 7. Finally, since
> 0 for every non-zero β, which establishes Assumption 8. Therefore, we also have asymptotic normality via Theorem 2, and under the condition that the prior estimate θn was consistent we have semiparametric efficiency via Theorem 3.
We now present a series of experiments to demonstrate our proposed methodologies. We present two kinds of experiments. First, we test the finite-sample performance of our kernel and neural VMM algorithms on a range of synthetic conditional moment problems. In this experiment, we compare their performance with the classical sieve minimum distance (SMD) approach of Ai and Chen (2003), which is a sieve-based method that has previously been proposed as a semiparametrically efficient approach to solving generic conditional moment problems. In addition, we compare their performance with the recently proposed maximum moment restriction (MMR) algorithm of Zhang et al. (2020), which as discussed in Section 8 is equivalent to the limit of our kernel VMM algorithm in the limit as α n → ∞. Second, we test our proposed inference algorithms on a subset of these scenarios, evaluating the quality of the resulting confidence intervals for different variations of our estimation and inference algorithms. Code for reproducing all experiments is available at https://github.com/CausalML/VMM.
SimpleIV. This is a simple parametric instrumental variable regression scenario, based on a simple data generating process where Z = sin (πU/10), T = -0.75U + 3.5H + 0.14η -0.6, Y = g(T; θ 0 ) + -10H + ϵ.
In this setup, η and H are exogenous iid N (0, 1) variables, and U is an exogenous iid Uniform ( -5, 5) random variable, and each of T, Z, and Y, are scalars. We note that the random variable H introduces endogeneity. Furthermore, we have g(t; θ) = θ 1 + θ 2 t + θ 3 t 2 where θ ∈ R 3 , with the true parameter value given by θ 0 = [0.5, 3.0, -0.5]. In this scenario, the conditional moment equation to be solved is E[Yg(T; θ 0 ) | Z] = 0; that is, we have X = (T, Y, Z), and ρ(X; θ) = Yg(T; θ). Note that in this scenario the relationship between treatment and instruments is nonlinear.
HeteroskedasticIV. This is a more challenging instrumental variable regression scenario, which introduces a more complex nonlinear regression function class and heteroskedastic noise. It follows a similar data generating process to the prior SimpleIV scenario, except here we have
where again η and H are iid N (0, 1) distributed, and each of U 1 and U 2 are iid Uniform( -5, 5) distributed. We also note the 'softplus' activation function is defined according to softplus(x) = log (1 + exp (x)). In this case, we have θ ∈ R 4 , and our regression class is defined according to
That is, our regression class is a smoothed version of a hinge function with slopes θ 3 and θ 4 and hinge point at (θ 1 , θ 2 ). The true parameter value is given by θ 0 = [2.0, 3.0, -0.5, 3.0]. As with our SimpleIV scenario, the conditional moment restriction is given by
We note that although the regression residual is not independent of the instruments Z in this setting, it is mean-indepedent, since
That is, we have heteroskedastic noise with respect to our instruments, which makes achieving efficiency more challenging.
PolicyLearning. Finally, this scenario is based on learning optimal binary treatment policies from surrogate loss reductions, following Bennett and Kallus (2020). Let T ∈ { -1, 1} denote the binary treatment variable, Z denote individual covariates, Y(t) denote the potential outcome for the individual that would occur if (possibly counter to fact) treatment t were assigned, and Y = Y(T) denote the actual outcome. Then, given logged data where treatments were decided using some randomized policy, and some well-specified parametric class of deterministic treatment policies Π = {π θ : θ ∈ Θ}, the task is to estimate the parameters of the optimal policy within Π That is, we wish to estimate θ 0 = arg max θ∈Θ E[Y(π(Z; θ))], where π(z; θ) denotes the treatment assigned by policy π θ given Z = z. For this problem, we assume the following data generating process:
for q n (Z; θ), but is not prescriptive about the methodology for computing Γ n . Given this, we experimented with various SMD estimators, using B-splines for q n (Z; θ), and multiple approaches for Γ n : (1) Identity, in which we simply set Γ n (z) = I ∀z; (2) Homoskedastic, in which we set Γ n = E n [ρ(X; θn )ρ(X; θn ) ⊤ ] ∀z; and (3) Heteroskedastic, in which we fit a diagonal Γ n (Z) by regressing ρ(X; θn ) 2 i on Z for each i ∈ [m] using neural networks. We provide additional details in the Online Supplementary Material.
The OWGMM estimator follows the method described in Section 2.1, for a flexible set of basis functions f 1 , . . . , f k . As with the SMD method we chose these sets of basis functions using B-splines, as this allowed for a very rich and flexible class of moment conditions. Again, we provide additional details in the Online Supplementary Material.
NCB. Finally, we implemented a simple non-causal baseline (NCB) that estimates θ 0 by ignoring Z and instead trying to solve E[ρ(X; θ 0 ) | X] = 0. For example, for our instrumental variable regression scenarios, this corresponds to assumption that there is no endogeneity in the treatments T. For this baseline, we simply minimize the objective
, which we implement using L-BFGS.
For each scenario and each n ∈ {200, 500, 1,000, 2,000, 5,000, 10,000}, we repeated the following process 50 times: we drew a training set of n random iid data points using the respective scenario's data generating process as well as an additional dataset of n random dev data points for early stopping, hyperparameter tuning, etc., and then we estimated θn using all of our methods and baselines using the sampled dataset. Then, for each combination of scenario and n we computed the mean squared error (MSE) of the estimated θn across these 50 replications. We summarize the results of this process in Table 1. In addition, we computed additional results based on the risk (SimpleIV and HeteroskedasticIV) or regret (PolicyLearning) of the estimated g( • ; θn ). However, these broadly followed the same trend as the main results here, so we leave them to the Online Supplementary Material. In addition, we provide additional tables of results that break down the MSE in terms of bias and standard deviation in the Online Supplementary Material.
Overall, we can see that in all scenarios the best performing methods are our VMM methods, with the neural VMM method performing best in the SimpleIV and PolicyLearning scenarios, and the kernel VMM method performing best in the HeteroskedasticIV scenario. And, in all cases both the kernel and neural VMM methods significantly outperform the baselines. In particular, apart from the easy SimpleIV scenario, in the more complex HeteroskedasticIV and PolicyLearning scenarios, our VMM methods yield errors that are orders of magnitude smaller.
In terms of the values of the regularization hyperparameters, we note that kernel VMM can be sensitive to the choice of α n when it takes extreme values. When α n is too small, the algorithm appears to suffer from high variance and the occasional catastrophically bad results, whereas when α n is too large the estimation becomes very biased, with performance converging to that of MMR. However, for α n in the range of 10 -2 -10 -6 , performance is good across all scenarios and n. It remains a question for future work how to automatically select this hyperparameter using observed data. However, we suspect that approaches based on the eigenvalues of (Q( θn ) + α n L) -1 appearing in Lemma 6 might be productive.
Conversely, we note that our neural VMM algorithm is generally very insensitive to the choice of λ n , with very little change in performance even for relatively large values of λ n , and very strong and stable performance even when λ n = 0. The one minor exception to this is in the challenging PolicyLearning scenario, where using the largest value of λ n results in somewhat better performance than other choices for low values of n, but worse performance for large n. This reinforces the notion that the neural network function class and optimization algorithms are naturally regularizing, and that explicit regularization is not necessarily important.
In general, for both VMM algorithms, we note that there is a wide range of regularization hyperparameter values where performance is generally very good. Furthermore, we note that for both cases the choices of F used were very generic and the same across all scenarios; either an RKHS with a completely generic data-driven kernel, or a very generic shallow MLP. Together, this
α n = 10 -6 10.7 ± 13.6 1.8 ± 1.7 1.6 ± 1.0 1.6 ± .78 1.9 ± .57 2.1 ± .47
4 ± .93 2.6 ± .65 2.8 ± .53 α n = 10 -2 4.2 ± 3.5 3.9 ± 1.9 4.3 ± 1.6 4.1 ± 1.0 4.6 ± .74 4.8 ± .71 α n = 1 6.9 ± 4.8 8.2 ± 2.8 8.7 ± 2.1 8.4 ± 1.8 8.6 ± 1.1 8.6 ± .99 N-VMM λ n = 0 >100 53.4 ± 88.6 6.6 ± 12.0 1.1 ± .77 .50 ± .33 .92 ± .39 λ n = 10 -4 >100 >100 7.9 ± 18.0 1.1 ± .70 .54 ± .46 .92 ± .47 (continued)
suggests that VMM can generally do very well with generic choices for all hyperparameters and is not very sensitive to these choices as long as they do not take extreme values.
In the SimpleIV scenario, where E[ρ(X; θ) | Z] is very simple and easy to fit uniformly over Θ, the SMD and OWGMM baselines performed competitively with our VMM algorithms. However, in the other more challenging scenarios, their behaviour was generally inconsistent and poor. We note that although the average squared error obtained by these methods was extremely high, this seems to be mostly dominated by some outliers, and the typical performance was much more reasonable. For example, in the HeteroskedasticIV scenario when n = 10,000, the median squared error of the Identity, Homoskedastic, and Heteroskedastic versions of SMD were 48.3, 10.7, and 0.22, respectively, which is much less bad than the average squared error. This is also evident, for example, from the separate bias and standard deviation results in the Online Supplementary Material. We also note that for both SMD and OWGMM algorithms, we experimented with a wide range of choices for the underlying sieve basis sets, including the number of knots and polynomial degree for the B-splines that we used, as well as ridge-regularization values, and the results presented are for the least-bad choices. We speculate that the superior performance of our approach is due to the kernel-based regularization of the critic class, which in practice is better able to approximate the efficient instruments with good accuracy and stability. Indeed, it is plausible that sieve-based approaches could also achieve competitive performance using better choices of sieves, with appropriate regularization. In general, however, the use of such sieve spaces, rather than simple linear sieves with optional ridge regularization, as we experimented with, is either intractable or redundant. In the case of SMD, the corresponding sieve estimates q n (z; θ) for E[ρ(X; θ) | Z = z] would no longer have closed-form solutions in θ in general, and we would somehow have to solve a bi-level optimization problem. On the other hand, if we were to introduce such regularization to the sieve space that implicitly arises from the variational reformulation of OWGMM, we would just end up with our VMM approach as in Equation ( 2).
On the other hand, we see that the MMR baseline performed in a way that was relatively very stable, but consistently sub-optimal. The results of MMR were in general similar to kernel VMM with the largest choices for α n , which is expected given that it is equivalent to kernel VMM with α n → ∞. In addition, as expected, the non-causal baseline is consistently biased with very poor performance.
Finally, we provide a breakdown of these mean squared error results in terms of bias and variance in the Online Supplementary Material. One interesting observation there is that some methods, in particular our neural VMM algorithm and the OWGMM baseline, do not display the expected behaviour of bias vanishing at a more rapid rate than standard deviation; rather, even though both shrink, their ratio often remains approximately constant. This could be explained by a couple of factors. First, in the case of neural VMM we are not exactly optimizing the minimax optimization problem; rather, we are trying to approximate this using an alternating gradient ascent/descent approach. Therefore, such discrepancies may be explained by this deviation from theory in the practical implementation of the algorithm. Note that issue does not exist for kernel Note. We write >100 whenever the MSE or standard error was greater than 100.
VMM, which performs the optimization over F n analytically. Second, in the case of OWGMM, this discrepancy seems to be explained by the instability and poor performance described above. This could be be interpreted as 'finite sample' behaviour, reflecting the fact that we are not yet in the asymptotic regime for this method. Alternatively, it may reflect intractable bias due to approximation errors of the sieve basis for the efficient instruments.
SimpleIV. Our first considered scenario for our inference experiments is based on the same SimpleIV scenario as in our estimation experiments. For this scenario here, our target for inference is the instantaneous treatment effect at T = 0; that is, we wish to estimate ψ(θ 0 ) where
HeteroskedasticIV. For our second inference scenario, we consider again the same HeteroskedasticIV scenario from our prior estimation experiments. Here, our target for inference is the change in slope in the true hinge function g( • ; θ 0 ). This corresponds to the function ψ(θ) = θ 4 -θ 3 .
Kernel inference. For our kernel inference method, we implemented the algorithm described by Theorem 5. We used the same kernel function as for our Kernel VMM estimation algorithm in our prior estimation experiments, and we present results for a variety of values of α n .
Neural inference. For our neural inference method, we solved the game objective described by Equation ( 9). We used the same choice of F n and a similar alternating SGD optimization procedure as for our NeuralVMM estimation method. We provide additional details in the Online Supplementary Material. As in our estimation experiments, we used Frobenius norm regularization, and we present results for varying values of λ n .
For each scenario and each n ∈ {200, 2,000}, we repeated the following procedure 200 times: (1) we drew a training set of n random iid data points using the respective scenario's data generating process; (2) we estimated θn using each of our VMM methods; and (3) we estimate the efficient asymptotic variance using each of our inference methods and each of the estimated θn as plug-ins. That is, for each random draw of data, we estimate the efficient asymptotic variance using each combination of VMM estimation method and inference method. In all cases, we compute an estimated 95% confidence interval as where v is the estimated asymptotic variance of ψ( θn ) via the delta method, which we computed using the corresponding inference method as detailed in Section 5. In addition, for each combination of n, scenario, estimation method, and inference method, we computed the following summary statistics: (1) the coverage rate of our estimated confidence intervals; (2) the corresponding coverage when we adjust the confidence intervals by subtracting the bias of ψ( θn ) (which we estimated by 1 200
, where θ(i) n denotes the estimate from the i'th replication); and (3) the 5%, 50%, and 95% percentiles of the estimated standard deviation of ψ( θn ) (given by ���� � v/n √ ) across the 200 replications. Given our previous results that kernel VMM performed very consistently with α n in the range of 10 -2 -10 -6 , for brevity, we only present results in the main paper for when the estimation method is kernel VMM with α n = 10 -4 . However, we present additional results using other estimation methods in the Appendix. We summarize the results from this procedure in Table 2.
Overall, we see that in both scenarios, the results are very good when n = 2,000, with very accurate estimates of the standard deviation of ψ( θn ), and high coverage. For the HeteroskedasticIV scenario, all inference methods produce almost perfect (95%) coverage when n = 2,000, and for the SimpleIV scenario the coverage is only slightly lower, and becomes very close to 95% when bias of θn is taken into account. Note. For each inference method and value of n, we list: Cov the coverage of the respective 95% confidence intervals; CovBC the corresponding bias-corrected coverage, by subtracting the bias of ψ( θn ) from the confidence intervals; and PredSD(q) the q'th percentile of the estimated standard deviation of ψ( θn ), for q ∈ {5, 50, 95}.
When n = 200, our inference results are slightly poorer. This likely reflects several distinct issues when n is small: the bias of θn may be significant, the variance may not be well characterized by the asymptotic variance and the tails by normal tails, and the estimates of the asymptotic variance of ψ( θn ) may be poor. Any of these issues may lead to invalid confidence intervals and lower than expected coverage. Indeed, we can see some or all of these issues at play in our results. In Table 2a we see that coverage is very good when we account for bias, and that the range of the predicted standard deviation of θn is reasonably close to the empirically observed standard deviation of 0.34, which suggests we are suffering from the first issue. Conversely, in Table 2b we see that, even accounting for bias, the coverage is lower than expected when n = 200, and that the range of predicted standard deviations of ψ( θn ) is low compared to the empirically observed standard deviation of 1.9, which suggests that we are suffering from the second and/or third issues.
Regarding the difference in performances between our inference methods, we observe that, as expected given Theorem 5, larger values of α n for our kernel method lead to wider confidence intervals. For n = 2,000, where our asymptotic theory seems to be more relevant, we see very overly wide confidence intervals for our kernel method when α n is very large, with typically good results when α n takes the same range of values that worked well for estimation in our prior experiments (i.e., in the range of 10 -6 -10 -2 ). This suggests that we can tune α n for estimation, and use similar values for inference, and also that we can err on the side of caution and wider confidence intervals by using larger values of α n . Conversely, we found our neural inference method to be very insensitive to λ n , and in general we found that it produced relatively narrow confidence intervals, with widths similar to those from our kernel method using the smallest values of α n .
Finally, we make a note to emphasize the fact that biased-corrected coverage values are listed merely so we can analyze, in cases where coverage is poor, to what extent this is due to bias in the estimate θn , versus due to poor estimates of the standard deviation of θn . Indeed, the biascorrection we perform is not something that can be done in practice, and these bias-corrected coverages should not be interpreted as actual coverages that can be obtained.
For the general conditional moment problem, one classical approach is to solve Equation (3) using a growing sieve basis expansion for {f 1 , . . . , f k } based on, e.g., splines, Fourier series, or power series (Chamberlain, 1987). It would be expected, however, that such methods would suffer from curse of dimension issues and therefore their application would be limited to low-dimensional settings. Furthermore, it has been observed in past work (Bennett & Kallus, 2020;Bennett et al., 2019) that methods of this kind can be very unstable and perform very poorly in comparison to VMM estimators.
A very similar method to this is the SMD approach of Ai and Chen (2003), which instead uses a growing sieve basis expansion to approximate the conditional function E[ρ(X; θ) | Z = z] for every θ ∈ Θ. They propose to minimize a loss of the form J n (θ) = E n [ q(Z; θ) ⊤ Γ(Z) -q(Z; θ)], where q(z; θ) is the sieve estimate for E[ρ(X; θ)
One nice feature of this kind of approach is that it can readily handle infinite-dimensional nuisance components. In the case that θ can be partitioned as θ = (β, γ), where β is a finite-dimensional parameter of interest and γ is an infinite-dimensional functional nuisance component, Ai and Chen (2003) propose to model γ using a second growing sieve basis expansion, and minimize J n (θ) over both β and the sieve coefficients for γ. There is a long line of work on the theoretical efficiency of this kind of approach, even in the presence of infinite-dimensional nuisance components (Ai & Chen, 2003;Chen & Pouzo, 2009, 2012), which is something that our theory does not address. However, these methods have similar practical drawbacks to using a sieve basis expansion for OWGMM, which seems to particularly be the case when the conditional expectation function q(z; θ) = E[ρ(X; θ) | Z = z] is complex, as highlighted by the experimental results in this paper. A very similar approach was also proposed concurrently by Newey and Powell (2003), however their approach has the same drawbacks, and furthermore they do not address efficiency.
Another related classical approach is to solve Equation (3) using estimates of the efficient instruments, which are the set of b functions {f * 1 , . . . , f * b } mapping Z to R b , given by f * i (z) j = F * (z) i,j , where
Past work such as Newey (1990Newey ( , 1993) ) provide sufficient conditions for such estimators to be efficient. However, since θ 0 is unknown, such methods require some other method for first-stage estimation of θ 0 , and are likely sensitive to the quality of this method; indeed, if the estimates of f * i are heavily biased due to poor first-stage estimation, it is unclear whether the corresponding moments will be sufficient for identification, let alone efficiency. By contrast, our method is guaranteed to be well behaved as long as our regularized critic class F n can approximate the optimal instruments, regardless of the quality of our first-stage estimate. Furthermore, estimators that have been previously proposed based on this approach (Newey, 1990(Newey, , 1993) ) employ nearest neighbour or sieve methods with similar weaknesses as discussed above.
The continuum GMM estimators of Carrasco and Florens (2000) are theoretically closely related to our proposed kernel VMM estimators. However, the form of their proposed estimators is very different. Suppose that we define some set of functions {f ( • ; t) : t ∈ T} of the form Z 7 ! R m indexed by set T, and we let H T be some Hilbert space of functions in the form T 7 ! R. In addition, define h
and the linear operator
and θn is some prior estimate for θ 0 . Then, Carrasco and Florens (2000) study estimators of the form arg min θ∈Θ ‖((
In that case, we choose T to be an RKHS class F , with functions indexed by themselves, and H chosen as the dual of this RKHS, then it easily follows that the terms C ′ n and h ′ n defined here are equivalent to the terms C n and h n defined in Section 3. However, the form of Tikhonov regularization applied in the inversion of C 1/2 n is slightly different; by Lemma 2 we regularize using (C n + α n I) -1/2 , whereas they regularized using (C 2 n + α n I) -1/2 C 1/2 n . This difference is significant, since our form of regularization gives rise to the simple minimax VMM-style interpretation, whereas theirs does not. Furthermore, their proposed estimators use the index set T = [0, t max ] for some t max > 0, with H T chosen as the L 2 space on T. This choice is much less flexible than ours of using a function class as the index set and makes it more difficult to guarantee that θ 0 is uniquely identified or to guarantee semiparametric efficiency, which they do not. More concretely, the main efficiency claim they provide is that their estimator is efficient compared to other estimators of the form sup θ∈Θ ‖B ′ n h n (θ)‖ 2 , for any choice of bounded linear operator B ′ n . Finally, they propose to solve their optimization problem by computing an explicit rank-n eigenvalue, eigenvector decomposition of C ′ n , and constructing a cost function to minimize based on this decomposition. In particular, if we define g i (θ) ∈ H T according to
and θ ∈ Θ, then their objective function is given by a quadratic form on all terms of the form 〈g i (θ), g j (θ)〉 H T for i, j ∈ [n]. This involves n 4 terms in total, and is therefore very computationally expensive to compute for large n. In comparison, the cost function in θ implied by our kernel VMM estimator could be calculated analytically as a quadratic form in n 2 terms based on the representer theorem. Furthermore, our variational reformulation allows for estimators based on alternating stochastic gradient descent, which may be more practical in some situations, for example when n is large.
Another recently proposed and related class of estimators are given by the adversarial GMM estimators of Lewis and Syrgkanis (2018), which were recently extended to the more general class of minimax GMM estimators by Dikkala et al. (2020). In general, these estimators are defined according to arg min θ∈Θ sup
, where R n is some regularizer on f, and Ψ n is some regularizer on θ. In particular, Dikkala et al. (2020) analyze estimators where F and Θ are both normed function spaces, and the regularizes take the form R n
On the theoretical side, they provide general results bounding the the L 2 distance between E[ρ(X; θn ) | Z] and E[ρ(X; θ 0 ) | Z] for this form of estimator. Furthermore, they propose various specific estimators of this kind, for example with F chosen as a RKHS or a class of neural networks. We note that these are similar to our proposed kernel and neural VMM estimators, with the difference that they do not include the -(1/4)E n [(f (Z) ⊤ ρ(X; θn )) 2 ] term motivated by optimal weighting, and that they explicitly regularize θ. In a sense, the focus of their estimators and theory is very different than ours; we focus on the question of efficiency, and provide theoretical guarantees of efficiency when Θ is finitedimensional, whereas they focus on the case where Θ is a function space, but restrict their analysis to providing finite-sample bounds rather than addressing efficiency. We speculate that the benefits of both kinds of approaches could be combined, and by using both the optimal weighting-based term and regularizing θ one could construct estimators that are semiparametrically efficient when θ 0 is finite-dimensional, and have explicit risk guarantees in the more general setting. However, we leave this question to future work.
Recall that for the instrumental variable regression problem we have X = (Z, T, Y), where T is the treatment we are regressing on, Y is the outcome, and Z is the instrumental variable, and ρ(X; θ) = Yg(T; θ), for some regression function g parameterized by θ. In this setup, Θ may either be a finite-dimensional parameter space, which corresponds to having a parametric model for g, or alternatively we may allow allow Θ to be some infinite-dimensional function space and simply define g(z; θ) = θ(z), which corresponds to performing nonparametric regression.
Perhaps, the most classic method for instrumental variable regression is two-stage least squares (2SLS). First, we perform least-squares linear regression of ϕ(T) on ψ(Z), where ϕ and ψ are finitedimensional feature maps on T and Z, respectively. That is, we learn some linear model h( • ; γn ), where h(z; γ) = γ ⊤ ψ(z), and γn = arg min γ n i=1 ‖ϕ(T i ) -γ ⊤ ψ(Z i )‖ 2 . Then, we again perform least squares linear regression, this time of Y on h(ψ(Z); γn ). That is, we learn a linear model g( • ; θn ), where g(t; θ) = θ ⊤ ϕ(t), and θn = arg min θ n i=1 (Y i -θ ⊤ h(Z i ; γn )) 2 . Under the assumption that these linear models are correctly specified, then the resulting 2SLS estimator is known to be consistent for θ 0 (Angrist & Pischke, 2008, Section 4.1.1). However, such estimators are limited in that they require finding some finite-dimensional feature map ϕ such that the linear model given above is well-specified, which in practice may be infeasible. The sieve methods of Newey and Powell (2003), Ai and Chen (2003) discussed in Section 8.1 applied specifically to the instrumental variable regression problem could be viewed as similar approaches, but using growing sieve basis expansions for ϕ and ψ. However, as discussed already these methods may be problematic in practice.
Alternatively, a couple of recent works propose extending the 2SLS method in the case where both stages are performed using infinite-dimensional feature maps and ridge regularization; i.e., both stages are performed using kernel ridge regression. The Kernel IV method of Singh et al. (2019) proposes to do this in a very direct way, by regressing ϕ(T) on ψ(Z), and then regressing Y on h(ϕ(Z)), where both the feature maps ϕ and ψ are infinite dimensional, and implicitly defined by some kernels K Z and K T under Mercer's theorem. In the case of learning h, this corresponds to solving for a linear operator between two RKHSs and in general is ill-posed, so this regression is performed using Tikhonov regularization. Then, the second-stage problem corresponds to performing RKHS regression using some implicit kernel depending on h, and is performed again using Tikhonov regularization. Ultimately, however, by appealing to the representer theorem the regressions don't need to be performed separately, and there is a simple closed form solution. Similarly, the Dual IV method of Muandet et al. (2019) considers 2SLS using RKHSs for each stage and formulates this as a minimax problem of the form arg min θ∈Θ sup
Ultimately, both this work and that of Singh et al. (2019) propose closed-form estimators that are superficially similar to ours in Lemma 7, but without any terms corresponding to optimal weighting. However, their focus is slightly different to ours; their theoretical analysis where present is in terms of consistency or regret, whereas the focus of our theoretical analysis is semiparametric efficiency.
Bennett and Kallus
The recent Deep IV method of Hartford et al. (2017) proposes to extend 2SLS using deep learning. Specifically, they propose in the first stage to fit the conditional distribution of X given Z, for example using a mixture of Gaussians parametrized by neural networks, or by fitting a generative model using some other methodology such as generative adversarial networks or variational autoencoders. Then, in the second stage, they propose to minimize
, where the conditional expectation Ê[• | z] is estimated using the model from the first stage, and g is parameterized using some neural network architecture. This approach has the advantage of being flexible and building on recent advances in deep learning, however they do not provide any concrete theoretical characterizations, and since the first stage is bound to be imperfectly specified this can suffer from the 'forbidden regression' issue (Angrist & Pischke, 2008, Section 4.6.1). Zhang et al. (2020) recently proposed the maximum moment restriction instrumental variable algorithm. They present multiple estimators for approximately solving arg min θ∈Θ sup
, where F is an RKHS, and Ψ n is an optional regularizer on θ in the case that it is infinite-dimensional (however, they also analyze case where θ is finite-dimensional.) Of particular note, the 'V-statistic' version of their algorithm is equivalent to minimizing
, where ρ(θ) and L are defined as in Lemma 6. Letting J K-VMM n (θ; α) denote our kernel VMM objective with regularization strength α, and assuming Ψ n (θ) = 0, Lemma 6 immediately implies that αJ K-VMM n (θ; α) → J MMR n (θ) as α → ∞. In other words, there is an equivalence between MMR and kernel VMM with infinite regularization. Zhang et al. (2020) provide theory showing that their estimators are consistent and asymptotically normal under various assumptions. However, unlike us, they do not establish efficiency.
Finally, we discuss some past work where VMM estimators have been applied. The original such work was by Bennett et al. (2019), who proposed the DeepGMM estimator for the problem of instrumental variable regression. Specifically, the proposed estimator takes the form arg min θ sup f ∈F E n [f (Z) ⊤ (Yg(T; θ))] -(1/4)E n [f (Z) 2 (Yg(T; θn )) 2 ], where {g( • ; θ) : θ ∈ Θ} and F are both given by neural network function classes. That is, the DeepGMM estimator can be interpreted as a neural VMM estimator for the instrumental variable problem in the form of equation ( 6) with R n (f ) = 0 and fixed F n that does not grow with n. In their experiments DeepGMM consistently outperformed other recently proposed methods (Hartford et al., 2017;Lewis & Syrgkanis, 2018) across a variety of simple low-dimensional scenarios, and it was the only method to continue working when using high-dimensional data where the treatments and instruments were images. In addition, DeepGMM has continued to perform competitively in more recent experimental comparisons (Muandet et al., 2019;Singh et al., 2019). Bennett et al. (2019, Theorem 2) provided conditions under which DeepGM is consistent. In addition, we could also justify that it is asymptotically normal and semiparametrically efficient by Theorem 4, under some additional assumptions and by introducing kernel-based regularization.
In addition, this style of estimator was applied to the problem of policy learning from convex surrogate loss reductions by Bennett and Kallus (2020). A common approach for optimizing binary treatment decision policies from logged cross-sectional data is to construct a surrogate cost function to minimize of the form E n [|ψ|l(g(X; θ), sign(ψ))], where X denotes observed pretreatment information about the individual, ψ is some weighting variable depending on all observed pre-and post-treatment information about the individual, the function g( • ; θ) encodes the policy we are optimizing which we assume is parameterized by θ ∈ Θ, and l is some smooth convex loss function such as logistic regression loss. Bennett and Kallus (2020) showed that the model where this surrogate loss is correctly specified is given by the conditional moment problem E[|ψ|l ′ (g(X; θ), sign(ψ)) | X] = 0, where l ′ is the derivative of l with respect to its first argument. Consequently, they proposed the empirical surrogate loss policy risk minimization (ESPRM) estimator, according to arg min θ∈Θ sup f ∈F E n [f (X)ρ(X, ψ; θ)] -(1/4)E n [f (X) 2 ρ(X, ψ; θn )], where ρ(X, ψ; θ) = |ψ|l ′ (g(X; θ), sign(ψ)), and F is a neural network function classes. That is, again this estimator can be interpreted as a neural VMM estimator as in equation ( 6), with R n (f ) = 0. Not only did the authors demonstrate that this algorithm led to consistently improved empirical performance over the standard approach of empirical risk minimization using the surrogate loss, but they proved that if the resulting estimator θn is semiparametrically efficient, then this implies optimal asymptotic regret for the learnt policy compared with any policy identified by the model given by correct specification. We note that, although the authors did not address the question of how to guarantee such efficiency for θn , we could guarantee it by Theorem 4 under some additional assumptions and kernel-based regularization, or under Theorem 3 by instead using a kernel VMM estimator.
Finally, Bennett et al. (2021) applied this style of estimator to the problem of reinforcement learning using offline data logged from some fixed behaviour policy, also known as the problem of off policy evaluation (OPE). They proposed an algorithm for the OPE problem under unmeasured confounding, which requires as an input an estimate of the state density ratio d between the behaviour policy and the target policy they are evaluating. As stated in Section 1, d can be identified by a conditional moment problem, up to a constant factor, with the normalization constraint E[d(S)] = 1. Bennett et al. (2021) proposed a VMM-style estimator for d, using both the conditional moment condition E[d(S)β(A, S) -d(S ′ ) | S ′ ] = 0 and the marginal moment condition E[d(S) -1] = 0, based on a slightly more general form of Lemma 1 where the vector of conditional moment restrictions can depend on different random variables to be conditioned on. That is, the more general problem is given by the m moment conditions E[ρ i (X; θ 0 ) | Z i ] = 0 for i ∈ [m], for some set of random variables Z 1 ∈ Z 1 , . . . , Z m ∈ Z m . Specifically, they propose a kernel VMM-style estimator, where both d and f are optimized over balls in RKHSs. In practice, by successively applying the representer theorem to this two-stage optimization problem, they presented a closed-form solution for the estimate dn (in a similar vein to Lemma 7). Note that since their kernel VMM estimator is based on a slightly more intricate conditional moment formulation than we considered, with varying conditioning sets, our theoretical analysis may not apply to it. We leave the question of extending our theoretical analysis to this more general problem to future work.
In this paper, we presented a detailed theoretical analysis for the class of VMM estimators, which are motivated by a variational reformulation of the optimally weighted generalized method of moments and which encompass several recently proposed estimators for solving conditional moment problems. We studied multiple varieties of these estimators based on kernel methods or deep learning, and provided appropriate conditions under which these estimators are consistent, asymptotically normal, and semiparametrically efficient. This is in contrast to other recently proposed approaches for solving conditional moment problems using machine learning tools, which do not provide any results regarding efficiency. In addition, we proposed inference algorithms based on the same kind of variational reformulation, again with specific algorithms based on both kernel methods and deep learning. Finally, we demonstrated in a detailed series of experiments that our VMM estimators achieve very strong estimation performance in comparison to relevant baselines and that the confidence intervals we generate are reliable.
Our paper suggests a few immediate directions for future work. First, unlike, e.g., the sieve minimum distance approaches of Ai and Chen (2003); Chen andPouzo (2009, 2012), our efficiency theory when θ 0 is finite-dimensional does not accommodate possible infinite-dimensional nuisance components. Furthermore, as discussed in Section 3, the latter two works allow for weaker assumptions on the smoothness and complexity of ρ(X; θ). We suspect that our theory could be extended accordingly without fundamentally changing the VMM algorithm, but this is left to future work.
Second, we only consider conditional moment restrictions using a single conditioning variable Z. In some settings, such as longitudinal studies or the RL application discussed in Section 8.3, one faces conditional moment problems with different, nested conditioning variables for each conditional moment restriction, and our current theory does not accommodate such formulations. Again, we believe that our theory could naturally be extended to this kind of setting.
Third, we only present theory for neural VMM estimators using a kernel-based regularizer, yet we see compelling empirical results for simpler regularizers. We speculate that under appropriate conditions on the neural net classes F n , our efficiency result in Theorem 4 could be extended to neural VMM estimators with such regularizers.
Next, an important further direction is the automatic selection of the hyperparameter α n for our kernel VMM method and corresponding inference algorithm. We speculate, for instance, that it may be possible to approximate the resulting bias and variance for different values of α n and optimize a bias-variance trade-off. At the same time, work on approximating the bias of our estimator could be helpful for improving the quality of confidence intervals from our proposed inference algorithm, as we observed that in many cases coverage of our confidence intervals significantly improved when they were corrected for bias. Similarly, it is known that continuously updating GMM can have lower bias than k-step GMM algorithms (Hansen et al., 1996), which suggests that we may be able to reduce bias using a continuously updating VMM where instead of using a prior estimate θn in the second term of the game objective we use the same θ that we are optimizing over.
Finally, we hope that this work will help motivate the construction of efficient VMM estimators for other conditional moment problems.
JR Stat Soc Series B: Statistical Methodology, 2023, Vol. 85, No. 3
Downloaded from https://academic.oup.com/jrsssb/article/85/3/810/7146121 by Cornell University user on 02 October 2023
Bennett and Kallus Downloaded from https://academic.oup.com/jrsssb/article/85/3/810/7146121 by Cornell University user on 02 October 2023
Bennett and Kallus
Bennett and Kallus Downloaded from https://academic.oup.com/jrsssb/article/85/3/810/7146121 by Cornell University user on 02 October 2023
Bennett and Kallus