<?xml-model href='http://www.tei-c.org/release/xml/tei/custom/schema/relaxng/tei_all.rng' schematypens='http://relaxng.org/ns/structure/1.0'?><TEI xmlns="http://www.tei-c.org/ns/1.0">
	<teiHeader>
		<fileDesc>
			<titleStmt><title level='a'>Brain kernel: A new spatial covariance function for fMRI data</title></titleStmt>
			<publicationStmt>
				<publisher></publisher>
				<date>12/01/2021</date>
			</publicationStmt>
			<sourceDesc>
				<bibl> 
					<idno type="par_id">10319989</idno>
					<idno type="doi">10.1016/j.neuroimage.2021.118580</idno>
					<title level='j'>NeuroImage</title>
<idno>1053-8119</idno>
<biblScope unit="volume">245</biblScope>
<biblScope unit="issue">C</biblScope>					

					<author>Anqi Wu</author><author>Samuel A. Nastase</author><author>Christopher A. Baldassano</author><author>Nicholas B. Turk-Browne</author><author>Kenneth A. Norman</author><author>Barbara E. Engelhardt</author><author>Jonathan W. Pillow</author>
				</bibl>
			</sourceDesc>
		</fileDesc>
		<profileDesc>
			<abstract><ab><![CDATA[A key problem in functional magnetic resonance imaging (fMRI) is to estimate spatial activity patterns from noisy high-dimensional signals. Spatial smoothing provides one approach to regularizing such estimates. However, standard smoothing methods ignore the fact that correlations in neural activity may fall off at different rates in different brain areas, or exhibit discontinuities across anatomical or functional boundaries. Moreover, such methods do not exploit the fact that widely separated brain regions may exhibit strong correlations due to bilateral symmetry or the network organization of brain regions. To capture this non-stationary spatial correlation structure, we introduce the brain kernel , a continuous covariance function for whole-brain activity patterns. We define the brain kernel in terms of a continuous nonlinear mapping from 3D brain coordinates to a latent embedding space, parametrized with a Gaussian process (GP). The brain kernel specifies the prior covariance between voxels as a function of the distance between their locations in embedding space. The GP mapping warps the brain nonlinearly so that highly correlated voxels are close together in latent space, and uncorrelated voxels are far apart. We estimate the brain kernel using resting-state fMRI data, and we develop an exact, scalable inference method based on block coordinate descent to overcome the challenges of high dimensionality (10-100K voxels). Finally, we illustrate the brain kernel's usefulness with applications to brain decoding and factor analysis with multiple task-based fMRI datasets.]]></ab></abstract>
		</profileDesc>
	</teiHeader>
	<text><body xmlns="http://www.tei-c.org/ns/1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink">
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="1.">Introduction</head><p>An important problem in neuroscience is to characterize the covariance of high-dimensional neural activity. Understanding this covariance structure could provide insight into the brain's functional organization and help regularize estimates of encoding or decoding models. Although advances have been made in both theory and methodology for estimating large covariance <ref type="bibr">Bickel and Levina (2008)</ref> ; <ref type="bibr">Sch&#228;fer et al. (2005)</ref> and precision matrices <ref type="bibr">Hsieh et al. (2013)</ref> ; <ref type="bibr">Treister and Turek (2014)</ref> , few methods have been designed with the particular challenges of functional magnetic resonance imaging (fMRI) data in mind (see <ref type="bibr">Varoquaux et al. (2010)</ref> for an exception).</p><p>One of the challenges of modeling the covariance of fMRI data is that the spatial discretization of the brain may differ across experiments. FMRI measures blood oxygenation level dependent (BOLD) signals in discrete spatial regions called "voxels ". Each voxel represents a tiny cube of brain tissue. Although brains are typically registered to an anatomical from a generative model of fMRI data, and seeks to describe covariance of neural signals across the entire brain <ref type="bibr">Stein (2012)</ref> . Specifically, the brain kernel defines, for any finite set of &#119899; voxel locations in the brain, a positive definite &#119899; &#215; &#119899; covariance matrix over &#119899; -dimensional vectors of neural activity at those locations. The brain kernel improves upon prior work by i) capturing fMRI voxel covariance matrices for any registration reference, and ii) capturing spatial nonstationarity through a nonlinear latent manifold.</p><p>Our approach uses a Gaussian process to parametrize a continuous nonlinear mapping from 3D brain coordinates to a latent embedding space, such that correlations in neural activity fall off as a fixed function of distance in the latent space. Thus, the nonlinear function seeks to warp the 3D brain in order to place locations with correlated neural activity at nearby locations in the latent space. Locations with uncorrelated activity, conversely, are mapped to more distant points in the latent space, even if they are physically close together in the brain.</p><p>The paper is organized as follows. In Section 2 we provide a brief overview of Gaussian process models. In Section 3 , we formally introduce the brain kernel model for fMRI data. In Section 4 , we describe an efficient inference method for fitting the brain kernel, and illustrate the challenges and benefits of an exact inference method using simulated and real fMRI datasets. In Section 5 , we describe the brain kernel fit to whole-brain resting-state fMRI data. Finally, in Section 6 , we demonstrate the usefulness of the inferred brain kernel with applications to decoding and factor modeling.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="2.">Mathematical background</head><p>Before introducing the brain kernel model, we briefly review the mathematical building blocks for Gaussian process models.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="2.1.">Gaussian processes (GPs)</head><p>Gaussian processes provide a flexible and tractable prior distribution over nonlinear functions <ref type="bibr">Rasmussen (2004)</ref> . A GP is parametrized by a mean function &#119898; ( &#119857;) , which specifies the mean value of the function &#119891; ( &#119857;) at input point &#119857;, and a covariance function &#119896; ( &#119857; 1 , &#119857; 2 ) , which specifies cov ( &#119891; ( &#119857; 1 ) , &#119891; ( &#119857; 2 ) ) , the covariance between the function values &#119891; ( &#119857; 1 ) and &#119891; ( &#119857; 2 ) , for any pair of inputs &#119857; 1 and &#119857; 2 .</p><p>Technically a GP is a random process for which the values taken at any finite set of input points has a well-defined multivariate Gaussian distribution. The mean and covariance of that Gaussian are given by evaluating the mean and covariance functions at the corresponding set of input points. Let ( &#119857; 1 , &#8230; , &#119857; &#119899; ) denote a collection of &#119899; points in the input domain. If a function &#119891; has a Gaussian process distribution, &#119891; &#8764; &#57907;&#57916; ( &#119898;, &#119896; ) , then the vector of function values &#119839; = ( &#119891; ( &#119857; 1 ) , &#8230; , &#119891; ( &#119857; &#119899; ) ) &#8868; has a multivariate Gaussian distribution:</p><p>where &#119846; = ( &#119898; ( &#119857; 1 ) , &#8230; , &#119898; ( &#119857; &#119899; )) &#8868; is the mean vector, and &#119818; is the ( &#119899; &#215; &#119899; ) covariance matrix whose &#119894;, &#119895;'th element is &#119896; ( &#119857; &#119894; , &#119857; &#119895; ) .</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="2.2.">GP regression</head><p>A common application of GPs is to predict function values at test points given a set of training data consisting of observed inputs and function values. In GP regression, these predictions come from the conditional distribution over unknown function values given the observed values.</p><p>Let &#119831; = ( &#119857; 1 , &#8230; , &#119857; &#119899; ) denote a set of &#119899; input points and let &#119839; = ( &#119891; ( &#119857; 1 ) , &#8230; , &#119891; ( &#119857; &#119899; ) ) &#8868; denote the vector of function values observed at these points. Here we assume the observed data is noiseless, and we will consider noisy observations in Section 3 . We consider a set of By applying the standard formula for Gaussian conditional distributions, we obtain the following conditional distribution of &#119839; * given &#119839; :</p><p>where mean and covariance are given by</p><p>&#120590; 2 ( &#119831; * ) = &#119818; * * -&#119818; * &#119899; &#119818; -1 &#119899;&#119899; &#119818; &#119899; * .</p><p>(5)</p><p>GP regression uses eq. 4 , the posterior mean of the function given the training data, to predict function values at test points &#119831; * given the training data { &#119831; , &#119839; } . Although we have assumed so far that the function &#119891; is scalar-valued, we can extend the GP regression framework to vector-valued functions by using a separate GP for each output dimension of &#119891; .</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="3.">The brain kernel model</head><p>Here we introduce the brain kernel (BK) model, which is a probabilistic model of fMRI measurements at an arbitrary set of 3D spatial voxel locations. The brain kernel itself is a covariance function for neural activity that arises under this BK model, which we will infer from registered fMRI data.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="3.1.">Nonlinear embedding function</head><p>The first component of the brain kernel model is a nonlinear function, &#119891; &#8758; &#8477; 3 &#10230; &#8477; &#119889; , which provides a continuous nonlinear mapping from 3D brain coordinates to a &#119889;-dimensional latent embedding space. The goal of this mapping is to embed brain regions with similar activity at nearby locations in the embedded space. Typically, we consider &#119889; &gt; 3 , so that the embedding is higher-dimensional than the three physical dimensions of the brain. This gives the embedding flexibility to capture complex non-smooth dependencies between brain regions. One example of such a dependency is the functional symmetry of the two hemispheres Di <ref type="bibr">Lollo (1981)</ref> ; <ref type="bibr">Kitterle and Kaye (1985)</ref> ; <ref type="bibr">Westcott (1973)</ref> , which suggests that one might wish to map symmetric points on the two hemispheres to nearby points in the embedded latent space. This would not be possible with a continuous mapping in three dimensions. But, in four dimensions, one can fold the three-dimensional brain along the fourth dimension, analogous to the way that folding a 2D brain slice along the mid-line would allow for close alignment of symmetric points from the two hemispheres.</p><p>Let &#119857; &#8712; &#8477; 3 denote an input vector, specifying the three-dimensional location of a voxel in the brain, and let &#119859; &#8712; &#8477; &#119889; denote the output of &#119891; , so that &#119859; = &#119891; ( &#119857;) is the &#119889;-dimensional embedding location of a voxel at &#119857;. Thus, for a set of voxel locations &#119831; = ( &#119857; 1 , &#8230; , &#119857; &#119899; ) , the embedded locations in the latent space are &#119833; = ( &#119859; 1 , &#8230; , &#119859; &#119899; ) = ( &#119891; ( &#119857; 1 ) , &#8230; , &#119891; ( &#119857; &#119899; ) ) . To impose smoothness on the embedding function, we place a GP prior on &#119891; . We use a linear mean function, &#119898; ( &#119857;) = &#119809;&#119857;, where &#119809; is a &#119889; &#215; 3 matrix. This choice ensures that the embedding defaults to a linearly stretched version of the brain in the absence of likelihood terms. Because &#119891; is a vector function with outputs of dimension &#119889;, the mean function output is also &#119889;-dimensional. For the covariance function, we use a Gaussian or "radial basis function " (RBF) covariance for each output dimension of &#119891; :</p><p>The hyperparameters governing this covariance function consist of a marginal variance &#119903; and a length-scale &#120575;, which control the range and smoothness of &#119891; , respectively. The GP prior over each output dimension of &#119891; can therefore be written as</p><p>where &#119891; &#119895; ( &#8901;) denotes the &#119895;th output dimension of the function &#119891; ( &#8901;) , and &#119835; &#119895; is the &#119895;th row of the matrix &#119809; . The prior is therefore governed by a set of hyperparameters denoted &#120579; &#119891; = { &#119809; , &#119903;, &#120575;} . Thus, all output dimensions of the function &#119891; are assumed a priori independent with the same covariance function and differing mean functions. This GP prior over the function &#119891; implies a multivariate normal prior over any set of embedded voxel locations &#119833; . Let &#119859; &#119895; denote the &#119895;th latent embedding of the entire set of brain voxels in the training data. Then the prior over &#119859; &#119895; given the true voxel locations &#119831; is</p><p>where &#119818; &#8712; &#8477; &#119899; &#215;&#119899; is the covariance matrix with the &#119894;, &#119895;th element given by &#119896; ( &#119857; &#119894; , &#119857; &#119895; ) (eq. 6 ).</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="3.2.">From embedding space to neural activity</head><p>The second component of the brain kernel model is a probability distribution over neural activity as a function of locations in embedding space. Our modeling assumption is that neural activity changes smoothly as a function of locations in embedding space, or equivalently, that correlations in neural activity decrease smoothly with distance in latent space. We formalize this assumption using the brain kernel, which provides a mapping from latent embedding locations to a covariance matrix for neural activity.</p><p>Let &#119855; &#8712; &#8477; &#119899; denote a vector of neural activity from &#119899; voxels with positions &#119831; = ( &#119857; 1 , &#8230; , &#119857; &#119899; ) and latent embedding locations &#119833; = ( &#119891; ( &#119857; 1 ) , &#8230; , &#119891; ( &#119857; &#119899; )</p><p>)</p><p>. The BK model assumes this neural activity vector has a multivariate Gaussian distribution with zero mean and covariance determined by the brain kernel:</p><p>where</p><p>is the covariance matrix, which results from applying the brain kernel &#120581; &#119861;&#119870; ( &#8901;, &#8901;) to every pair of voxel locations ( &#119857; &#119894; , &#119857; &#119895; ) in the set &#119831; .</p><p>The brain kernel itself is the bivariate function &#120581; &#119861;&#119870; &#8758; &#8477; 3 &#215; &#8477; 3 &#10230; &#8477; from pairs of 3D voxel locations to a covariance of neural activity at those pairs of locations:</p><p>where &#120588; is the marginal variance. The length-scale is omitted here because &#119859; is an unknown latent variable that we need to optimize which absorbs the unknown length-scale for simplicity. The brain kernel therefore specifies a positive semidefinite covariance matrix for neural activity at any set of 3D voxel locations, which is a function of the embedded latent locations of those voxels via the nonlinear function &#119891; . The brain kernel model transforms the data representation from voxel space to the latent embedding space, and this transformation explicitly estimates the covariance over neural activity ( Fig. <ref type="figure">1</ref> ).</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="3.3.">From neural activity to BOLD signal</head><p>Next, we assume that the experimenter does not directly measure the neural activity vector &#119855; , but instead receives measurements corrupted by independent Gaussian noise. If &#119858; denotes the vector of fMRI measurements, we assume &#119910; &#119894; = &#119907; &#119894; + &#120585; &#119894; , where &#119894; is the index for voxels and &#120585; &#119894; &#8764; &#57914; (0 , &#120590; 2 ) represents measurement noise. The vector &#119858; denotes a single measurement of neural activity for all voxels. This induces the following marginal distribution over fMRI measurements given the embedding:</p><p>Note however that the brain kernel generates &#119810; &#119899;&#119899; , the covariance of the underlying neural activity &#119855; , as opposed to the covariance of the noisy fMRI measurements &#119858;; the covariance of these measurements is ( &#119810; &#119899;&#119899; + &#120590; 2 &#119816; &#119899; ) . For simplicity, we will omit the subscript in &#119810; &#119899;&#119899; in the following text and write simply &#119810; .</p><p>The full set of hyperparameters governing the brain kernel model are therefore &#120579; = { &#119809; , &#119903;, &#120575;, &#120588;, &#120590; 2 } , where { &#119809; , &#119903;, &#120575;} describe the nonlinear embedding function, &#120588; is the marginal variance of neural activity, and &#120590; 2 is the variance of additive Gaussian noise.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.">Inference methods</head><p>To fit the brain kernel model, we estimate the latent embeddings of all voxels &#119833; as well as the model hyperparameters &#120579; = { &#119809; , &#119903;, &#120575;, &#120588;, &#120590; 2 } given a time series of whole-brain fMRI measurements &#119832; as well as voxels' 3D locations &#119831; . More specifically, &#119832; = ( &#119858; 1 , &#8230; , &#119858; &#119879; ) &#8712; &#8477; &#119899; &#215;&#119879; , where &#119858; &#119905; &#8712; &#8477; &#119899; refers to the vector of fMRI measurements at time in-</p><p>We propose two empirical estimators: maximum a posteriori (MAP) and penalized least squares (PLS). Briefly, MAP solves the problem using conventional Bayesian probabilistic inference. Since we have already defined the distribution to generate fMRI measurements from the latent embeddings (eq. 12 ) and the prior for the latent embeddings (eq. 8 ), we can estimate the latent embeddings &#119833; using MAP methods. PLS formulates an objective function by minimizing the squared error between the sample covariance of the data cov ( &#119832; ) , and the model-defined covariance &#119810; (eq. 10 ). The goal of both inference methods is to find the latent embedding locations &#119833; and the model hyperparameters such that the neural activity covariance &#119810; resembles the sample covariance of &#119832; as closely as possible.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.1.">Maximum a posteriori (MAP) estimation</head><p>Estimating large covariance matrices is a fundamental problem in modern multivariate analysis. One common approach uses maximum likelihood methods. We have already defined the data distribution &#119832; given the latent embedding &#119833; in eq. 12 and described the prior over &#119833; in eq. 8 .</p><p>The joint distribution is then</p><p>where &#119810; is a function of &#119833; and &#120588;. Thus, the loss function for the maximum a posteriori (MAP) estimator is A . The nonlinear latent embedding of the 3D coordinates into a &#119889;-dimensional latent space using a function &#119891; . This &#119891; function is sampled from a GP prior. B . Given the voxel locations and the BOLD activity of these two locations in an fMRI dataset (top right), our goal is to construct the brain kernel with the locations as inputs that matches their covariance of the BOLD activity (bottom right). The covariance is equal to the Euclidean-based kernel of the voxels' embedding in the latent space.</p><p>&#119826; is the sample covariance of measured neural activity, defined as</p><p>Minimizing &#57912; MAP w.r.t. &#119833; and &#120579;, we derive the MAP estimators &#7824; MAP and &#120579; MAP .</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.2.">Penalized least squares (PLS) estimation</head><p>Another common approach aims at finding an estimator that resembles the sample covariance while also satisfying structural assumptions about the data <ref type="bibr">Fan et al. (2008</ref><ref type="bibr">Fan et al. ( , 2016) )</ref> . In prior work, Fan et al. <ref type="bibr">Fan et al. (2016)</ref> showed that a generalized thresholding covariance estimator can be cast as a penalized least squares (PLS) problem:</p><p>where &#119877; ( &#8901;) is a penalty function that imposes structure on the covariance matrix &#119810; . Sparsity in &#119810; is often encoded using a shrinkage penalty. However, we abandon sparsity in the brain kernel because the covariance of brain activity may have dense structures. Instead, we regularize the latent subspace of the covariance matrix using the normal prior on the latent embedding &#119833; (eq. 8 ), which is a Bayesian regularization of the log likelihood with the form tr</p><p>. This is equivalent to an &#119897; 2 -norm penalty on &#119818; -1 2 ( &#119833; -&#119809;&#119831; ) &#8868; . Including the noise variance term &#120590; 2 , the loss function for the empirical estimator is</p><p>Minimizing &#57912; PLS w.r.t. &#119833; and &#120579;, we derive the empirical PLS estimators &#7824; PLS and &#120579; PLS . Here, the least square term estimates the embedding &#119833; using the sample covariance, while the second term regularizes &#119833; with the kernel matrix &#119818; and mean values &#119809;&#119831; constructed from &#119831; . Eq. 18 can also be considered as inheriting the log of the GP prior from eq. 15 and replacing the data likelihood term with a squared loss.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.3.">Exact inference by block coordinate descent</head><p>Calculating the log posterior (eq. 15 ) for MAP inference has a computational complexity of &#119874;( &#119899; 3 ) , due to the need to compute the inverse and the determinant of the covariance matrix. This cost is often impractical in fMRI settings, where the number of voxels &#119899; may be on the order of thousands to hundreds of thousands.</p><p>To optimize &#119833; and the model hyperparameters, we could use gradient descent or Newton's method as the optimizer; however, this approach is computationally impractical. Thus, we need to consider a scalable inference method. Existing scalable inference methods for large datasets <ref type="bibr">Damianou et al. (2014)</ref> ; <ref type="bibr">Hensman et al. (2013)</ref> ; <ref type="bibr">Lawrence (2007)</ref> exploit low-rank approximations to the full Gaussian process. However, these approximations suffer from a loss of accuracy in covariance estimation. Thus, we develop a block coordinate descent (BCD) algorithm as an exact inference method for the brain kernel model (see Methods, Algorithm 1 ). Coordinate descent has been successfully </p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>&#119868;</head><p>Find the optimal &#119833; ( &#119905; ) &#119868; and &#119809; ( &#119905; ) by solving eq.~29 or 31 Estimate the optimal length-scale &#120575; * and the optimal marginal variance &#119903; * by solving eq.~43 Update &#119818; -1 11 ( &#119905; ) and &#119818; -1 12 ( &#119905; ) given &#120575; * and &#119903; * according to eq.~42 end for applied to solve penalized regression models <ref type="bibr">Wu and Lange (2008)</ref> , to estimate covariance graphical lasso models <ref type="bibr">Wang (2014)</ref> , and to compute large-scale sparse inverse covariance matrices <ref type="bibr">Treister and Turek (2014)</ref> . Our PLS and MAP estimators are non-convex smooth functions. We apply an iterative block coordinate descent method solved by the proximal Newton approach <ref type="bibr">Tseng and Yun (2009)</ref> . Given such a scalable optimizer, we alternate between optimizing &#119833; and the hyperparameters using either eq. 15 (MAP) or eq. 18 (PLS) as the objective loss function. More details about the optimization can be found in the Methods section.</p><p>In practice, we used the PLS estimate to initialize the MAP estimate, as PLS optimization is faster and requires less memory, but the MAP estimate is more principled and achieves a higher accuracy. We used the BCD algorithm to optimize both the PLS and MAP objectives.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.4.">Predicting activity for new voxels</head><p>Although we fit the brain kernel model to data collected with a particular grid of voxels, our framework allows us to apply the model to fMRI measurements collected using different voxel grids. To do so, we use the fact that the brain kernel is defined using a Gaussian process; the mean of this GP provides a smooth mapping from 3D voxel space to the &#119889;-dimensional latent embedding space, which can be evaluated at any 3D brain locations. We obtain the embedding location of a novel 3D voxel location &#119857; * using the posterior mean of this GP:</p><p>where </p><p>with &#119859; * &#119894; and &#119859; * &#119895; the corresponding latent embeddings (eq. 19 ). Now given the newly estimated brain kernel and the new voxel location &#119857; * , we could predict its activity via</p><p>where</p><p>is the predicted activity of the new voxel across all measurements.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="4.5.">Synthetic experiments</head><p>To illustrate the performance of our proposed inference method to optimize the brain kernel objective function, we began with an application to simulated data, where the ground-truth embedding is known. We first compared our block coordinate descent (BCD) method Algorithm 1 ), which maximizes the exact log evidence (eq. ( <ref type="formula">15</ref>) , with two variational inference methods that optimize a lower bound on log evidence: an inducing-point method <ref type="bibr">Lawrence (2007)</ref> , and a dynamical variational inference method (dynamic VIP 1 ) <ref type="bibr">Damianou et al. (2014)</ref> .</p><p>We created a simulated dataset using a 1-dimensional brain and 1dimensional latent embedding function, sampled from the brain kernel model ( Fig. <ref type="figure">2</ref> ). Here, the latent embedding is a nonlinearly warped version of the 1D brain. We considered a set of 100 voxel locations on an evenly spaced 1D grid: &#119831; = [1 , 2 , &#8230; , 100] &#8868; . We then sampled the voxels' latent locations &#119833; from a GP with mean &#119898; ( &#119857;) = 0 . 6 &#119857; and RBF covariance with length-scale &#120575; = 10 and marginal variance &#119903; = 9 : &#119833; &#8764; &#57914; (0 . 6 &#119831; , &#119818; ) , where &#119818; &#119894;&#119895; = 9 exp (-( &#119894; -&#119895;) 2 &#8725;200) . Given this embedding function, the brain kernel defines a covariance matrix for neural activity at these 100 voxels, denoted &#119810; , with the &#119894;, &#119895;th entry given by &#119810; &#119894;&#119895; = exp (-( &#119859; &#119894; -&#119859; &#119895; ) 2 &#8725;2) ( Fig. <ref type="figure">2</ref> A top). To obtain simulated fMRI measurements, we sampled 750 observations from a Gaussian distribution with zero mean and covariance &#119810; + 5 &#119868;, where &#120590; 2 = 5 represents the variance of additive measurement noise.</p><p>We compared the different estimators on the task of recovering the true covariance and latent embedding function from this dataset ( Fig. <ref type="figure">2 B</ref>). The inducing-point method with six inducing points (column 1) performed well at recovering the latent embedding function, although it was outperformed by our BCD estimator in terms of mean squared error (BCD; column 3). The dynamic VIP estimate with six inducing points 1 <ref type="url">https://github.com/SheffieldML/GPmat</ref> (column 2) converged to a local optimum far from the true latent embedding, yielding a substantially higher error. In contrast, exact inference using BCD outperformed both inducing point-based approximate methods in terms of accuracy at recovering the true latent from simulated data.</p><p>Next, we conducted a set of synthetic experiments to examine how well different models captured the covariance of simulated fMRI data, using voxels on a 1-dimensional, 2-dimensional, or 3-dimensional grid. We fit these simulated datasets using the brain kernel model optimized with (1) BCD, (2) variational inducing-point, and (3) the dynamic VIP method, and two additional models: (4) a linear brain kernel (LBK) model, and (5) a GP with RBF covariance function (RBF). Note that the first three methods are based on the original brain kernel model but have different inference methods, while the last two methods represent simplifications of the proposed brain kernel model. The linear brain kernel model assumes a purely linear embedding function; it thus allows rotating and linearly dilating the voxel grid, but does not allow for nonlinear warping of voxel locations. The covariance function for neural activity &#119855; under the LBK model is given by:</p><p>where &#119809; is a &#119889; &#215; &#119907; linear embedding matrix and &#119907; is the number of dimensions of &#119909; , e.g., &#119907; = 2 for a 2-dimensional grid of voxels, and &#119889; is the dimension of the latent space. To assess the importance of the structured latent embedding in covariance estimation, we fit the data with a standard GP with RBF covariance function:</p><p>where &#119897; is the length-scale for the RBF kernel. This model imposes smoothness using Euclidean distance, without estimating any (linear or nonlinear) transformation of the true voxel locations. To fit the LBK and RBF models, we minimized the following negative log likelihood for hyperparameters &#120579;:</p><p>where &#119810; ( &#119831; , &#120579;) is the model-generated covariance matrix (given by eq. 22 for the linear brain kernel model with &#120579; = { &#119809; , &#120588;} , and by eq. 23 for the RBF model with &#120579; = { &#119897;, &#120588;} ), &#119826; is the sample covariance (eq. 16 ), &#120590; 2 is the noise variance, and &#119879; is the number of samples in the simulated dataset.</p><p>For each grid dimension, we simulated ten independent datasets, each with different nonlinear embedding functions sampled from the Fig. <ref type="figure">3</ref>. Quantitative comparisons for BCD, inducing-point, dynamic VIP, LBK, and RBF on three simulated datasets with 1D, 2D, and 3D input voxel locations. A is the held-out voxel experiment, and B is the held-out sample experiment. Within each experiment, we show the normalized test likelihood values for each method with three different simulated datasets. brain kernel model. For the 1D experiments, we generated each dataset with 500 voxels and 750 samples, and embedded the 1D voxel space to a 1D latent embedding space. For the 2D experiments, we used a 25 &#215; 25 voxel grid, embedded nonlinearly in a 3D latent embedding space, and generated datasets of 1000 samples, where each sample is a vector of 625 noisy measurements of brain activity at voxel locations. For the 3D datasets, we used a 10 &#215; 10 &#215; 10 grid of voxels, embedded nonlinearly into a 6D latent space, and we generated 1500 samples per dataset.</p><p>To compare models, we performed two different cross-validation tests: (i) prediction on held-out voxels ( Fig. <ref type="figure">3</ref> A), and (ii) predictions on held-out samples ( Fig. <ref type="figure">3 B</ref>).</p><p>For the first of the two cross-validation tests, we removed ten randomly-selected voxels &#119831; * from the simulated 1D datasets and set them aside as test data, and we optimized the model parameters on a training set of measurements from the remaining 490 voxels. (For experiments with 2D and 3D grids, we set aside 62 and 100 voxels as held-out data, and we trained models using the remaining 563 and 900 voxels, respectively.) For the BK and LBK models, we computed the embedding locations &#119833; * for the test voxels using the GP posterior mean given the inferred embedding locations &#119833; (eq. 19 ), and we used these locations to evaluate the covariance of the test voxel activity (eq. 21 ). For the RBF model, there is no embedding, so the covariance of the test data depends only on the test voxel locations &#119831; * . We used the resulting predictive covariance to compute the log likelihood of the test data. We found that the BCD-optimized brain kernel model estimate outperformed other methods, while the LBK and RBF models performed worse, presumably due to their inability to capture the nonlinear embedding of the simulated data ( Fig. <ref type="figure">3 A</ref>).</p><p>Next, we evaluated cross-validation performance on held-out samples, which were measured at the same set of voxels as the training set. For these simulations, we randomly selected 75 samples as test data for the 1D datasets, and used the remaining 675 samples to fit the five models. (For 2D and 3D grids, we used a train-test split of 900:100 and 1350:150 samples, respectively.) We estimated a covariance function using measurements in the training set, and computed a test likelihood using this covariance function on the test set. We generated ten random splits for each dataset and computed the normalized test log likelihood. In this predictive task, the BCD estimate again outperformed other methods ( Fig. <ref type="figure">3</ref> </p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>B).</head><p>Overall, these simulations demonstrated that the inducing pointbased GP methods, which are often used due to their scalability, had higher errors than our exact BCD inference method. In addition, by comparing to the linear brain kernel model and the standard GP model with an RBF kernel, we showed that the ability to capture a nonlinear transformation of the voxel locations is critical for accurately modeling the covariance of simulated brain activity.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="5.">Inferring the brain kernel from resting-state fMRI data</head><p>Now that we have described the brain kernel model and validated our inference method using simulated data, we turn to the problem of inferring the brain kernel from real data. We fit the brain kernel model to large-scale publicly-available resting-state fMRI data from the Human Connectome Project (HCP) Van Essen et al. ( <ref type="formula">2013</ref>) . An advantage of this approach is the large size of the HCP sample, which mitigates overfitting and allows us to learn an embedding function that captures correlation patterns common to a vast collection of different brains. However, applying the resulting brain kernel to task-based fMRI datasets assumes that the correlations presented in resting-state fMRI data are applicable to activation patterns in other brain states. Previous studies have shown interesting relations between brain activity during a task and at rest. One study showed that coherent spontaneous activity accounted for variability in event-related BOLD responses <ref type="bibr">Fox et al. (2006)</ref> . A second study concluded that functional networks used by the brain in action were continuously and dynamically "active " even when at "rest " <ref type="bibr">Smith et al. (2009)</ref> . A third study showed that resting-state activation patterns had strong statistical similarities to cognitive task activation patterns <ref type="bibr">Cole et al. (2016)</ref> . These provide reasons for optimism, though the possibility of changes in correlation across different tasks could potentially affect the application of the brain kernel which we will show empirically later.</p><p>We examined resting-state fMRI data from 812 subjects collected via the Human Connectome Project (HCP) <ref type="bibr">Van Essen et al. (2013)</ref> . These data were acquired in four fMRI runs of approximately 15 minutes each, two runs in one session and two in another session, with eyes open and relaxed fixation on a projected bright cross-hair on a dark background. A sophisticated preprocessing pipeline was used to align voxels across subjects. Detailed information about imaging protocols, image acquisition, and preprocessing can be found in WU-Minn (2017) . The resulting dataset consisted of 59,412 voxels in a cortical surface coordinate system. Although the full covariance matrix of these data is of size &#8776; 59 &#119870; &#215; 59 &#119870;, we fit the model using the first 4500 eigenvectors of the full matrix provided by HCP.</p><p>We fit the brain kernel with different numbers of latent dimensions and computed the test log likelihood with held-out voxels and samples ( Fig. <ref type="figure">4 D</ref>). We found that test performance plateaued with increasing dimensionality, and selected &#119889; = 20 dimensions for subsequent analyses. We then estimated the brain kernel by fitting the 20-dimensional latent embedding for each voxel using BCD optimization of eq. 18 and eq. 15 . The resulting function is a matrix of embedding locations of size &#8776; 59 &#119870; &#215; 20 , where each row contains the embedded location of a single voxel in the resting-state fMRI dataset. This embedding, in addition to the hyperparameters (the linear projection matrix &#119809; and the hyperparameters { &#120574;, &#120575;} for &#119818; ), provides a full parametrization of the brain kernel. The entire fitting process took approximately 1 week on a single CPU, since we needed to optimize the latent embedding location of each voxel in the HCP dataset so that the covariance in BOLD activity for any pair of voxels was accurately described by the brain kernel.</p><p>Although it is impossible to visualize the 20-dimensional nonlinear embedding that defines the brain kernel, we can gain insight into its shape by plotting low-dimensional projections on subsets of voxels. To visualize the brain kernel, we selected two symmetric ROIs in the left and right parietal lobes ( Fig. <ref type="figure">4</ref> A, top). Taken together, these two ROIs contain 3K voxels. We visualize the sample covariance of these voxels ( Fig. <ref type="figure">4</ref> A, bottom). This covariance contains four identifiable blocks: two diagonal blocks that correspond to the covariance of voxels within each ROI, and two off-diagonal blocks that correspond to cross-ROI covariance. The off-diagonal blocks reveal that the two ROIs have reasonably strong correlations despite being spatially distant in the brain.</p><p>The original 3D voxel coordinates ( Fig. <ref type="figure">4</ref> B, top) and the covariance given by the best-fitting RBF kernel (eq. 23; Fig. <ref type="figure">4 B,</ref><ref type="figure">bottom</ref>) show that the RBF kernel model fails to capture the off-diagonal blocks of the sample covariance, corresponding to covariance between voxels in opposite hemispheres, due to the fact that the RBF kernel depends only on the Euclidean distance between voxels. In contrast, a 3-dimensional projection of the estimated 20-dimensional brain kernel embedding ( Fig. <ref type="figure">4</ref> C, top) and the corresponding brain kernel covariance ( Fig. <ref type="figure">4</ref> C, bottom) appear to capture this cross-ROI structure in the covariance matrix. The 3D projection corresponds to the first three principal components of the 20-dimensional latent embeddings, and shows that paired voxels from opposite hemispheres are embedded close to each other under the brain kernel.</p><p>Conversely, some voxels that are physically close together in the brain are embedded far apart in the embedding space. As an example, we presented a visualization of some selected 3D coordinates ( Fig. <ref type="figure">5 A</ref>) and their corresponding 3D latents ( Fig. <ref type="figure">5 B</ref>) of three regions on the left hemisphere (color coded). We can see the green region is closer to the orange region in the 3D voxel space, while it's closer to the blue region in the latent space. The blue region covers mostly the visual area. The green region contains motor functions including eye movement and orientation. The red region corresponds to higher mental functions such as planning and emotion. This could explain the stronger functional connectivity between green and blue in the latent space.</p><p>Overall, this nonlinear embedding allows the brain kernel to more accurately capture the covariance of the real data.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.">Applications</head><p>To illustrate the usefulness of the brain kernel, we applied it to two different fMRI data analysis problems: decoding and factor modeling ( Fig. <ref type="figure">6</ref> ). For both analyses, we used the latent embedding function fit to resting state data (as described in Section 5 ), and tuned two hyperparameters governing the amplitude and length-scale of the brain kernel, which allowed it to adapt to the statistics of each task dataset of interest. We refer to this as tuning of the brain kernel covariance for use in applications. We describe these applications in detail below.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.1.">Brain decoding</head><p>In this section, we illustrate how the brain kernel can be applied to fMRI classification (or "decoding ") tasks.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.1.1.">HCP tasks</head><p>We first examined the task fMRI datasets in the HCP database. We explored the working memory task, the gambling task <ref type="bibr">Delgado et al. (2000)</ref> , the language processing task <ref type="bibr">Binder et al. (2011)</ref> , the motor task <ref type="bibr">Buckner et al. (2011)</ref> , the emotion processing task <ref type="bibr">Hariri et al. (2006)</ref> , the relational processing task <ref type="bibr">Smith et al. (2007)</ref> , and the social cognition task (more details found in <ref type="bibr">Barch et al. (2013)</ref> ; WU-Minn ( <ref type="formula">2017</ref>) ). We will elaborate on the working memory and gambling tasks and finally summarize the result with all datasets.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Working memory task</head><p>We obtained the working memory task fMRI measurements from the HCP <ref type="bibr">Barch et al. (2013)</ref> . The stimuli consisted of four types of pictures: places, tools, faces, and body parts. Stimuli were presented for 2 s on each trial followed by a 500-ms inter-trial interval (ITI). Each task block consisted of ten trials of 2.5 s each. Each run contained 8 task blocks, half Fig. <ref type="figure">6</ref>. Schematic figure illustrating two applications of the brain kernel to task fMRI data. After fitting the brain kernel to resting-state data, we applied it to task fMRI data with &#119899; voxels by evaluating the brain kernel at the 3D voxel locations. This results in a &#119899; &#215; &#119899; prior covariance matrix for the task data, denoted &#119810; . We then used the covariance &#119810; as the prior covariance for two modeling tasks: (1) brain decoding and (2) factor analysis. Two unknown parameters &#119856; and &#119819; are both random variables with a Gaussian prior whose covariance is &#119810; . Thus, we effectively imposed assumptions on the structure of &#119856; and &#119819; via &#119810; .</p><p>for a 2-back working memory task and half for a 0-back working memory task, along with 4 fixation blocks. Two runs were collected for each subject, with 405 volumes per run or approximately 5 minutes. Because the task fMRI was from the same HCP project as the resting-state fMRI, they shared the same coordinate system and preprocessing pipeline. The task fMRI was aligned with the MNI template with the same 59,412 voxels as the resting-state fMRI data. Instead of analyzing the whole-brain data for brain decoding, we worked with functional ROIs. For each presented type of pictures, we took the group-average task contrast for the 2-back vs 0-back working memory task and kept all voxel coordinates whose z-statistics were at least -1.96 or +1.96 standard deviations away from the mean, indicating that the voxels were statistically significant in the contrast map. We repeated the same thresholding procedure for all objects and took a union set of all coordinates to formulate the functional ROIs for the working memory task. We didn't select the ROIs using the contrast among objects, therefore the resulting ROIs contained no discriminative information with respect to the decoding task.</p><p>The experiment required observers to perform a working memory task using four different types of objects. We tested the ability to decode these objects from fMRI data by fitting a binary linear classifier for each pair of objects. We used Bayesian linear regression classifiers to solve six binary classification problems. A Bayesian linear regression classifier has the form,</p><p>where &#119857; is a vector of all voxels for one fMRI measurement or sample, &#119910; is a &#177;1 label indicating the binary object category for that sample, &#119856; is a vector of regression coefficients, and &#120598; is independent zero-mean Gaussian noise with variance &#120590; 2 . To regularize the estimate of &#119856; , we assumed a zero-mean Gaussian prior with covariance &#119810; , i.e., &#119856; &#8764; &#57914; (0 , &#119810; ) . We considered three different choices of prior covariance: (1) a ridge prior, which corresponds to a diagonal covariance with a positive constant along the diagonal;</p><p>(2) a radial basis function (RBF) covariance (eq. 23 ), which imposes smoothness based on the voxels' 3D locations in the brain, and (3) the brain kernel defined as</p><p>where &#119891; is the nonlinear embedding function fitted in Section 5 and is fixed for decoding. { &#120588;, &#119897;} are tuneable hyperparameters that we optimized when tuning the brain kernel to the task data of interest. The computational cost of this tuning is the same as the cost for optimizing the standard kernels such as the RBF covariance, which has an equivalent pair of hyperparameters. This optimization is fast (e.g., 80 s for 3,000 voxels on a CPU), and there is no difference in computational cost between the brain kernel and the RBF smoothing prior.</p><p>We trained the classifier with these three priors on one run and calculated accuracy performance on the second run, then repeated the same procedure with test and training sets reversed. When we trained the model, we randomly split the training run into five folds and selected the optimal hyperparameters in the covariance functions via 5-fold cross validation. After cross validation, we applied the optimal hyperparameters to the test run for each prior model. We repeated this 5-fold cross validation experiment ten times to reduce variability. We used linear regression to train the classifier, so the &#177;1 labels were treated as continuous target values, and test accuracy was evaluated by taking the sign of the prediction. We plotted averaged accuracy across both runs and all repeats for the three priors for ten randomly-selected subjects ( Fig. <ref type="figure">7 A</ref>). The RBF prior outperformed the ridge prior, but the brain kernel prior outperformed both of the other priors, indicating that smoothing in a nonlinear embedding space defined by correlations of fMRI signals provided additional benefits in regularizing weights for a classification task. Moreover, the framework of the brain kernel for regularization allows us to visualize the inferred decoding weights overlaid on a 3D brain ( Fig. <ref type="figure">7 B</ref>).</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Gambling task</head><p>We next examined fMRI data in a gambling task from the HCP <ref type="bibr">Barch et al. (2013)</ref> , adapted from a prior study <ref type="bibr">Delgado et al. (2000)</ref> .</p><p>Participants were asked to play a card-guessing game. They were shown a mystery card with a number that could range from 1 to 9. They needed to guess whether it was more or less than 5 by pressing on of two buttons. If they made the correct guess, the card showed a green up arrow with "$1 " for rewards; if they guessed wrongly, the card showed a red down arrow with "-$0.5 " for losses; if the true value was 5, they got a neutral response without win or loss. Participants had 1,500 ms to guess, and the feedback was presented for 1000 ms, followed by a 1000-ms inter-trial interval (ITI). Each task block consisted of eight trials that were either mostly reward or mostly loss. Two runs were collected for each subject. Each run contained two mostly reward and two mostly loss blocks, interleaved with four fixation blocks. There were 253 volumes per run, which lasted approximately 3 minutes. The task fMRI was aligned to the MNI template and had the same 59,412 voxels as the resting-state fMRI data. Instead of analyzing the whole-brain data for brain decoding, we worked with functional ROIs which were selected using the same approach as described in the working memory task.</p><p>We formulated the task as a binary classification problem separating reward trials and punishment trials. We used the same Bayesian linear regression classifiers as described in the working memory section. We trained the classifier with three priors on one run and calculated the accuracy performance on the second run, then switched the training and test runs. This procedure was repeated ten times. The &#177;1 labels were treated as continuous target values during training, and the test accuracy was evaluated by taking the sign of the prediction. We computed the averaged accuracies across two runs and 10 repetitions for the three priors for 15 subjects ( Fig. <ref type="figure">8 A</ref>). For 11 out of 15 subjects, the brain ker- nel improved accuracy over both the ridge prior and the smooth RBF prior. For two of the remaining four subjects, the brain kernel outperformed the ridge prior. The discrimination problem with the gambling data was more difficult than the working memory task. Many activations occurred in the visual cortex and the prefrontal cortex for working memory, whereas critical activations for the reward task might be localized to the striatum, which was not included in the cortical data used here. However, with the cortical brain kernel, we were still able to improve the predictive ability of the Bayesian decoding model.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>All HCP tasks</head><p>We've elaborated on the working memory and gambling tasks above. We also achieved the classification accuracy performance for all other tasks (presented in Appendix B.1). Here we summarize the averaged accuracy over all subjects for each task in Fig. <ref type="figure">9</ref> . Consistent with the above results, we succeeded in achieving the best performance with the working memory and gambling tasks using the brain kernel. For other tasks, the brain kernel performed mildly better than the ridge prior and the smoothing prior estimates. The overall performance of these HCP task fMRI datasets indicates that the brain kernel is a better choice than the smoothing and the ridge priors.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.1.2.">Visual recognition task</head><p>Next, we examined the problem of decoding faces and objects from fMRI measurements during a visual recognition task. Just to remind, the  brain kernel was estimated using the resting-state fMRI from HCP, and we applied it to a popular fMRI dataset from a study of human ventral temporal cortex <ref type="bibr">Haxby et al. (2001)</ref> for the decoding task. We extended the application beyond HCP, i.e., constructing the brain kernel on HCP and then trying on a completely different dataset. Therefore, we were looking at cross-dataset generalization, not just cross-task generalization within HCP. In this visual recognition experiment, six subjects were asked to recognize eight different types of objects (bottles, houses, cats, scissors, chairs, faces, shoes, and scrambled control images, examples in Fig. <ref type="figure">10</ref> ). Each subject participated 12 scanning runs. In each run, the subjects viewed images of eight object categories, with 11 whole-brain measurements per category. Each subject's fMRI data was preprocessed using the fMRIprep package<ref type="foot">foot_0</ref> Esteban et al. ( <ref type="formula">2017</ref>) and aligned to the MNI template. Both voxels in this dataset and voxels in the HCP database were all aligned in the same MNI space, allowing us to use eq. 19 to get the brain kernel covariance for the present dataset. Instead of analyzing the whole-brain data, we extracted ROIs with 1,645 voxels in the ventral temporal cortex, which is thought to be involved in object recognition. The ROI mask was obtained from Nilearn <ref type="bibr">Abraham et al. (2014)</ref> .</p><p>We assessed performance by training Bayesian linear regression classifiers to discriminate between pairs of objects, e.g., face vs. bottle, for each of the 28 possible binary classifications among the eight objects ( Fig. <ref type="figure">10</ref> ). We trained the weights &#119856; for each model using linear regression from fMRI measurements &#119857; to binary labels &#119910; &#8712; {-1 , +1} , and assessed accuracy on the test set using predicted labels &#375; = sign ( &#119857;&#119856; ) . Here, we split the training and test sets by subjects. In this visual recognition dataset, we had six subjects but only 11 &#215; 2 &#215; 12 = 264 measurements in a 1,645-voxel space for each subject in a binary decoding task. The number of training measurements was not sufficiently large to train a Fig. <ref type="figure">11</ref>. Accuracy performance on the visual recognition task. The x-axis indicates subject IDs. The y-axis is accuracy performance. We compared our brain kernel with a ridge prior and a smooth RBF kernel, color coded. good classifier with a reasonable generalization performance. Therefore, different from the HCP tasks, we chose to do inter-subject analyses by using 5 subjects for training and one subject for test. We repeated this leave-one-subject-out manner for six times with each subject being used as the test set once and obtained the result in Fig. <ref type="figure">11</ref> .</p><p>The averaged accuracy performance across four repeated runs for six subjects shows that the brain kernel performed comparably to the ridge and smoothing priors with better accuracy performance for 5 out of 6 subjects ( Fig. <ref type="figure">11</ref> ). This indicates that the brain kernel can provide functional and structural support for most subjects and visual recognition tasks in this dataset. The improvement was statistically modest overall based on the standard errors, which could be a result of several factors: misalignment of the coordinate space to the HCP coordinate space used to estimate the brain kernel; mismatch between the resting-state covariance used to construct the brain kernel and covariance present during the visual recognition task; or the object recognition tasks may rely on fine-grained spatial response topographies that are poorly aligned across individuals.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.2.">Factor modeling</head><p>In this section, we illustrate a second type of application of the brain kernel. Instead of using it to regularize decoding weights, as in the previous section, we used it as a spatial prior for Bayesian factor analysis.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.2.1.">Sherlock movie watching task</head><p>We examined the Sherlock fMRI dataset, in which participants were scanned while they watched the British television program "Sherlock " for 50 min <ref type="bibr">Chen et al. (2017)</ref> . The fMRI data comprised 1,973 TRs (Repetition Time), where each TR was 1.5 s of the movie. Before performing any analysis, the fMRI data were preprocessed and aligned to MNI space using the techniques described in the prior work <ref type="bibr">Chen et al. (2017)</ref> . We examined the brain data averaged across all subjects to smooth out individual variability. We identified 11 ROIs previously implicated in processing naturalistic stimuli, comprising the default mode network (DMN-A, DMN-B), the ventral and dorsal language areas, and the primary auditory and visual cortices <ref type="bibr">Simony et al. (2016)</ref> .</p><p>For each ROI, we performed a standard factor analysis (FA) to factorize the voxel-by-time fMRI data into a latent source matrix and a factor matrix. Thus fMRI images can be considered as being generated by a covariate-dependent superposition of latent sources. Some following analyses, such as decoding and encoding tasks, can be performed with the factor matrix. The number of latent sources is much fewer than the number of time points, thus providing a parsimonious description of neural activity patterns that avoids many of the pitfalls of traditional voxel-based approaches. Similar FA based models have been proposed in <ref type="bibr">Gershman et al. (2011)</ref> ; <ref type="bibr">Manning et al. (2014)</ref> . Here, we performed a Bayesian factor analysis of the form where &#119831; &#8712; &#8477; &#119873;&#215;&#119879; is the fMRI image with &#119873; voxels and &#119879; time points, &#119819; &#8712; &#8477; &#119873;&#215;&#119870; is the latent source matrix encoding the canonical spatial pattern (over voxels) associated with each latent source. &#119813; &#8712; &#8477; &#119870;&#215;&#119879; is the factor matrix containing the timeseries for &#119870; latent sources. &#120598; is a Gaussian noise with &#120598; &#8764; &#57914; (0 , &#120590; 2 ) . We assumed that the &#119894; th factor (column) is from a standard normal distribution, i.e., &#119901; ( &#119813; &#119894; ) = &#57914; (0 , &#119816; ) . We also assumed that the &#119895;th latent source (column) is from a Gaussian prior &#119901; ( &#119819; &#119895; ) = &#57914; (0 , &#119810; ) where &#119810; is the covariance matrix. Like the decoding tasks, we used a ridge prior, a smooth RBF kernel, and the brain kernel in the place of the covariance matrix &#119810; . With different priors, the latent sources &#119819; showed different characteristics imposed by these distinct regularizations.</p><p>Our goal was to infer the latent variables &#119819; and &#119813; , and the hyperparameters for &#119810; and &#120590; 2 , denoted as &#120579;. To quantify performance, we compared data explanation under the three prior with the same number of latent sources which was set to 10. In practice, we first standardized the fMRI image &#119831; , then tuned the hyperparameters using the marginal distribution of &#119831; , and finally inferred the factor matrix and the latent sources via maximum likelihood parameter estimation (details in Methods). To evaluate the performance of the three priors, we used a method called "co-smoothing " <ref type="bibr">Wu et al. (2018)</ref> , depicted in Fig. <ref type="figure">12</ref> . We first split the time points into two equal sets by taking the first half as one set and the latter half as the other set. We took the first set for training (the blue region) and the second set for inference (the pink region) and test (the yellow region). We trained the model with the neural activity in the training set to obtain the estimated latent sources &#119819; * . We then kept the latent sources fixed for the inference and test purpose. Next we split the second set into five folds along the voxel axis, four for inference and one for test. We used the neural activity in the inference set to infer the factor matrix (mapping the latent sources to the time series) during the inference period given the latent sources &#119819; * . Finally we predicted the neural activity for the left-out voxels in the test set given the latent sources and factor matrix. We repeated the inference and test step five times in a cross-validation fashion and obtained an averaged &#119877; 2 value representing the performance of the prior. After the first run, we achieved three &#119877; 2 values for the three priors (ridge, smooth and brain kernel). To better visualize the difference, we normalized the three &#119877; 2 values so that the maximum was 1 and the minimum was 0. We then launched a second run by using the second set as the training set and the first set as the inference and test set. The final normalized test &#119877; 2 value was an average of the two runs.</p><p>We compared the normalized test &#119877; 2 value for the three priors for each ROI ( Fig. <ref type="figure">13</ref> ). The error bars indicate standard errors. In most regions, the brain kernel outperformed the ridge prior and the smooth RBF kernel. This implies that when performing Bayesian FA, the brain kernel provided a superior prior covariance for the latent source matrix and may enhance performance in terms of data explanation.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="6.2.2.">HCP tasks</head><p>We examined the task fMRI datasets in the HCP database with the same Bayesian FA model. We collected all the task fMRIs for the same 10 subjects and performed Bayesian FA for each subject in each task. We implemented the same "co-smoothing " evaluation and compared the normalized test &#119877; 2 for the three priors averaged over all subjects for each task ( Fig. <ref type="figure">14</ref> ). In most tasks, the brain kernel outperformed the ridge prior and the smooth RBF kernel. This implies that the brain kernel provided a superior prior covariance for the latent source matrix and may enhance performance in terms of data explanation for the HCP database.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head n="7.">Discussion</head><p>We introduced the brain kernel model, a new model for the spatial covariance of fMRI data. This model takes the form of a covariance function, meaning that it can take any two continuous voxel locations in the brain as inputs and return the prior covariance of their activities. Unlike standard smoothness-inducing covariance functions used in the Gaussian process literature, which assume that correlation falls off monotonically with distance, the brain kernel allows widely-separated voxel locations (e.g., on opposite sides of the brain) to have high correlation, while pairs of nearby voxels can be nearly uncorrelated. This property arises from the fact that the nonlinear embedding function, parametrized with Gaussian processes, warps and stretches the brain in a higher-dimensional space so that widely-separated pairs of voxels can be mapped to nearby embedding locations, while pairs of nearby voxels can be moved far apart in embedding space.</p><p>We fit the brain kernel model using a nonlinear embedding of the 3D brain in a 20D space. We introduced an exact inference method for fitting the brain kernel model using block coordinate descent (BCD), and estimated the brain kernel using a large publicly-available dataset of resting-state fMRI measurements from many subjects. We found that the resulting brain kernel accurately captured the covariance of restingstate fMRI measurements.</p><p>We further demonstrated the applicability of the brain kernel using two different modeling tasks: brain decoding and factor analysis. In both cases, we used the brain kernel to define a prior distribution in place of a more conventional prior based on &#120001; 2 shrinkage or local smoothness. We showed that the brain kernel achieved gains in performance, illustrating that the correlations in resting-state fMRI may be usefully mined to aid analyses of task fMRI. A comprehensive summary of application results can be found in Appendix B.</p><p>Moreover, to examine other choices of covariance function for parametrizing the brain kernel, we re-fit the brain kernel to restingstate fMRI data using a Matern-3/2 covariance function. We also compared the RBF and Matern-3/2 brain kernels using the working-memory decoding task fMRI data shown in the main paper. We found that the RBF covariance function outperformed the Matern-3/2 covariance function in both modeling the covariance of resting-state data and decoding of working memory task data. More details can be found in Appendix C. We have therefore decided to keep our focus on the brain kernel parametrized with RBF covariance in the main paper. But exploring a wider family of possible covariance functions is indeed a worthwhile direction for further research.</p><p>The estimation of a good quality of the brain kernel requires as much fMRI data as possible. However, we want to emphasize is that our intention was not to suggest that experimentalists should collect more data than needed for their study. We merely meant to suggest that -if there are pre-existing publicly or privately available datasets that exhibit similar functional connectivity / covariance structure to patterns of activity observed during a particular study, then that additional data might be leveraged using the brain kernel to improve decoding performance or latent variable modeling of the data collected for the study of interest. This would indeed incur a computational burden (researchers would need to fit the brain kernel using the pre-existing dataset). However, no additional data would be needed; on the contrary, the brain kernel would make it possible to exploit the structure of the pre-existing data so that the significance level / effect sizes are larger in the study of interest, thus effectively increasing statistical power or reducing data requirements. For the purpose of using the resting-state brain kernel as an alternative prior apart from the RBF covariance, people can just download it from  link and use it instead of the RBF covariance, since we have already fit it with a giant resting-state fMRI dataset from diverse subjects.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Limitations</head><p>Despite the successes presented above, it is important to acknowledge that we applied the brain kernel to several other datasets for both the decoding and the factor modeling tasks beyond the examples shown in the main section, for which it did not yield improvements over standard priors.</p><p>For decoding, we applied the brain kernel to the Sherlock movie dataset <ref type="bibr">Chen et al. (2017)</ref> whose decoded vectors contained semantic descriptions for the movie scenes. We observed that the smoothing prior was better than the brain kernel prior, both better than the ridge prior. Then we investigated the reason of the performance difference and figured out that the brain kernel was able to impose a strong cross-covariance assumption over many voxels; but much of the crosscovariance didn't exist in the Sherlock movie data or could hurt the decoding performance. The optimal brain kernel prior corresponded to a small length-scale that led to less smoothing assumptions compared with the smoothing kernel. So this Sherlock movie dataset is not an ideal dataset that could leverage most of the resting-state functional connectivity to do the decoding task. We also applied it to a public fMRI dataset in which subjects viewed 5000 visual images (BOLD5000) <ref type="bibr">Chang et al. (2019)</ref> . The binary labels were living objects versus nonliving objects. We observed that the brain kernel did not reliably outperform the ridge prior and the smoothing prior estimates across subjects. The optimal smoothing prior and the brain kernel prior both converged to the ridge prior after hyperparameter estimation, i.e., the estimated length-scale was very small and all three priors had the same performance. Therefore, smoothness did not help the classification task. More details can be found in Appendix B. For these cases, regularizing the decoding weights with the brain kernel did not improve performance, suggesting that the covariance of resting-state fMRI did not provide useful information for classifying fMRI measurements in these tasks, or at least that the brain kernel was not capable of capturing it. It remains possible, however, that a brain kernel fit from task fMRI datasets might offer benefits for decoding fMRI data from these tasks.</p><p>For factor modeling, we also applied the brain kernel to the visual recognition task and the BOLD5000 dataset except for the HCP dataset and the Sherlock movie dataset presented in the main paper. For most subjects in both datasets, the brain kernel didn't show a dominating performance over both the ridge prior and the smooth RBF kernel (Appendix B).</p><p>Beyond those reasons, we hypothesize that the lack of benefit observed on non-HCP (outside the HCP) datasets may arise from a mismatch between the covariance of resting-state fMRI observations used to fit the brain kernel and the covariance of task fMRI datasets. However, it might also arise from differences in acquisition parameters, preprocessing steps, or alignment between HCP and non-HCP datasets. Although we aligned all voxels with the MNI template, misalignment might still result from differences between processing pipelines.</p><p>We include more analyses about the reasons behind these limitations and under which circumstances the brain kernel could be a better prior option than the ridge prior and the smoothing prior in Appendix B. We can show only that if there is similar structure between the covariance of resting-state data used to estimate the brain kernel and the discriminative ROIs in the task data, the brain kernel will function as a better prior than a standard smoothing kernel. For factor modeling applications, the brain kernel is often more reliably effective because we typically use a large number of voxels to infer latents, rather than a few responsive voxels that may be selected in decoding tasks. Similarly, we would expect the brain kernel to give good performance when the task fMRI is composed of smooth latent sources that resemble the spatial correlation in the resting-state data. We can fit a best brain kernel or a best RBF kernel to the sample covariance and evaluate the similarity both qualitatively and quantitatively with the negative log-likelihood (NLL) value. If the best-fitting brain kernel resembles the sample covariance and the NLL value of the brain kernel is smaller, it's promising to use the brain kernel as a prior for Bayesian factor modeling analysis.</p><p>Given that the computational burden is not higher than that of standard smoothing or shrinkage priors (and potentially smaller than shrinkage-inducing regularizers like LASSO), we hope that researchers will incorporate the brain kernel into standard analysis pipelines, and apply it in cases where it is observed to offer improved performance. We consider this to analogous to the ways in which existing regularizers such as smoothing priors or sparsity-inducing priors like LASSO are currently employed: researchers may conduct exploratory analyses to determine whether incorporating smoothness or sparsity improves performance, and then adopt these regularizers as warranted by the data.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Outlook and future directions</head><p>Although the brain kernel did not achieve improved performance in all datasets and applications we considered, we feel it nevertheless holds great potential for fMRI data analysis. First, we cast the problem of estimating covariance of fMRI datasets as that of estimating a nonlinear mapping from 3D brain coordinates to a latent embedding space. This results in a compact representation of the full brain covariance matrix, requiring storage of only a &#119873; &#119907;&#119900;&#119909;&#119890;&#119897;&#119904; &#215; 20 matrix of embedding locations (when the embedding dimensionality is 20), as opposed to a full &#119873; &#119907;&#119900;&#119909;&#119890;&#119897;&#119904; &#215; &#119873; &#119907;&#119900;&#119909;&#119890;&#119897;&#119904; covariance matrix. Moreover, because the brain kernel is a continuous covariance function, we can use it to model the covariance at arbitrary voxel locations, even those not contained in the original training dataset.</p><p>In addition to providing a compact parametrization of fMRI covariance, the brain kernel's nonlinear embedding function may be useful for gaining insights into the geometry of correlations within and across brain regions. Examining the embedding of different brain regions (e.g., as shown in Fig. <ref type="figure">4</ref> ) allows researchers to directly visualize correlations in terms of distances between embedded voxels.</p><p>Although the brain kernel fit to the HCP resting-state fMRI data did not yield improvements on all task fMRI datasets we explored, it is possible that other methods for training or applying the brain kernel might produce bigger gains. For example, one might train the brain kernel on task fMRI datasets, or train a hierarchical version that gains statistical strength from combining datasets, while preserving flexibility to capture differences between the two kinds of data. A more ambitious possibility is to formulate a hierarchical version of the brain kernel that allows for per-subject variability. This would produce a hierarchical brain kernel in which each brain has own specific brain kernel, allowing for detailed differences in correlation maps across brains. All such applications will benefit from more robust methods for alignment and preprocessing, and it may be that these alone will be enough to improve the performance of the brain kernel. Although these directions are beyond the scope of the current paper, we feel that the idea of the brain kernel is one that researchers might extend and apply to novel settings and datasets.</p><p>Finally, an exciting possibility for future work is to combine the brain kernel with other advanced statistical modeling techniques. Methods for modeling structured variability, such as topographic ICA <ref type="bibr">Manning et al. (2014)</ref> , and methods for structured sparsity, such as GraphNet <ref type="bibr">Grosenick et al. (2013)</ref> , sparse overlapping sets lasso <ref type="bibr">Rao et al. (2013)</ref> , and dependent relevance determination <ref type="bibr">Wu et al. (2019</ref><ref type="bibr">Wu et al. ( , 2014) )</ref> , rely on capturing dependencies between nearby voxels. All such methods might thus be improved by using nearness in functional embedding space provided by the brain kernel, instead of nearness in 3D Euclidean space inside the brain. Likewise, methods for linear alignment of functional data from multiple subjects such as hyperalignment <ref type="bibr">Haxby et al. (2020)</ref> ; <ref type="bibr">Xu et al. (2012)</ref> might be extended using nonlinear warping of brain coordinates under the brain kernel. Therefore, we feel that the brain kernel holds promise for inspiring new datadriven prior distributions and new modeling approaches for capturing structure in fMRI data.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Methods</head></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Block coordinate descent for the brain kernel model</head><p>The penalized least squares (eq. 18 , PLS) has a computational complexity of &#119874;( &#119899; 2 ) and memory storage of &#119874;( &#119899; 2 ) ; however, the maximum a posteriori (eq. 15 , MAP) has a computational complexity of &#119874;( &#119899; 3 )to invert the covariance matrix -and memory storage of &#119874;( &#119899; 2 ) . &#119899; is the number of voxels, which often exceeds 10K. Gradient descent or Newton's method is computationally impractical as the optimizer. Thus, we need to use a scalable inference method. Existing inference methods for large datasets <ref type="bibr">Damianou et al. (2014)</ref> ; <ref type="bibr">Hensman et al. (2013)</ref> ; <ref type="bibr">Lawrence (2007)</ref> exploit low-rank approximations to the full Gaussian process, which, however, suffer from a loss of accuracy in covariance estimation. Thus, in this section, we develop a block coordinate descent algorithm as an exact inference method for the brain kernel model. Coordinate descent has been successfully applied to solve penalized regression models <ref type="bibr">Wu and Lange (2008)</ref> , to estimate covariance graphical lasso models <ref type="bibr">Wang (2014)</ref> , and to compute large-scale sparse inverse covariance matrices <ref type="bibr">Treister and Turek (2014)</ref> .</p><p>Our PLS and MAP estimators are non-convex smooth functions. We apply an iterative block coordinate descent method solved by the proximal Newton approach <ref type="bibr">Tseng and Yun (2009)</ref> . We first divide the voxel set {1 , &#8230; , &#119899; } into blocks. Next, we iterate over all blocks, minimizing the functions with respect to the voxels within each block. Without loss of generality, we split the voxel set into two blocks, {1 , &#8230; , &#119898; } (block 1) and { &#119898; + 1 , &#8230; , &#119899; } (block 2), where &#119898; &#8810; &#119899; , and focus on the first &#119898; columns of &#119833; for the update. We partition &#119833; , &#119810; , &#119826; , and &#119818; -1 as follows:</p><p>Here, subscripts represent the block indices, so &#119833; 1 and &#119833; 2 are the first &#119898; columns and the last &#119899; -&#119898; columns of &#119833; and subscript 12 indicates the block matrix across the first &#119898; variables and the last &#119899; -&#119898; variables.</p><p>Only &#120631; and &#120632; contain the active variables in &#119833; 1 to optimize. &#119833; 2 is fixed.</p><p>Applying the block representation to eq. 18 , we get the block PLS objective function for solving &#119833; 1 :</p><p>where &#119831; 1 and &#119831; 2 are the first &#119898; columns and the last &#119899; -&#119898; columns of &#119831; .</p><p>To formulate the block MAP estimator for &#119833; 1 , we first apply the block matrix inversion to &#119810; ,</p><p>Incorporating this matrix inversion into the MAP estimator, we get the objective function</p><p>The time complexity of the block PLS estimator is &#119874;( &#119899;&#119898; 2 ) per iteration, where &#119898; is the size of the block, and the time complexity of the MAP estimator is &#119874;( &#119899; 2 &#119898; ) per iteration. For comparison, greedy gradient descent has complexity &#119874;( &#119899; 3 ) per iteration. In the experiments, the block MAP estimator for &#119833; with the block PLS estimator as a warm start is a practical optimization approach. We now describe the block coordinate descent (BCD) update. We assume that the voxel indices {1 , &#8230; , &#119899; } are divided into &#119896; blocks { &#119868; &#119895; } &#119896; &#119895;=1 , where &#119868; &#119895; is the set of indices corresponding to the columns of &#119833; in the &#119895;'th block. Denote &#119868; &#119895; columns of &#119833; to be &#119833; &#119868; &#119895; . We cluster indices into blocks based on the spatial locations of the voxels and assume smooth measurements for nearby voxels. Thus, the size of a block should be at least one length-scale of the region defined by the kernel in &#119857; space to encourage dependencies among neighboring voxels. This smoothness assumption leads to a block-wise but not an element-wise update, which separates our BCD method from Informative Vector Machine (IVM) <ref type="bibr">Lawrence (2003)</ref> . At each iteration &#119905; , we choose a nonempty index subset &#119868; &#8712; { &#119868; &#119895; } &#119896; &#119895;=1 . Then the objective functions &#119897;( &#119833; &#119868; ) in eq. 29 and 31 are optimized w.r.t. &#119833; &#119868; via L-BFGS <ref type="bibr">Nocedal (1980)</ref> . After the &#119905; 'th iteration, &#119833; &#119868; is updated, holding all other blocks fixed.</p><p>For high-dimensional non-convex problems, a good initialization is essential to finding a good optimum in practice. Two steps we exploited during implementation to mitigate the optimization issue with multimodal, high-dimensional non-convexity.</p><p>First, we assumed the nonlinear latent embedding &#119859; to be a local warping of the linear embedding which is the mean of the posterior distribution for &#119859; (eq. 8 ). We first found a good estimate of &#119835; at the beginning of the optimization. This is equal to fitting a linear brain kernel (LBK) model. We estimated &#119809; in eq. 22 which is a matrix form of &#119835; . This involves an easier optimization since the parameter space of &#119835; is much smaller than &#119859;. Given the estimated &#119835; , &#119859; = &#7825; + &#119835;&#119831; is optimized via estimating &#7825; and fixing &#119835; . This eases the high-dimensional non-convex issue by learning a small parameter &#119835; and a local warping &#7825; separately.</p><p>Secondly, we introduced the Laplacian eigenmap algorithm, an effective and tractable initialization for a single block of &#119833; inspired by Laplacian eigenmaps.</p><p>The Laplacian eigenmap (LE) algorithm is a popular dimensionality reduction method that solves a generalized eigendecomposition <ref type="bibr">Belkin and Niyogi (2003)</ref>  </p><p>with the symmetric affinity matrix &#119829; &#8712; &#8477; &#119899; &#215;&#119899; , the degree matrix &#119811; = diag ( &#8721; &#119899; &#119894; =1 &#119907; &#119894;&#119895; ) &#8712; &#8477; &#119899; &#215;&#119899; , the graph Laplacian matrix &#119819; = &#119811; -&#119829; &#8712; &#8477; &#119899; &#215;&#119899; , and &#120783; = [1 , &#8230; , 1] &#8868; . Constraints eliminate the two trivial solutions &#119833; = &#120782; by setting an arbitrary scale and &#119859; 1 = &#8230; = &#119859; &#119899; by removing &#120783; , which is an eigenvector of &#119819; associated with a zero eigenvalue.</p><p>Following the previous two-block example, let &#119833; be partitioned into &#119833; 1 and &#119833; 2 . To update &#119833; 1 given &#119833; 2 , the objective function is:</p><p>We don't need to use the constraints from eq. 32 because the trivial solutions are removed given &#119833; 2 . The solution is</p><p>Given the current latent embeddings for all other coordinates, the algorithm seeks the best latent embedding of the unknown dimensions.</p><p>Because of the computational efficiency of this approach, we use Z 1 to initialize the BCD algorithm for each iteration and &#119833; 2 collapses all the fixed blocks.</p><p>In addition, if &#119829; has a Gaussian affinity function, the latent embedding &#119833; is mapped from the observation &#119832; with a radial basis function nonlinearity. For covariance estimation, we enforce the resemblance between another radial basis function (RBF) kernel on &#119833; and the sample covariance of &#119832; . We are essentially trying to map the observation space to itself with double layers of exponential transformations, which would result in a bad latent embedding for initialization. Therefore, instead of using a Gaussian function for the weights &#119829; , we use a function that inverts the RBF nonlinearity on a covariance matrix, defined as &#119907; &#119894;&#119895; = -&#119891; ( &#119858; &#119894; &#119858; &#8868; &#119895; ) . Here, &#119891; ( &#119909; ) = sign ( &#119909; ) log ( |&#119909; | + 1) is the log-modulus transformation <ref type="bibr">John and Draper (1980)</ref> , which distributes the magnitude of the data while preserving the sign of the data in order to control against negative covariance values when taking the logarithm.</p><p>We chose the Laplacian eigenmap algorithm because it has the nice closed-form expression for the conditional expression &#119833; 1 given &#119833; 2 . We were able to use &#119833; 2 to efficiently find a better initial position for &#119833; 1 that reduced the search over the entire parameter space of &#119833; 1 . We tried a random start for &#119833; 1 which led to a very bad local optimum. We also tried to use the previous estimate to initialize &#119833; 1 . That resulted in a stuck in a bad local optimum that was very close to the previous solution. So Laplacian eigenmap allowed us to move away from the previous estimate but also leverage &#119833; 2 effectively.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Hyperparameter estimation for the brain kernel model</head><p>Our model includes five hyperparameters: { &#120575;, &#119903;, &#119809; , &#120588;, &#120590; 2 } . We set these parameters as follows.</p><p>Estimate &#120575;: &#120575; is the length-scale of the GP kernel mapping from coordinate space to latent space. We can optimize this parameter by taking the derivative of the GP log-likelihood w.r.t. &#120575;; this involves inverting the kernel matrix with an &#119874;( &#119899; 3 ) cost, which is computationally infeasible here. An inducing-point method <ref type="bibr">Damianou et al. (2014)</ref> introduced extra inducing variables to optimize, which further increased the computational burden. Instead, we use a scalable spectral formulation for learning the length-scale &#120575; of the GP kernel.</p><p>For a stationary Gaussian process &#119891; &#8764; &#57907;&#57916; ( &#119898;, &#119896; ) , when &#119891; has a high degree of smoothness, the prior covariance &#119818; becomes approximately low rank, meaning that it has a small number of non-negligible eigenvalues. Because the kernel function for &#119857; space is shift invariant, the eigenspectrum of &#119818; has a diagonal representation in the Fourier domain, a consequence of Bochner's theorem L&#225;zaro-Gredilla et al. ( <ref type="formula">2010</ref>) ; <ref type="bibr">Wu et al. (2017)</ref> ,</p><p>where &#119892; is the Fourier transform of &#119891; , &#120654; is the frequency, &#120572;( &#120654; ) is the Fourier transform of the mean function &#119898; ( &#119857;) , and &#931; = diag ( &#119904; ( &#120654; )) is diagonal. This means that Fourier components are a priori independent, with prior variance</p><p>where &#8462; is the dimension of the input &#119857;. Without loss of generality, we assume the size of the spectral domain for each input dimension is &#119908; , and thus &#120630; = &#120572;( &#120654; ) &#8712; &#8477; &#119908; &#8462; and &#120506; = &#931;( &#120654; ) &#8712; &#8477; &#119908; &#8462; &#215;&#119908; &#8462; given a spectral representation &#120654; . The mapping between &#119891; ( &#119857;) and its Fourier transform &#119892;( &#120654; ) is then</p><p>where &#119838; is a column vector with entries &#119890; 2 &#120587;&#119894; &#119857; &#8868; &#120654; &#119895; on the &#119895;'th position for the &#119895;'th frequency. Let &#119838; &#119894; &#8712; &#8477; &#119908; &#8462; denote the Fourier vector for &#119857; &#119894; ( &#119894; is the index for voxels), then we can further define &#57902; = ( &#119838; 1 , &#8230; , &#119838; &#119899; ) &#8712; &#8477; &#119908; &#8462; &#215;&#119899; to be the Fourier matrix for an input coordinate matrix &#119831; . Let &#119859; &#119895; &#8712; &#8477; &#119899; denote the &#119895;th latent embedding of &#119831; and &#119895; &#8712; {1 , &#8230; , &#119889;} . Note that &#119859; &#119895; is the &#119895;'th row of the matrix &#119833; . We can write &#119859; &#119895; = &#119840; &#8868; &#119895; &#57902; , where &#119840; &#119895; &#8712; &#8477; &#119908; &#8462; &#8764; &#57914; ( &#120630; &#119895; , &#120506;) . This implies that each latent embedding deterministically depends on a unique spectral function. These spectral functions are all sampled from multivariate Gaussian priors with different mean functions but the same covariance function in the spectral domain. Bringing the Fourier matrix &#57902; into the Gaussian prior (eq. 35 ), we derive the Gaussian prior over the latent &#119859; &#119895; expressed with the spectral formulation as</p><p>We can use this representation to optimize &#120575;. If &#120575; is in a largevalue regime, we can ignore Fourier components above a certain highfrequency cutoff which leads to a lower-dimensional &#120654; and a lowerdimensional optimization problem. Because &#8462; = 3 in fMRI analyses, we can control &#119908; to be small enough so that &#119908; &#8462; is tractable relative to the large &#119899; , and &#57902; is a manageable high-dimensional matrix. Because we generally do not assume uniform gridding of coordinates &#119831; , &#57902; is a nonuniform DFT (not orthogonal). Although the Kronecker tricks cannot be used for scaling up with non-uniform DFT, we can employ block matrix inverse lemma to transform the spectral formulation into a lowerdimensional problem.</p><p>Then the standard negative GP log-likelihood to optimize for &#120575; is</p><p>where &#120598; is a small noise variance in the latent space to avoid illconditions for &#119818; . Let &#119808; = ( &#120630; 1 , &#8230; , &#120630; &#119889; ) &#8868; &#8712; &#8477; &#119889;&#215;&#119908; &#8462; . With the spectral representation, eq. 39 can be re-written as</p><p>Calculating ( &#57902; &#8868; &#120506;&#57902; + &#120598;&#119816; ) -1 is still computationally expensive, but with the spectral factorization, we are able to use the block matrix inverse lemma as in eq. 30 ,</p><p>where &#57902; 1 and &#57902; 2 correspond to the Fourier bases for &#119831; 1 and &#119831; 2 respectively. Note that neither &#57902; 1 nor &#57902; 2 is invertible. Moreover,</p><p>All matrices have size &#119908; &#8462; &#215; &#119908; &#8462; , which is tractable to invert. We know that the spectral expression of &#119818; -1 is ( &#57902; &#8868; &#120506;&#57902; + &#120598;&#119816; ) -1 . We can also express &#119818; -1 11 and &#119818; -1 12 with the spectral formulation as With eq. 43 , the computational cost is reduced to max ( &#119874; ( &#119899;&#119908; &#8462; ) , &#119874; ( &#119908; 3 &#8462; ) ) , where &#119908; is the dimension of the spectral form per input dimension and &#8462; is the number of input dimensions. This complexity is much smaller than &#119874;( &#119899; 3 ) when &#119899; &#8811; &#119908; and &#8462; &#8804; 3 . Another benefit of this formulation is that &#120575; only exists in the diagonal of &#120506; (eq. 36 ), which makes the estimation straightforward.</p><p>Estimate &#119903; : &#119903; determines the scale of the latent embedding &#119833; . Since &#119903; also exists on the diagonal of &#120506; (eq. 36 ), the optimization for &#119903; can be combined with learning &#120575; using the same spectral representation in eq. 43 .</p><p>Estimate &#120588;: &#120588; is the marginal variance of the covariance function. We assume that the data has been normalized to have zero mean and variance one. Thus we set &#120588; = 1 .</p><p>Estimate &#119809; : &#119809; is the linear projection of the mean function. We can estimate &#119809; jointly with &#119833; in each BCD iteration by optimizing the same objective function (eq. 29 and 31 ).</p><p>Estimate &#120590; 2 : &#120590; 2 is the observation noise variance. We estimate &#120590; 2 using the eigenvalues of the sample covariance &#119826; <ref type="bibr">Tipping and Bishop (1999)</ref> .</p><p>Algorithm 1 describes the complete BCD algorithm for the brain kernel model. The convergence of the BCD algorithm (without parameter estimation) to a stationary point is addressed in the theoretical results in previous work <ref type="bibr">Tseng and Yun (2009)</ref> . There, a general block-coordinatedescent approach is analyzed to solve minimization problems of the form &#119865; ( &#119909; ) = &#119891; ( &#119909; ) + &#120582;&#8462; ( &#119909; ) , which is composed of the sum of a smooth function &#119891; ( &#8901;) and a separable convex function &#8462; ( &#8901;) , in our case &#8462; ( &#119909; ) = 0 . According to Part (e) of Theorem 4.1 in <ref type="bibr">Tseng and Yun (2009)</ref> , if &#119868; &#119905; at the &#119905; 'th iteration is chosen by the generalized Gauss-Seidel rule, &#8899; &#119894; =0 , 1 , &#8230;,&#119879; -1</p><p>which is necessary to ensure that all variables are updated every &#119879; steps and &#57922; is the set of all variables, then each coordinate-wise minimum point of { &#119833; &#119868; &#119905; } is a stationary point of &#119891; ( &#119833; ) .</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>Hyperparameter optimization and latent variable inference for Bayesian factor analysis</head><p>Our goal was to infer the latent variables &#119819; and &#119813; and the hyperparameters for &#119810; and &#120590; 2 , denoted as &#120579;. We assumed that</p><p>&#119819; &#119895; &#8764; &#57914; ( &#120782; , &#119810; ) , for the &#119895;th column,</p><p>and &#119813; &#119894; = &#57914; ( &#120782; , &#119816; ) , for the &#119894; th column.</p><p>(47)</p><p>We first aimed at estimating the hyperparameters &#120579; by marginalizing over &#119819; and &#119813; . The marginal distribution for &#119831; is</p><p>which is intractable. However, we could match the second order moment of &#119831; to the sample covariance &#119826; = X X &#8868; where X is the standardized data sample. The second order moment is derived as follows  We now prove that if inequality 58 is true, then &#119809; &#10928; &#120782; , using proof by contradiction:</p><p>If &#119809; &#57423; &#120782; , then there exists an eigenvector &#119857; s.t. &#119809;&#119857; = &#120582;&#119857;, &#120582; &lt; 0 . Then &#119857; &#8868; &#119809;&#119826; 2 &#119857; = &#120582;&#119857; &#8868; &#119826; 2 &#119857; &#8804; 0 due to the p.s.d. of &#119826; 2 . Similarly, for the same eigenvector &#119857;, we could arrive at the same conclusion for &#119808;&#119809; , i.e., &#119857; &#8868; &#119808;&#119809;&#119857; = &#120582;&#119857; &#8868; &#119808;&#119857; &#8804; 0 . This implies that &#119857; &#8868; ( &#119809;&#119826; 2 + &#119808;&#119809; ) &#119857; &#8804; 0 which is contradict to inequality 58 . Therefore &#119809; &#10928; &#120782; . Now we could continue the deduction as follows, </p></div><note xmlns="http://www.tei-c.org/ns/1.0" place="foot" n="2" xml:id="foot_0"><p>https://github.com/poldracklab/fmriprep</p></note>
		</body>
		</text>
</TEI>
