<?xml-model href='http://www.tei-c.org/release/xml/tei/custom/schema/relaxng/tei_all.rng' schematypens='http://relaxng.org/ns/structure/1.0'?><TEI xmlns="http://www.tei-c.org/ns/1.0">
	<teiHeader>
		<fileDesc>
			<titleStmt><title level='a'>An Adversarial Learning Based Approach for 2D Unknown View Tomography</title></titleStmt>
			<publicationStmt>
				<publisher></publisher>
				<date>2022 Fall</date>
			</publicationStmt>
			<sourceDesc>
				<bibl> 
					<idno type="par_id">10351784</idno>
					<idno type="doi">10.1109/TCI.2022.3197939</idno>
					<title level='j'>IEEE Transactions on Computational Imaging</title>
<idno>2573-0436</idno>
<biblScope unit="volume"></biblScope>
<biblScope unit="issue"></biblScope>					

					<author>Mona Zehni</author><author>Zhizhen Zhao</author>
				</bibl>
			</sourceDesc>
		</fileDesc>
		<profileDesc>
			<abstract><ab><![CDATA[The goal of 2D tomography is to recover an image given its projections from various views. It is often presumed that viewing angles associated with the projections are known in advance. Under certain situations, however, these angles are known only approximately or are completely unknown. It becomes more challenging to reconstruct the image from a collection of random projections with unknown viewing directions. We propose an adversarial learning based approach to recover the image and the viewing angle distribution by matching the empirical distribution of the measurements with the generated data. Fitting the distributions is achieved through solving a minmax game between a generator and a critic based on Wasserstein generative adversarial network structure. To accommodate the update of the viewing angle distribution through gradient back propagation, we approximate the loss using the Gumbel-Softmax reparameterization of samples from discrete distributions. Our theoretical analysis verifies the unique recovery of the image and the projection distribution up to a rotation and reflection upon convergence. Our extensive numerical experiments showcase the potential of our method to accurately recover the image and the viewing angle distribution under noise contamination.]]></ab></abstract>
		</profileDesc>
	</teiHeader>
	<text><body xmlns="http://www.tei-c.org/ns/1.0" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xlink="http://www.w3.org/1999/xlink">
<div xmlns="http://www.tei-c.org/ns/1.0"><p>FBP reconstructed images from a low-dose sinogram <ref type="bibr">[12]</ref>- <ref type="bibr">[17]</ref> in a supervised manner or provide a prior i.e. regularizer, over the space of target images <ref type="bibr">[18]</ref>, <ref type="bibr">[19]</ref>.</p><p>However, the knowledge of the viewing angles is not always available or accurate. To avoid adverse effects on the quality of the reconstructed image, it is important to account for uncertainties in the viewing angles. Previous methods devoted to 2D UVT estimate the viewing angles either prior to <ref type="bibr">[20]</ref>- <ref type="bibr">[25]</ref> or jointly with the image reconstruction <ref type="bibr">[26]</ref>. In addition, in limited settings, <ref type="bibr">[27]</ref>- <ref type="bibr">[30]</ref> bypass the estimation of the projection views via the use of invariant features.</p><p>In this paper, we present an unsupervised adversarial learning based approach for 2D tomography with unknown random viewing angles, namely UVTomo-GAN. Our approach does not require large paired training sets and reconstructs an image given merely its unordered tomographic measurements. By employing generative adversarial networks (GAN) <ref type="bibr">[31]</ref>, our approach recovers the image and viewing angle distribution through matching the distributions of the generated projections with the measurements. Our proposed method is inspired by CryoGAN <ref type="bibr">[32]</ref> in which a 3D cryo-EM map is reconstructed given a large set of noisy projection images with unknown orientations by employing Wasserstein-GAN <ref type="bibr">[33]</ref>. The main assumption in CryoGAN is that the distribution of the orientations of the particles is known beforehand. However, in cryo-EM experiments, the distribution of the orientations is hard to obtain a-priori. Therefore, under the 2D UVT set-up, we remove the assumption that the viewing angle distribution is given and develop a new approach to recover both the viewing angle distribution and the 2D image simultaneously.</p><p>To recover the viewing angle distribution in a GAN framework, the original generator's loss involves sampling from the viewing angle distribution which is non-differentiable. To enable the flow of gradients in the backward pass through this non-differentiable operator, we modify the loss function at the generator side using Gumbel-Softmax approximation of samples from a categorical distribution <ref type="bibr">[34]</ref>. Our proposed idea is general and applicable to a vast range of similar inverse problems which involve latent variables with unknown probability distributions such as multi-segment reconstruction <ref type="bibr">[35]</ref>.</p><p>This manuscript is an extension of our previous work <ref type="bibr">[36]</ref>. In this paper, we use the truncated Hartley-Bessel expansion of the image in the Hartley domain in our reconstruction pipeline. This truncated expansion regularizes the images and allows for the direct use of central slice theorem (CST) to generate the projections efficiently. As noted in <ref type="bibr">[24]</ref>, 2D tomography from noisy projections taken at unknown random directions with non-uniform distribution is more challenging than its 3D analogue, since we cannot directly use the geometric constraints given by CST in 3D. Our theoretical analysis and numerical results affirm the ability of our method in recovering the image and projection distribution accurately from both clean and noisy measurements.</p><p>The organization of this paper is as follows. Section II summarizes related work to UVT. We introduce the projection formation model and the reconstruction method in sections III and IV. The analysis and experimental results are described in V and VI. The discussions and future directions are presented in VII. We conclude the paper in VIII.</p><p>II. RELATED WORK In this section, we review related literature on 2D UVT and unsupervised solutions for 3D UVT task. 2D UVT: One family of 2D UVT solutions determine the viewing angles first <ref type="bibr">[20]</ref>- <ref type="bibr">[25]</ref> and reconstruct the image given the estimated views subsequently. Other approaches include iterative methods that solve for the 2D image and the viewing angles in alternating steps <ref type="bibr">[26]</ref>. While proven effective, these methods are computationally expensive and sensitive to initialization. In another class of methods, to circumvent the estimation and refinement of the viewing angles, a set of rotation invariant features are estimated from the noisy projections. These features are later on used to reconstruct the unknown image <ref type="bibr">[27]</ref>- <ref type="bibr">[30]</ref>. Note that these methods require only one pass through the projection dataset and are therefore computationally more efficient. However, they are mainly used, when the underlying object is sparse <ref type="bibr">[27]</ref>, <ref type="bibr">[28]</ref>, projections in the form of tilt series are available <ref type="bibr">[30]</ref> or to recover a low-resolution ab-initio model <ref type="bibr">[29]</ref>. Adversarial Learning for 3D UVT: Gupta et al. in CryoGAN <ref type="bibr">[32]</ref> proposed an unsupervised learning approach through a distribution matching lens for cryo-EM single particle reconstruction. In CryoGAN, the goal is to estimate the underlying 3D density such that the distribution of the observed projection image dataset and the one generated from the estimated volume match. Due to its distribution matching criterion, CryoGAN bypasses the estimation of individual projection parameters. In CryoGAN, the distribution distance is chosen as Wasserstein-1 (W 1 ), i.e. Earth Mover's distance. Thus, the reconstruction problem is stated as:</p><p>where P real is the distribution of the observed (i.e. real) projection image dataset. Also, P sim (v; p latent ) is the distribution of the simulated projection image dataset generated from the volume v following an a-priori known distribution for the latent variables p latent . In a cryo-EM setup, each projection image is obtained from the volume following a forward model. This forward model is parameterized by the projection view, in-plane translation and the contrast transfer function (CTF) parameters corresponding to the projection image. Given a projection image dataset, the collection of these parameters (projection view, in-plane translation and CTF parameters), is considered a random latent variable with p latent probability distribution, which in CryoGAN is assumed to be known. Thus, to sample from P sim given v and p latent , one samples latent variables based on p latent and then adopt the projection forward model to generate random simulated projections of v.</p><p>As computing W 1 between two high-dimensional distributions is highly intractable, W 1 minimization is often done in its dual form, following Kantrovich-Rubinstein duality <ref type="bibr">[33]</ref>:</p><p>(2) where f represents a 1-Lipschitz function, mapping its input (i.e. a projection image) to a single real-valued score.</p><p>Due to the close link between (2) and Wasserstein-GAN (WGAN) frameworks <ref type="bibr">[33]</ref>, CryoGAN specifically proposes the use of WGAN with gradient-penality (GP) (WGAN-GP) <ref type="bibr">[37]</ref> to solve <ref type="bibr">(2)</ref>. In a WGAN-GP setup, the mapping f is modeled via a neural network named critic and its 1-Lipschitz continuity constraint is enforced via the GP term.</p><p>In this paper, we extend the CryoGAN framework for the 2D UVT problem defined in section III. In a 2D UVT setting, the projection views form the underlying latent variable. Unlike CryoGAN, we assume the latent variable probability distribution (p latent in CryoGAN context) is unknown and we develop a novel approach to handle its joint recovery with the image. In addition, we compare our method against the baselines formed by the adaptations of CryoGAN for 2D UVT in section VI.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>III. PROJECTION FORMATION MODEL AND PROBLEM</head><p>FORMULATION We define the 1D projection formation model as,</p><p>where I : B 2 &#8594; R 1 is an unknown 2D compactly supported image in the unit ball B 2 we wish to estimate. We restrict I to the space of absolute and square integrable functions on B 2 , i.e. I &#8712; L 1 (B 2 ) &#8745; L 2 (B 2 ). P &#952; denotes the tomographic projection operator that takes the line integral along the parallel beams whose normal direction makes an angle &#952; &#8712; [0, 2&#960;) with the x-axis,</p><p>where x = [x, y] T represents the 2D Cartesian coordinates. R &#952; is a 2 &#215; 2 rotation matrix associated with &#952;. As I is compactly supported in B 2 , its projection along any direction would also be compactly supported in the unit ball, i.e.</p><p>). We assume the viewing angles {&#952; } L =1 are unknown and randomly drawn from an unknown distribution p. Finally, the discretized projection lines of length m are corrupted by additive white Gaussian noise &#949; with zero mean and variance &#963; 2 . Here we consider &#963; to be known, although an unbiased estimator of &#963; is attainable from the variance of the boundary pixels of the projections that only contain noise <ref type="bibr">[24]</ref>.</p><p>In this paper, given a large set of noisy projections, i.e. {&#950; } L =1 , we aim to recover the image I and the unknown distribution of the viewing angles p.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>IV. METHOD A. Image Representation</head><p>To alleviate the computational cost of generating projections in practice, (3) is evaluated in Fourier domain using nonuniform fast Fourier transform <ref type="bibr">[38]</ref> according to central slice for t = 0, ..., n disc -1 do  Update the critic following gradient ascent steps using the gradient of ( <ref type="formula">13</ref>) with respect to &#966;.  Update c and p using stochastic gradient descent steps by taking the gradients of ( <ref type="formula">20</ref>) with respect to c and p. 9: end while where L denotes the loss, B and b represent the batch size and the index of a sample in the mini-batch, respectively. Also, &#950; real and &#950; syn mark the real and synthesized projections in Hartley domain. &#950; syn is generated from the estimated image c and projection distribution p following</p><p>In our experiments, we used spectral normalization (SN) <ref type="bibr">[42]</ref> to regularize the critic and found that SN is sufficient for stabilizing the training. Following common practice, we solve <ref type="bibr">(14)</ref> by alternating updates between &#966; and the generator's variables, i.e. c and p, based on the associated gradients.</p><p>The loss at the generator side for a fixed D &#966; is,</p><p>While ( <ref type="formula">15</ref>) is differentiable with respect to c, its gradient of p is not defined, as it involves sampling &#952; b from the distribution p. This hinders updating p through gradient back-propagation.</p><p>To address this, we aim to design an alternative approximation of ( <ref type="formula">15</ref>) which is differentiable with respect to p.</p><p>To accommodate this approximation, we first discretize the support of the viewing angles, i.e. [0, 2&#960;) into N &#952; equal-sized bins. This makes p a probability mass function (PMF) of length N &#952; with the following properties:</p><p>Now p corresponds to a discrete or categorical distribution over &#952;, which implies the sampled viewing angles from p can only belong to N &#952; discrete categories. Therefore, we re-write the loss function <ref type="bibr">(15)</ref> as:</p><p>A closer look at <ref type="bibr">(17)</ref> reveals that &#948;(&#952; t -&#952; b ), &#952; b &#8764; p is a sample from the discrete distribution p. This enables us to incorporate the notion of Gumbel-Softmax distribution and approximate <ref type="bibr">(15)</ref> as:</p><p>where &#964; is the softmax temperature factor. As &#964; &#8594; 0, r i,b (p) &#8594; one-hot (argmax i [g b,i +log(p i )]). Moreover, to obtain samples from the Gumbel(0, 1) distribution, it suffices to draw u &#8764; Unif(0, 1), g = -log(-log(u)) <ref type="bibr">[34]</ref>. Note that due to the reparametrization trick applied in <ref type="bibr">(18)</ref>, the approximated generator's loss has a tangible gradient with respect to p. We also add prior knowledge on the image and projection distribution in the form of regularization terms. Hence, the regularized loss function we optimize at the generator side is:</p><p>where we include total variation (TV) and 2 regularization terms for the image, with &#947; 1 and &#947; 2 weights. To construct the TV of the image in terms of c, we use <ref type="bibr">(11)</ref> to render I on a Cartesian grid in spatial domain and then compute total variation of I. Furthermore, we assume that the unknown PMF is a piece-wise smooth function of viewing angles (which is a valid assumption especially in single particle analysis in cryo-EM <ref type="bibr">[43]</ref>), therefore adding TV and 2 regularization terms for the PMF with &#947; 3 and &#947; 4 weights. We present the pseudo-code for UVTomo-GAN in Alg. 1.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>C. Maximum Marginalized Likelihood Estimation via Expectation-Maximization</head><p>As a baseline for UVTomo-GAN, we consider maximum marginalized likelihood estimation (MMLE). We solve MMLE in Fourier domain via expectation-maximization (EM) and represent F(I) with its expansion coefficients a on Fourier-Bessel bases. Thus, MMLE is formulated as</p><p>To solve <ref type="bibr">(21)</ref>, we take the gradients with respect to a and p and set them to zero. For p, we further impose</p><p>This yields the following alternating updates for a and p, in the form of:</p><p>(M-step) :</p><p>where </p><p>) where r i,j denotes the probability that the i-th projection is associated with &#952; j angle and t is the iteration index. Also, H &#952; a generates the projection at &#952; direction in Fourier domain given FB expansion coefficients a. In (23), A t is indexed by (k, q) and (k , q ) pairs and the discretization in &#958; is identical to the projection dataset. The advantages of using truncated FB expansion is that: (1) similar to HB representation, it provides an implicit regularization on the image, and (2) building matrix A t in <ref type="bibr">(22)</ref> in each iteration only requires rescaling the entries of a pre-computed matrix</p><p>In ( <ref type="formula">22</ref>)-( <ref type="formula">23</ref>), we update the probabilistic angular assignments for the projections in the E-step while updating a and p in the M-step. Note that, in the absence of noise, i.e. &#963; = 0, the E-step reduces to template matching <ref type="bibr">[44]</ref>. To solve a t from the equation A t a t = b t , we use preconditioned conjugate gradient descent <ref type="bibr">[45]</ref>.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>D. Computational Complexity</head><p>We conclude this section by comparing the computational complexity per iteration of UVTomo-GAN and EM. UVTomo-GAN Complexity: Based on Alg. 1, we split the computational cost of UVTomo-GAN between: 1) the critic and 2) the generator (i.e. c and p) updates. Let C D denote a fixed computational cost related to forward and backpropagation passes through the critic D &#966; . As expected, C D depends on the batch size, network architecture and the size of its input. Thus, the larger the critic network, the higher the C D . For our critic architecture, we use a cascade of N m fully connected (FC) layers with intermediate ReLU non-linearities. Therefore, C D points to the cost of matrix multiplications and backward passes through these N layers. Furthermore, we keep the input and output sizes of these FC layers to be O(m) (m is the image/projection size). Therefore,</p><p>). As these operations can be parallelized on GPU, forward and backward passes through D &#966; are timeefficient. For batch size</p><p>For updating the generator according to <ref type="bibr">(18)</ref>, first we generate N &#952; = O(m) projections or templates. This is done in O(m 3 ). A thorough discussion on the derivation of this computational complexity term is deferred to Appendix IX-A.</p><p>In our implementation of (18), instead of using B different noise realizations { &#949; b } B b=1 for each of the clean templates, we consider N &#952; noisy templates in total. This means the loss function we use at the generator side is:</p><p>Indeed in the absence of noise, ( <ref type="formula">27</ref>) matches <ref type="bibr">(18)</ref>. However, in the noisy case, the benefits of ( <ref type="formula">27</ref>) are two-fold: 1) having the same performance as (18) empirically, 2) reducing the number of passes through the critic.</p><p>Consequently, adding up the cost of passing N &#952; projection templates through D &#966; leads to a total computational cost of O(m 3 + mC D ) per generator update step. We update c and p every n disc iterations. Therefore, the average cost of UVTomo-GAN per iteration including the generator and critic's updates is O(</p><p>EM Complexity: For EM, we specify the computational cost of E-step and M-step. At each E-step, we generate N &#952; projection templates. If these templates are generated following CST and using the non-uniform Fourier transform of the image, they require O(m 2 log m) computations. Next, we update the angular assignments of L projections by comparing them against O(m) templates, hence a cost of O(m 2 L). Then, the total cost of</p><p>For the M-step, computing b t from the projections costs O(m 2 L) (or O(m log mL) if using FFT) while updating FB coefficients a in (23) using conjugate gradient descent has O( &#8730; &#954;&#969;) computational cost <ref type="bibr">[45]</ref> where &#969; is the number of non-zeros of A t and &#954; is its condition number. Note that &#969; = O(&#951; m 3 ) depends on the number of non-zero elements in p t , i.e. &#951;. If all entries in p t are non-zero (&#951; = O(m)), then the M-step's computational cost is O( &#8730; &#954; m 4 ). Finally, the overall</p><p>In terms of convergence, we empirically observe that UVTomo-GAN requires more training iterations. We attribute this to the difference between the convergences of stochastic gradient descent used in UVTomo-GAN versus full batch processing in EM. On the other hand, we show that while UVTomo-GAN is robust to the choice of initialization, EM is likely to get stuck in a bad locally optimal solution with random initialization. This observation is also reported in cryo-EM settings in <ref type="bibr">[43]</ref>, <ref type="bibr">[46]</ref>.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>V. ANALYSIS</head><p>In this section, we first define our notations and then formally state the reconstruction guarantees of UVTomo-GAN.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>A. Notations</head><p>We assume the image f &#8712; L 1 (B 2 )&#8745;L 2 (B 2 ) has a bandlimit 0 &lt; s &#8804; 0.5 and compactly supported in the unit ball B 2 . In addition,</p><p>s (&#958;)cas(k&#952;). Thus, the Hartley transform of f is expanded on a HB basis set. A measurement &#950; associated with the projection angle &#952; &#8764; p is &#950; = P &#952; f + &#949; with &#949;[n] &#8764; q &#949; denoting additive IID noise. We assume q &#949; has full support in Fourier domain, i.e. {Fq &#949; }(&#969;) = 0, &#8704;&#969;.</p><p>Let O(2) denote the group of all possible rotations and reflections, i.e. &#915; T &#915; = I and det(&#915;) = &#177;1, &#8704;&#915; &#8712; O(2). The action of the O(2) group on f is defined as,</p><p>) where x = [x, y] denotes the Cartesian coordinate. On the other hand, the action of &#915; on a probability distribution p defined over [0, 2&#960;) manifests as a combination of flip or circular shift. The group O(2) partitions the space of span{u s k,q } &#8486; into  </p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>B. Theoretical Results</head><p>Here we elaborate upon the theoretical reconstruction guarantees of our proposed method. Theorem 1: Consider f, g &#8712; L 1 (B 2 ) &#8745; L 2 (B 2 ) and the associated bounded probability distributions p f and p g on the viewing angles distributed in [0, 2&#960;). Then,</p><p>Furthermore, if f = &#915;g, &#915; &#8712; O(2), then p f = &#915;p g . The proof is provided in Appendix IX-B. Intuitively, Theorem 1 states that if f and g have the same induced clean projection distribution, then the underlying objects and projection distributions are equivalent up to a rotation and reflection. We link the proof of this theorem to unique angular recovery in unknown view tomography <ref type="bibr">[20]</ref>, <ref type="bibr">[21]</ref>. Theorem 2: Assume f &#8712; L 1 (B 2 ) &#8745; L 2 (B 2 ) denoting the ground truth (GT) image and p representing the bounded GT probability distribution over the viewing angles &#952; &#8712; [0, 2&#960;). Let f and p stand for the recovered image and the bounded probability distribution after the convergence of UVTomo-GAN. Consider the asymptotic case as L &#8594; &#8734;. Then,</p><p>for a unique &#915; &#8712; O(2).</p><p>The proof is available in Appendix IX-C. This theorem validates that upon the convergence of UVTomo-GAN in the presence of noise and infinite number of noisy projections, the GT image and viewing angle distribution is recovered up to a rotation-reflection transformation. We defer the study of sample complexity of UVTomo-GAN with finite size projection dataset to future work.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>VI. NUMERICAL RESULTS</head></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>A. Experiment Setup</head><p>Datasets: To evaluate the generalization of our method to images of different properties, we conduct experiments on four different images (for additional results, refer to Appendix IX-F). Two are biomedical images of lung and abdomen from low dose CT (LDCT) dataset <ref type="bibr">[47]</ref>. We defined the third image as a set of randomly located and shaped ellipses with various intensities. For the last image, we generated the 3D map of 100S Ribosome <ref type="bibr">[48]</ref> using its protein sequence in Chimera <ref type="bibr">[49]</ref> and took a 2D projection of the generated map along a random view. All images are resized to 101 &#215; 101 dimension. We refer to these images as Lung, Abdomen, Ellipses and a 2D projection image of 100S Ribosome (Rib-Proj). We synthesize the real projection dataset in Hartley domain following <ref type="bibr">(10)</ref> where p is a smooth probability distribution over the viewing angles and is chosen randomly. To generate the real dataset, we discretize the projection angle domain [0, &#960;) with 240 equal sized bins and use non-uniform polar FFT <ref type="bibr">[50]</ref> and CST to generate the projections. We also add the flipped projections to the dataset, such that &#952; covers [0, 2&#960;). This means p(&#952;) = p(&#952; + &#960;), for &#952; &#8712; [0, &#960;). Therefore, when estimating p, we only recover p in [0, &#960;) range. Throughout this draft, we visualize p on [0, &#960;). For the reconstruction, we consider a coarser grid for the viewing angles with N &#952; = 240 bins for the interval [0, 2&#960;). This way we are taking into account the approximated discretization of &#952; at the reconstruction time which might differ from how the real viewing angles are obtained. We study two noise regimes: 1) no noise, and 2) noisy with SNR = 3, SNR denoting the ratio of signal-to-noise variance of the projections, We set &#947; 1 = 0.001 and &#947; 2 = 0 for Ellipses while having &#947; 1 = &#947; 2 = 0 for the Rib-Proj reconstruction. In the noisy case, to obtain the best results in various settings and take into account the difference in the projection datasets, we select &#947; 1 from {0.0005, 0.001, 0.002, 0.005} and &#947; 2 from {0.0005, 0.005, 0.02, 0.04}.</p><p>We have separate learning rates for D &#966; , c and p denoted by &#945; &#966; , &#945; c and &#945; p , but often choose &#945; &#966; = &#945; c . We select the initial values of &#945; &#966; , &#945; c and &#945; p from [0.002, 0.01] with a step-decay schedule. We update D &#966; , c and p using stochastic gradient descent (SGD) steps. We clip the gradients of D &#966; and c by 1 and 10 respectively and normalize the gradients of p to have norm 0.1. We train the critic n disc = 4 times per updates of c and p. Although, after training for a while, we increase the frequency of updating c and p by setting n disc = 2. Once converged, we use the reconstructed HB expansion coefficients to re-render the image in spatial domain according to <ref type="bibr">(11)</ref>.</p><p>Our critic consists of four fully connected (FC) layers with In each row, the subplots share the same vertical axis. For no noise settings, dT V is computed between p (red) and original p (green), while for the noisy case, dT V is computed between p (red) and sample estimation of p (blue).</p><p>image emerges, the details are not successfully recovered. This highlights the importance of updating p to retrieve details accurately in the reconstruction. A similar observation, although in a different setting is reported in <ref type="bibr">[32]</ref>, <ref type="bibr">[52]</ref>. Furthermore, in the clean case, GLT is able to recover the correct ordering of the viewing angles. However, as the viewing angle distribution is non-uniform, assigning equispaced angles to the sorted projections causes a distorted reconstructed image. On the other hand, MADE+GL is able to reconstruct the image accurately.</p><p>For SNR = 3, while GLT's performance on the Lung and Ellipses images is similar to the clean case, GLT's sorting of the projections for Abdomen and Rib-Proj images is erroneous despite tuning the hyperparameters (see Appendix IX-E). Furthermore, we find the angle differences output by MADE for SNR = 3 extremely noisy. This led to an erroneous angular difference estimation and incorrect projection embedding. As MADE+GL failed in reconstructing all images at SNR = 3, we excluded the results of this baseline in Fig. <ref type="figure">5</ref>.</p><p>In the presence of noise, we noticed that to obtain better results for EM starting from a random initialization, in the Estep <ref type="bibr">(22)</ref>, we need to inflate the noise standard deviation &#963;, otherwise EM can get stuck easily at poor local optima. In our EM experiments, we inflated &#963; by &#8730; 2 for all datasets. Fig. <ref type="figure">5</ref> (and Fig. <ref type="figure">10</ref> in Appendix IX-F) display the effect of noise in the final reconstruction. We observe that the presence of noise makes the reconstruction task more challenging and degrades the reconstruction quality compared to the no noise case. This happens as the critic is having a harder time distinguishing signal from noise given the noisy projections.</p><p>Overall, in the no noise setting, among baselines with unknown or assumed uniform distribution, MADE+GL and our method perform the best in terms of PSNR and CC. In the noisy case, our approach alongside Adapted CryoGAN with unif. p are the top-performing methods. Quality of Reconstructed p: Comparison between the GT distribution of the viewing angles and the one recovered by UVTomo-GAN with unknown p is provided in Fig. <ref type="figure">7</ref> (and Fig. <ref type="figure">11</ref> in Appendix IX-F). Note that the recovered p matches the GT distribution both visually and quantitatively in terms of TV distance. Although, the quality of the recovered PMF in the noisy cases (Fig. <ref type="figure">7-(b), (d</ref>), (f), (h)) is not as good as the no noise case, it still closely resembles the GT projection distribution. This shows the ability of our approach to recover p accurately under different distributions and noise regimes. Convergence: To evaluate the effect of using HB representation on the convergence, we compare against an experiment with pixel domain representation of the image. We call this baseline pixel UVTomo-GAN versus our method HB UVTomo-GAN. In this comparison, we use the same dataset, initialization, batch-size, learning rate decay and schedules for both pixel and HB UVTomo-GANs. For HB UVTomo-GAN, to only examine the effect of the representation, we use no TV regularization on the image, i.e. &#947; 1 = 0. However, for pixel UVTomo-GAN, to further help with the convergence, we set a small TV regularization weight as 5 &#215; 10 -5 and enforce the image to be non-negative by defining it to be the output of a ReLU. For HB UVTomo-GAN, we choose &#945; &#981; = &#945; c = 0.008, &#945; p = 0.0008 while for pixel UVTomo-GAN, we fine-tuned these parameters as &#945; &#981; = &#945; I = 0.01, &#945; p = 0.001, &#945; I denoting the learning rate of the image. To implement the projection operator in pixel domain, we use Astra toolbox <ref type="bibr">[53]</ref>.</p><p>In Fig. <ref type="figure">8</ref>, we show the results of this comparison. While both representations lead to accurate image and p recovery, their convergence behaviours are different. For HB UVTomo-GAN, as we are operating in Hartley domain and the images tend to have larger low-frequency components compared to the high-frequency details, initially the gradients corresponding to lower frequency components are larger, leading to faster updates of c k,q s for smaller (k, q)s. This helps in more stable convergence of HB versus pixel UVTomo-GAN.</p><p>Note that, for HB UVTomo-GAN, we obtain a reasonable image and PMF at early stages of training, i.e., after 20k-40k iterations (which takes roughly 6-12 minutes). As expected, the image is further refined with more training iterations. In addition, we compare the convergence of our method versus Adapted CryoGAN in Fig. <ref type="figure">12</ref> in Appendix IX-G.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>VII. DISCUSSION AND FUTURE WORK</head><p>As noted in <ref type="bibr">[24]</ref>, the angular estimation problem in 2D UVT is more challenging than the 3D problem (such as the cryo-EM single particle reconstruction) when the distribution of the viewing angles is non-uniform. In the 3D problem, the central slice theorem implies that any two central slices share a common line of intersection that can be used to find the unknown imaging directions even when they are not uniformly distributed. However, in the 2D problem, since Intuitively, one can imagine two objects f and g which have the same projections, however the order of the viewing angles of f can be a shuffled version of the viewing angles for g. Now the question that arises is: Given the class of functions f and g belong to, is it possible to have two distinct objects that produce identical projection sets?</p><p>This question is related to the feasibility of unique angle recovery in unknown view tomography, comprehensively studied in <ref type="bibr">[20]</ref>, <ref type="bibr">[21]</ref>. Based on our discussions so far, we seek to prove the following:</p><p>In <ref type="bibr">(35)</ref>, the LHS implies that f and g have the same set of projections, in other words we have: &#8704;&#947; &#8712; {P &#952;i f } N &#952; i=1 , &#947; &#8712; {P &#952;j g} N &#952; j=1 and &#8704;&#947; &#8712; {P &#952;j g} N &#952; j=1 , &#947; &#8712; {P &#952;i f } N &#952; i=1 . To prove the above, we borrow the definitions and various theoretical results in <ref type="bibr">[20]</ref>. Helgasson-Ludwig (HL) consistency conditions <ref type="bibr">[54]</ref> link the geometric moments of a 2D object to its projections. Let v and &#181; define the geometric moment of the image f and its projection as:</p><p>Object moments of order d are the ones that satisfy i + k = d. Let v(f ), denote the set of geometric moments of order d &#8712; D for object f . Given the object moments v(f ), we construct a family of trigonometric polynomials as:</p><p>Given the definition <ref type="bibr">(38)</ref>, we state the HL conditions as:</p><p>We have defined equivalence for 2D images before. If two images are equivalent, then they are related through a rotation and reflection. Similarly, we can define equivalence on the viewing angles. Assume two vectors of viewing angles of length</p><p>As the projection set for f and g objects are the same (based on ( <ref type="formula">35</ref>)), we conclude &#8704;&#952;, &#8707; &#952; such that:</p><p>) After invoking HL conditions <ref type="bibr">(39)</ref> for object f on the RHS of (40) we get:</p><p>) Note that, we have narrowed down the identical projection sets for f and g to <ref type="bibr">(41)</ref>. Now we restate our question as: what is the relationship between &#952; and &#952;?</p><p>To find the answer to this question, we first limit the set of moment orders to d &#8712; D = {1, 2} (as (41) holds for &#8704;d &#8805; 0, we can simply do this). Note that for &#952; &#8712; [0, 2&#960;), the projections corresponding to &#952; &#8712; [&#960;, 2&#960;) are a flipped version of projections associated to &#952; &#8712; [0, &#960;) and do not constitute new information <ref type="bibr">[20]</ref>. Thus, in <ref type="bibr">[20]</ref>, the authors limit their analysis to the projections that are &#960;-distinct, i.e., there are no two angles that are different by a factor of &#960;. Following the same lines, given the projection sets corresponding to &#952;, &#952; &#8712; [0, 2&#960;), we select a &#960;-distinct projection subset by choosing a set of projections that have positive (or negative) 1st order geometric moment. We now invoke Corollary 5 of Theorem 9 in <ref type="bibr">[20]</ref>. We restate this corollary in the following. Corollary 1 (Corollary 5 of Theorem 9 <ref type="bibr">[20]</ref>): Suppose &#952; is a set of &#960;-distinct view angles and N &#952; &gt; 8. Suppose v satisfies the following condition: &#8707;&#946;, &#947; &#8712; R such that:</p><p>If &#952; &#8712; UAS(v) with UAS (unidentifiable angle set) defined as:</p><p>where,</p><p>then, the only view angles &#952; that produce the same projection moments of order D = {1, 2} are equivalent to &#952;. This implies that &#952; &#8764; &#952;.</p><p>Adhering to Corollary 1, if v(f ) satisfies the conditions in <ref type="bibr">(42)</ref> or <ref type="bibr">(43)</ref>, then for d &#8712; {1, 2}, the only viewing angles &#952; for which <ref type="bibr">(41)</ref> holds are equivalent to &#952; and thus &#952; &#8764; &#952;. On the other hand, based on Corollary 1, the viewing angles recovered for f , i.e. &#952;, are equivalent to the GT viewing angles &#952; used for generating the projections of f , i.e &#952; &#8764; &#952;. Based on the transitivity property of equivalence relation, this leads to &#952; &#8764; &#952; Given &#952; &#8764; &#952; &#8764; &#952; and the fact that the projection sets corresponding to the objects f and g are identical, the objects f and g reconstructed from the projection sets and viewing angles would also be the same (up to a rotation and reflection), i.e. [ f ] = [ g]. We now link the reconstructed objects and their ground truths.</p><p>If we have sufficiently large N &#952; , we can directly recover HB expansion coefficients c by solving a set of linear equations linking the projections to the HB expansion coefficients. Given the HB expansion coefficients, we have a continuous representation of the image as defined in <ref type="bibr">(11)</ref>. This leads to f = f and g = g and finally concludes </p><p>Following <ref type="bibr">(46)</ref>, the LHS of ( <ref type="formula">47</ref>) is 0. Thus, based on the non- Fig. <ref type="figure">11</ref>: Comparison between the original GT p (green) used to sample the viewing angles from, the empirical sample distribution of the viewing angles (blue) and the one estimated by our method p (red). For more details on the computation of dT V , refer to section VI-A and Fig. <ref type="figure">7</ref>.</p><p>the performance of UVTomo-GAN in terms of the quality of the recovered projection angle distribution p. We notice that for the Walnut and Shepp-Logan images, in the noisy case, as shown in Fig. <ref type="figure">10</ref>-11, the reconstruction is less sensitive to the quality of the estimated p.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>G. Convergence Results</head><p>We exhibit the convergence curves in terms of PSNR versus training iteration for no noise and noisy experiments in Fig. <ref type="figure">12</ref>. To obtain this curve, at each iteration, we align the reconstructions with the GT. We compare the convergence of UVTomo-GAN versus Adapted CryoGAN and Adapted CryoGAN with unif. p baselines.</p><p>For Adapted CryoGAN with unif. p baseline, after a certain number of iterations, we see no improvement in the reconstructed image. This is attributed to having an inaccurate PMF which hinders the correct distribution matching of synthetic and real measurements. Thus, the high frequency details in the final reconstructed image do not appear correctly (as also seen in Fig. <ref type="bibr">[4]</ref><ref type="bibr">[5]</ref><ref type="bibr">[9]</ref><ref type="bibr">[10]</ref>. This once again indicates the importance of recovering p to have high quality reconstructions.</p></div>
<div xmlns="http://www.tei-c.org/ns/1.0"><head>X. ACKNOWLEDGEMENT</head></div><note xmlns="http://www.tei-c.org/ns/1.0" place="foot" xml:id="foot_0"><p>This article has been accepted for publication in IEEE Transactions on Computational Imaging. This is the author's version which has not been fully edited and content may change prior to final publication. Citation information: DOI 10.1109/TCI.2022.3197939This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see https://creativecommons.org/licenses/by/4.0/</p></note>
		</body>
		</text>
</TEI>
