INTRODUCTION

Image compression is an important step for enabling efficient transmission and storage of images in many applications. It is widely used in biomedical imaging due to the highdimensional nature of data. While traditional image compression methods are based on fixed image transforms [1,2], deep learning (DL) has recently emerged as a powerful data-driven alternative. The majority of DL-based compression methods are based on training autoencoders to be invertible mappings from image pixels to quantized latent representations [3][4][5].

In this work, we seek an alternative to the autoencoder-based compression methods by focusing on a recent paradigm using implicit neural representations (INRs). INR refers to a class of DL techniques that seek to learn a mapping from input coordinates (e.g., (x, y)) to the corresponding physical quantities (e.g., density at (x, y)) by using a multi-layer perceptron (MLP) [6][7][8][9]. Recent studies have shown the potential of INR in image compression [10][11][12][13][14]. The key idea ⇤ These authors contributed equally to this work. This material is based upon work supported by the NSF CAREER award under CCF-2043134.

behind INR based compression is to train a MLP to represent an image and consider the weights of the trained model as the compressed data. One can then reconstruct the image by evaluating the pre-trained MLP on the desired pixel locations. The traditional training strategy for image compression using INRs seeks to enforce image consistency between predicted and groundtruth image pixels. On the other hand, it is well-known that image quality can be improved by infusing prior knowledge on the desired images [15,16]. Based on this observation, we propose Structural regularIzatioN for INR COmpression (SINCO) as new method for improving INRbased image compression using a new structural regularizer. Our structural regularizer seeks to improve the Dice score between the groundtruth segmentation maps and those obtained from the INR compressed image using a pre-trained segmentation network. We validate SINCO on brain MR images by showing that it can lead to significant improvements over the traditional INR-based image compression methods. We show that the combination of the traditional image-consistency loss and our structural regularizer enables SINCO to learn an INR that can better preserve desired image features.

BACKGROUND

INR (also referred to as neural fields) denotes a class of algorithms for continuously representing physical quantities using coordinate-based MLPs (see a recent review [6]). Recent work has shown the potential of INRs in many imaging and vision tasks, including novel view synthesis in 3D rendering [7], video frame interpolation [8], and dynamic imaging [9]. The key idea behind INR is to train a MLP to map spatial coordinates to corresponding observed physical quantities. After training, one can evaluate the pre-trained MLP on desired coordinates to predict the corresponding physical quantities, including on locations that were not part of training. Let c denotes a vector of input coordinates, v the corresponding physical quantity, and M ✓ a MLP with trainable parameter ✓ 2 R n . The INR training can be formulated as

(1) where N 1 denotes the number training pairs (c, v). The common choices for `inr include `2 and `1 norms.

INRs have been recently used for image compression [10][11][12][13] (see also a recent evaluation in medical imaging [14]). COmpressed Implicit Neural representations (COIN) [10] is a pioneering work based on training a MLP by mapping the pixel locations (i.e., c = (x, y)) of an image to the pixel values. The pre-trained MLP in COIN is quantized and then used as the compressed data. In order to reconstruct the image, one can evaluate the model on the same pixel locations used for training. Several papers have investigated the meta-learning approach to accelerate COIN by first training a MLP over a large collection of datapoints and then fine-tuning it on an instance-dependent one [11,12,17]. Two recent papers proposed to regularize INR-based image compression by using `0-and `1-norm penalties on the weights of the MLP to improve the compression rate [11,13].

The structural regularization in SINCO is based on image segmentation using a pre-trained convolutional neural network (CNN) (see a comprehensive review of the topic [18]). There exists a rich body of literature in the context of DL-based image segmentation that can be integrated into SINCO [19][20][21].

To the best of our knowledge, no prior work has considered higher-level structural regularization in the context of INRbased image compression. It is worth mentioning that our structural regularizer is fully compatible to the existing INR compression methods; for example, one can easily combine our structural regularizer with an additional `0-regularizer.

PROPOSED METHOD

SINCO consists of two components: (a) a MLP M ✓ that represents an image by mapping its coordinates to corresponding pixel values and (b) a CNN g ' that predicts a segmentation mask given the compressed image produced by MLP. Specifically, let x 2 R H⇥W denote an image of height H and width W that we seek to compress. Let c 2 R HW ⇥2 represents all the pixel locations within x. Then, M ✓ is trained to take c as input and predict all the corresponding HW pixels values. We format the output of M ✓ to be the compressed image b x 2 R H⇥W . The function g ' denotes the segmentation CNN that takes the compressed image and predicts a segmentation mask ŝ = g ' (b x). The SINCO pipeline is illustrated in Fig. 1.

Network Architecture

We implemented two different architectures for M ✓ : (a) SIRENfoot_1 [22] that consists of linear layers followed by sine activation functions and (b) NeRF that incorporates the positional encoding to expand c before passing it to M ✓foot_2 [7] (c) = sin(2 0 ⇡c), cos(2 0 ⇡c)... sin(2 where L f > 0 denotes the number of frequencies. M ✓ of NeRF consists of linear layers followed by ReLU activation functions. For the NeRF architecture, we also add residual connections from the input to intermediates layers. We subsequently denote SINCO based on the two MLPs as SINCO (SIREN) and SINCO (INR). We adopt the widelyused U-Net [21] architecture as the CNN for the segmentation network g ' .

Training Strategy

SINCO is trained by minimizing the following loss

where s denotes the reference segmentation mask of x, and 0 is a parameter that balances image consistency with structural regularization. We implement `compress as a `2-norm between the compressed image and the groundtruth image. We implement `regularize as 1 Dice(ŝ, s), where Dice(ŝ, s) is the Dice score coefficent the segmentation mask predicted from the compressed image and the groundtruth one. In eq. ( 3), the segmentation network g ' is assumed to be pre-trained, which implies that we only optimize the parameters of M ✓ during training. Note that when = 0, eq. ( 3) is equivalent to the traditional INR-based image compression. After the training, we follow the approach in [10] by quantizing the weights of M ✓ from 32-bits to 16-bits.

NUMERICAL EXPERIMENTS

Setup and comparison

For our experiments, we used MR images of brain tumors with the corresponding segmentation masks obtained from the Decathlon dataset [23] (Task01). We selected ten test images with resolution 240 ⇥ 240. Fig. 2 illustrates several images used in our experiments with the corresponding segmentation masks for tumors.

We compared SINCO against two reference INR methods: (a) COINfoot_3 , a recent INR method discussed in Sec. 2, and (b) Vanilla INR, a variant of SINCO (INR) that sets in (3) to 0. Note that since COIN also uses SIREN as its MLP architecture, it can be viewed as SINCO (SIREN) using a different loss. For all methods we used the same compression rate quantified with bits per pixel (bpp) bpp = bit per parameters ⇥ #parameters #pixels .

We set bbp to 1.2 in our experiments, corresponding to the compression rate of 2% relative to the raw file size. We evaluated all the methods on both image compression and image segmentation using compressed images. For image compression, we used two widely-used quantitative metrics, peak signal-to-noise ratio (PSNR), measured in dB, and structural similarity index (SSIM). We used the Dice score coefficient to evaluate image segmentation.

Implementation Details

We followed [21] to train the segmentation network g ' . We used about 500 images from the same dataset in training. The corresponding training loss can be written as

where M 1 denotes the number of training samples and `seg corresponds to binary cross entropy [21]. We used Adam [24] as an optimizer with learning rate 0.0001. We set training epochs to 75 and batch size to 8. In the training of SINCO, we set L f in (2) to 12. We have experimented with different values of in (3). The best results were obtained when = 1. We used Adam [24] as an optimizer with learning rate 0.001. We set training epochs to 50,000. We performed our experiments on a machine equipped with an Intel Xeon Gold 6130 Processor and an NVIDIA GeForce RTX 1080Ti GPU.

Results

Figure 2 shows that the performance of SINCO (INR) can be improved by using the optimized value of .

CONCLUSION

We present SINCO as a new structurally regularized image compression method using implicit neural representation.

The key idea behind SINCO is to use a pre-trained segmentation network to ensure that the INR compressed images produce accurate segmentation masks. Our experiments on brain MR images show that SINCO can quantitatively and qualitatively outperform traditional INR approaches. In the future work, we will further investigate SINCO for higher dimensional data compression (e.g., 3D or 4D MRI [25]) and leverage recent development of meta-learning strategies to accelerate the INR training [17].