Introduction

The clinical histopathology workflow is used to introduce contrast to allow for microscopic visualization of tissue constituents. However, the traditional staining process is time-consuming, labor-intensive, expensive, and destructive to the tissue specimen, and only one stain can be performed on each tissue section. Recently, virtual staining of unlabeled tissue sections, avoiding histochemical staining steps, has been demonstrated using deep neural networks [1]. Here we present a method that allows the users to utilize a single deep neural network to generate multiplexed and micro-structured virtual stains on the same tissue section, [2] which is currently not feasible with standard histological staining processes.

Experimental Methods

We trained a class-conditional Generative Adversarial Network (GAN) to enable multiplexing and micro-structuring of virtual stains. The network has two inputs: (1) autofluorescence microscopy images of unlabeled tissue sections, and (2) a digital staining matrix defined by the user (see Fig. 1) [2].

The label (ground truth) used to train the staining network consists of images of the same tissue captured after the standard histochemical staining. During the training, we mixed three different staining datasets, which consisted of the histochemically stained brightfield microscopy images (i.e., H&E, Masson's trichrome, and Jones' silver stains) [2].

Results and Discussion

As shown in Fig. 1, we demonstrated a technique that allows a single neural network to virtually stain unlabeled autofluorescence images of tissue sections; the neural network was trained for virtual staining of H&E, Masson's trichrome, and Jones' silver stain (kidney tissue). By editing the digital staining matrix that is input to the network, we were able to perform "stain micro-structuring", defining a micro-structure map that virtually applies different stains to each specific area or region of interest on the unlabeled tissue area [2]. The same digital staining matrix also enables the blending of different virtual stains. Stain blending is performed by mixing two or more stains in the digital staining matrix with controllable ratios in the desired tissue areas (see Figs. 12. This stain blending can virtually synthesize entirely new stains, as illustrated in Fig. 2. To show that high-quality stains are generated using this technique, we quantitatively compared the inference outcome of the multi-stain neural network with the results of a previously validated single-stain neural network [1] as well as histochemicallystained ground truth by calculating the corresponding structural similarity index (SSIM) values. The comparison results (shown in Table 1) reveal that the virtual staining images generated by this network are highly accurate, and statistically equivalent to previous state-of-the-art virtual staining techniques which perform a single stain [2].

In conclusion, we demonstrated multiplexed and micro-structured virtual staining on unlabeled tissue. We believe that this technique has the potential to fundamentally change and improve the workflow of the tissue staining process including human and animal tissue.