More Like this

1. Segmentation for Images of a Single Stem Cell Using Morphological Operations and Statistical Region Merging

   https://doi.org/10.1109/ICSIP57908.2023.10270995  

   Zheng, Xiqiang (July 2023, IEEE)

   For some images of a single stem cell, the boundary delineating the cell is discontinuous or blurry at some locations. If the statistical region merging (SRM) method is applied directly on such images, the image segmentation results may not be ideal. In this paper, for each such image, we add some gradient information into the image; then apply a discontinuous filter on the image so that the image is smoothed a bit and the edges of the image are kept well. Next, closing operations of morphology are applied on the filtered image; and the processed image is segmented using SRM. Finally, apply a threshold on the segmented image to obtain a binary image; apply a hole-filling function to the binary image; extract the biggest connected component in the hole-filled image; and apply a linear transform on the image of the biggest component to match the input image as well as possible in terms of the least squares fitting. This transformed image is the segmentation result. We have applied SRM using connectivity of 4 and 8 as well as a hexagonal lattice. The corresponding segmentation results are tabulated for convenient comparisons; and the results can show that the proposed method may be helpful for the segmentation of such images. 

   more » « less
2. All-optical geometric image transformations enabled by ultrathin metasurfaces

   https://doi.org/10.1038/s41467-023-43981-x  

   Zhang, Xingwang; Zhang, Xiaojie; Duan, Yao; Zhang, Lidan; Ni, Xingjie (December 2023, Nature Communications)

   Abstract Image processing plays a vital role in artificial visual systems, which have diverse applications in areas such as biomedical imaging and machine vision. In particular, optical analog image processing is of great interest because of its parallel processing capability and low power consumption. Here, we present ultra-compact metasurfaces performing all-optical geometric image transformations, which are essential for image processing to correct image distortions, create special image effects, and morph one image into another. We show that our metasurfaces can realize binary image transformations by modifying the spatial relationship between pixels and converting binary images from Cartesian to log-polar coordinates with unparalleled advantages for scale- and rotation-invariant image preprocessing. Furthermore, we extend our approach to grayscale image transformations and convert an image with Gaussian intensity profile into another image with flat-top intensity profile. Our technique will potentially unlock new opportunities for various applications such as target tracking and laser manufacturing. 

   more » « less
3. Exploring CLIP for Real World, Text-based Image Retrieval

   https://doi.org/10.1109/aipr60534.2023.10440710  

   Sultan, Manal; Jacobs, Lia; Stylianou, Abby; Pless, Robert (September 2023, IEEE)

   Abstract—We consider the ability of CLIP features to support text-driven image retrieval. Traditional image-based queries sometimes misalign with user intentions due to their focus on irrelevant image components. To overcome this, we explore the potential of text-based image retrieval, specifically using Contrastive Language-Image Pretraining (CLIP) models. CLIP models, trained on large datasets of image-caption pairs, offer a promising approach by allowing natural language descriptions for more targeted queries. We explore the effectiveness of textdriven image retrieval based on CLIP features by evaluating the image similarity for progressively more detailed queries. We find that there is a sweet-spot of detail in the text that gives best results and find that words describing the “tone” of a scene (such as messy, dingy) are quite important in maximizing text-image similarity. 

   more » « less
4. Sketch-to-image synthesis via semantic masks

   https://doi.org/10.1007/s11042-023-16704-z  

   Baraheem, Samah S; Nguyen, Tam V (March 2024, Multimedia Tools and Applications)

   Sketch-to-image is an important task to reduce the burden of creating a color image from scratch. Unlike previous sketch-to-image models, where the image is synthesized in an end-to-end manner, leading to an unnaturalistic image, we propose a method by decomposing the problem into subproblems to generate a more naturalistic and reasonable image. It first generates an intermediate output which is a semantic mask map from the input sketch through instance and semantic segmentation in two levels, background segmentation and foreground segmentation. Background segmentation is formed based on the context of the foreground objects. Then, the foreground segmentations are sequentially added to the created background segmentation. Finally, the generated mask map is fed into an image-to-image translation model to generate an image. Our proposed method works with 92 distinct classes. Compared to state-of-the-art sketch-to-image models, our proposed method outperforms the previous methods and generates better images. 

   more » « less
5. Image synthesis: a review of methods, datasets, evaluation metrics, and future outlook

   https://doi.org/10.1007/s10462-023-10434-2  

   Baraheem, Samah Saeed; Le, Trung-Nghia; Nguyen, Tam V. (February 2023, Artificial Intelligence Review)

   Image synthesis is a process of converting the input text, sketch, or other sources, i.e., another image or mask, into an image. It is an important problem in the computer vision field, where it has attracted the research community to attempt to solve this challenge at a high level to generate photorealistic images. Different techniques and strategies have been employed to achieve this purpose. Thus, the aim of this paper is to provide a comprehensive review of various image synthesis models covering several aspects. First, the image synthesis concept is introduced. We then review different image synthesis methods divided into three categories: image generation from text, sketch, and other inputs, respectively. Each sub-category is introduced under the proper category based upon the general framework to provide a broad vision of all existing image synthesis methods. Next, brief details of the benchmarked datasets used in image synthesis are discussed along with specifying the image synthesis models that leverage them. Regarding the evaluation, we summarize the metrics used to evaluate the image synthesis models. Moreover, a detailed analysis based on the evaluation metrics of the results of the introduced image synthesis is provided. Finally, we discuss some existing challenges and suggest possible future research directions. 

   more » « less