An official website of the United States government
This work explores the use of a triplet neural net-work for assessing the similarity of paper textures in a collection of Henri Matisse’s lithographs. The available dataset contains digital photomicrographs of papers in the lithograph collection, consisting of four views: two raking light orientations and both sides of the paper. A triplet neural network is first trained to extract features sensitive to anisotropy, and subsequently used to ensure that all papers in the dataset are in the same orientation and side. Another triplet neural network is then used to extract the texture features that are used to assess paper texture similarity. These results can then be used by art conservators and historians to answer questions of art historical significance, such as artist intent.
more »
« less
Semi-supervised Convolutional Triplet Neural Networks for Assessing Paper Texture Similarity
In the context of papers used in the graphic arts, including silver gelatin, inkjet, and wove papers, prior work has studied measures of texture similarity for purposes of classifying such papers. The majority of prior work has been based on classical image processing approaches such as Fourier, wavelet, and fractal analysis. In this work, recent advances in deep learning are used to develop a texture similarity approach for measuring paper texture similarity. Since the available datasets generally lack labels, the convolutional neural network is trained using triplet loss to minimize the feature distance of tiles from the same image while simultaneously maximizing the feature distance of tiles drawn from different images. The approach is tested on three paper texture image databases considered in prior works and the results suggest the proposed approach achieves state-of-the-art performance.
more »
« less
- Award ID(s):
- 1836695
- PAR ID:
- 10349350
- Date Published:
- Journal Name:
- 2020 54th Asilomar Conference on Signals, Systems, and Computers
- Page Range / eLocation ID:
- 142 to 146
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
We present a new scientific document similarity model based on matching fine-grained aspects of texts. To train our model, we exploit a naturally-occurring source of supervision: sentences in the full-text of papers that cite multiple papers together (co-citations). Such co-citations not only reflect close paper relatedness, but also provide textual descriptions of how the co-cited papers are related. This novel form of textual supervision is used for learning to match aspects across papers. We develop multi-vector representations where vectors correspond to sentence-level aspects of documents, and present two methods for aspect matching: (1) A fast method that only matches single aspects, and (2) a method that makes sparse multiple matches with an Optimal Transport mechanism that computes an Earth Mover’s Distance between aspects. Our approach improves performance on document similarity tasks in four datasets. Further, our fast single-match method achieves competitive results, paving the way for applying fine-grained similarity to large scientific corpora.more » « less
-
Abstract PurposeSynthetic digital mammogram (SDM) is a 2D image generated from digital breast tomosynthesis (DBT) and used as a substitute for a full‐field digital mammogram (FFDM) to reduce the radiation dose for breast cancer screening. The previous deep learning‐based method used FFDM images as the ground truth, and trained a single neural network to directly generate SDM images with similar appearances (e.g., intensity distribution, textures) to the FFDM images. However, the FFDM image has a different texture pattern from DBT. The difference in texture pattern might make the training of the neural network unstable and result in high‐intensity distortion, which makes it hard to decrease intensity distortion and increase perceptual similarity (e.g., generate similar textures) at the same time. Clinically, radiologists want to have a 2D synthesized image that feels like an FFDM image in vision and preserves local structures such as both mass and microcalcifications (MCs) in DBT because radiologists have been trained on reading FFDM images for a long time, while local structures are important for diagnosis. In this study, we proposed to use a deep convolutional neural network to learn the transformation to generate SDM from DBT. MethodTo decrease intensity distortion and increase perceptual similarity, a multi‐scale cascaded network (MSCN) is proposed to generate low‐frequency structures (e.g., intensity distribution) and high‐frequency structures (e.g., textures) separately. The MSCN consist of two cascaded sub‐networks: the first sub‐network is used to predict the low‐frequency part of the FFDM image; the second sub‐network is used to generate a full SDM image with textures similar to the FFDM image based on the prediction of the first sub‐network. The mean‐squared error (MSE) objective function is used to train the first sub‐network, termed low‐frequency network, to generate a low‐frequency SDM image. The gradient‐guided generative adversarial network's objective function is to train the second sub‐network, termed high‐frequency network, to generate a full SDM image with textures similar to the FFDM image. Results1646 cases with FFDM and DBT were retrospectively collected from the Hologic Selenia system for training and validation dataset, and 145 cases with masses or MC clusters were independently collected from the Hologic Selenia system for testing dataset. For comparison, the baseline network has the same architecture as the high‐frequency network and directly generates a full SDM image. Compared to the baseline method, the proposed MSCN improves the peak‐to‐noise ratio from 25.3 to 27.9 dB and improves the structural similarity from 0.703 to 0.724, and significantly increases the perceptual similarity. ConclusionsThe proposed method can stabilize the training and generate SDM images with lower intensity distortion and higher perceptual similarity.more » « less
-
null (Ed.)Texture-based features computed on eye movement scan paths have recently been proposed for eye movement biometric applications. Feature vectors were extracted within this prior work by computing the mean and standard deviation of the resulting images obtained through application of a Gabor filter bank. This paper describes preliminary work exploring an alternative technique for extracting features from Gabor filtered scan path images. Namely, features vectors are obtained by downsampling the filtered images, thereby retaining structured spatial information within the feature vector. The proposed technique is validated at various downsampling scales for data collected from 94 subjects during free-viewing of a fantasy movie trailer. The approach is demonstrated to reduce EER versus the previously proposed statistical summary technique by 11.7% for the best evaluated downsampling parameter.more » « less
-
A cornerstone of materials science is that material properties are determined by their microstructure. While the community has already developed a wide variety of approaches to describe microstructure, most of these are tailored to specific material systems or classes. This work proposes a way to quantitatively measure the similarity of microstructures based on the geometry of the grain boundary network, a feature which is fundamental to and characteristic of all polycrystalline materials. Specifically, a distance on all single-phase polycrystalline microstructures is proposed such that two microstructures that are close with regard to the distance have grain boundary networks that are statistically similar in all geometric respects below a user-specified length scale. Given a pair of micrographs, the distance is approximated by sampling windows from the micrographs, defining a distance between pairs of windows, and finding a window matching that minimizes the sum of the pairwise window distances. The approach is used to compare a variety of synthetic microstructures and to develop a procedure to query a proof-of-concept database suitable for general single-phase polycrystalline microstructures.more » « less
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Conference Paper:
- https://doi.org/10.1109/IEEECONF51394.2020.9443454
