More Like this

1. Semi-supervised Convolutional Triplet Neural Networks for Assessing Paper Texture Similarity

   https://doi.org/10.1109/IEEECONF51394.2020.9443454  

   Lackey, Leah; Grootveld, Arick; Klein, Andrew G. (November 2020, 2020 54th Asilomar Conference on Signals, Systems, and Computers)

   In the context of papers used in the graphic arts, including silver gelatin, inkjet, and wove papers, prior work has studied measures of texture similarity for purposes of classifying such papers. The majority of prior work has been based on classical image processing approaches such as Fourier, wavelet, and fractal analysis. In this work, recent advances in deep learning are used to develop a texture similarity approach for measuring paper texture similarity. Since the available datasets generally lack labels, the convolutional neural network is trained using triplet loss to minimize the feature distance of tiles from the same image while simultaneously maximizing the feature distance of tiles drawn from different images. The approach is tested on three paper texture image databases considered in prior works and the results suggest the proposed approach achieves state-of-the-art performance. 

   more » « less
2. TPDNet: Texture-Guided Phase-to-DEPTH Networks to Repair Shadow-Induced Errors for Fringe Projection Profilometry

   https://doi.org/10.3390/photonics10030246  

   Li, Jiaqiong; Li, Beiwen (March 2023, Photonics)

   This paper proposes a phase-to-depth deep learning model to repair shadow-induced errors for fringe projection profilometry (FPP). The model comprises two hourglass branches that extract information from texture images and phase maps and fuses the information from the two branches by concatenation and weights. The input of the proposed model contains texture images, masks, and unwrapped phase maps, and the ground truth is the depth map from CAD models. A loss function was chosen to consider image details and structural similarity. The training data contain 1200 samples in the verified virtual FPP system. After training, we conduct experiments on the virtual and real-world scanning data, and the results support the model’s effectiveness. The mean absolute error and the root mean squared error are 1.0279 mm and 1.1898 mm on the validation dataset. In addition, we analyze the influence of ambient light intensity on the model’s performance. Low ambient light limits the model’s performance as the model cannot extract valid information from the completely dark shadow regions in texture images. The contribution of each branch network is also investigated. Features from the texture-dominant branch are leveraged as guidance to remedy shadow-induced errors. Information from the phase-dominant branch network makes accurate predictions for the whole object. Our model provides a good reference for repairing shadow-induced errors in the FPP system. 

   more » « less
3. Unsupervised Feature Learning for Point Cloud by Contrasting and Clustering with Graph Convolutional Neural Network

   Zhang, Ling; Zhu, Zhigang (June 2019, CVPR Workshop on 3D Scene Understanding for Vision, Graphics, and Robotics)

   To alleviate the cost of collecting and annotating large- scale point cloud datasets for 3D scene understanding tasks, we propose an unsupervised learning approach to learn features from unlabeled point cloud ”3D object” dataset by using part contrasting and object clustering with deep graph neural networks (GNNs). In the contrast learn- ing step, all the samples in the 3D object dataset are cut into two parts and put into a ”part” dataset. Then a contrast learning GNN (ContrastNet) is trained to verify whether two randomly sampled parts from the part dataset belong to the same object. In the cluster learning step, the trained ContrastNet is applied to all the samples in the original 3D object dataset to extract features, which are used to group the samples into clusters. Then another GNN for cluster- ing learning (ClusterNet) is trained to predict the cluster IDs of all the training samples. The contrasting learning forces the ContrastNet to learn high-level semantic features of objects but probably ignores low-level features, while the ClusterNet improves the quality of learned features by be- ing trained to discover objects that belong to the same se- mantic categories by using cluster IDs. We have conducted extensive experiments to evaluate the proposed framework on point cloud classification tasks. The proposed unsupervised learning approach obtained comparable performance to the state-of-the-art unsupervised learning methods that used much more complicated network structures. The code and an extended version of this work is publicly available via: https://github.com/lingzhang1/ContrastNet 

   more » « less
4. UNDERSTANDING FEATURE DISCOVERY IN WEBSITE FINGERPRINTING ATTACKS

   https://doi.org/10.1109/WNYIPW.2018.8576379  

   Mathews, Nate; Sirinam, Payap; Wright, Matthew (October 2018, Proceedings of the 2018 IEEE Western New York Image and Signal Processing Workshop)

   The Tor anonymity system is vulnerable to website fingerprinting attacks that can reveal users Internet browsing behavior. The state-of-the-art website fingerprinting attacks use convolutional neural networks to automatically extract features from packet traces. One such attack undermines an efficient fingerprinting defense previously considered a candidate for implementation in Tor. In this work, we study the use of neural network attribution techniques to visualize activity in the attack's model. These visualizations, essentially heatmaps of the network, can be used to identify regions of particular sensitivity and provide insight into the features that the model has learned. We then examine how these heatmaps may be used to create a new website fingerprinting defense that applies random padding to the website trace with an emphasis towards highly fingerprintable regions. This defense reduces the attacker's accuracy from 98% to below 70% with a packet overhead of approximately 80%. 

   more » « less
5. A Picture or a Thousand Words: Design Description Crafting to Replicate Human Similarity Judgments in Large Language Models

   https://doi.org/10.1115/DETC2024-143634  

   Keeler, Matthew; Fuge, Mark; Peng, Aoran; Miller, Scarlett (August 2024, American Society of Mechanical Engineers)

   Abstract Well-studied techniques that enhance diversity in early design concept generation require effective metrics for evaluating human-perceived similarity between ideas. Recent work suggests collecting triplet comparisons between designs directly from human raters and using those triplets to form an embedding where similarity is expressed as a Euclidean distance. While effective at modeling human-perceived similarity judgments, these methods are expensive and require a large number of triplets to be hand-labeled. However, what if there were a way to use AI to replicate the human similarity judgments captured in triplet embedding methods? In this paper, we explore the potential for pretrained Large Language Models (LLMs) to be used in this context. Using a dataset of crowdsourced text descriptions written about engineering design sketches, we generate LLM embeddings and compare them to an embedding created from human-provided triplets of those same sketches. From these embeddings, we can use Euclidean distances to describe areas where human perception and LLM perception disagree regarding design similarity. We then implement this same procedure but with descriptions written from a template that attempts to isolate a particular modality of a design (i.e., functions, behaviors, structures). By comparing the templated description embeddings to both the triplet-generated and pre-template LLM embeddings, we identify ways of describing designs such that LLM and human similarity perception better agree. We use these results to better understand how humans and LLMs interpret similarity in engineering designs. 

   more » « less