More Like this

1. Cross-Modal Center Loss for 3D Cross-Modal Retrieval

   Jing, L; Vahdani, E; Tan, J; Tian, Y. (June 2021, IEEE Conference on Computer Vision and Pattern Recognition (CVPR))

   null (Ed.)

   Cross-modal retrieval aims to learn discriminative and modal-invariant features for data from different modalities. Unlike the existing methods which usually learn from the features extracted by offline networks, in this paper, we pro- pose an approach to jointly train the components of cross- modal retrieval framework with metadata, and enable the network to find optimal features. The proposed end-to-end framework is updated with three loss functions: 1) a novel cross-modal center loss to eliminate cross-modal discrepancy, 2) cross-entropy loss to maximize inter-class variations, and 3) mean-square-error loss to reduce modality variations. In particular, our proposed cross-modal center loss minimizes the distances of features from objects belonging to the same class across all modalities. Extensive experiments have been conducted on the retrieval tasks across multi-modalities including 2D image, 3D point cloud and mesh data. The proposed framework significantly outperforms the state-of-the-art methods for both cross-modal and in-domain retrieval for 3D objects on the ModelNet10 and ModelNet40 datasets. 

   more » « less
2. SUMMIT: Source-Free Adaptation of Uni-Modal Models to Multi-Modal Targets

   Cody Simons, Dripta S. (October 2023, IEEE/CVF Int. Conf. on Computer Vision)
3. Modal assertions for actor correctness

   https://doi.org/10.1145/3358499.3361221  

   Gordon, Colin S. (October 2019, Proceedings of the 9th ACM SIGPLAN International Workshop on Programming Based on Actors, Agents, and Decentralized Control)

   The actor model is a well-established way to approach to modularly designing and implementing concurrent and/or distributed systems, seeing increasing adoption in industry. But deductive verification tailored to actor programs remains underexplored; general concurrent logics could be used, but the logics are complex and full of features to reason about behaviors the actor model strives to avoid. We explore a relatively lightweight approach of extending a system for proving sequential program correctness with means to prove safety properties of actor programs (currently, assuming no faults). We borrow ideas from hybrid logic, a modal logic for stating assertions are true at a particular point in a model (in this case, a particular actor’s local state). To make such assertions useful, we stabilize them using rely-guarantee-style reasoning over local actor states, and only permit sending stable versions of these assertions to other actors. By carefully restricting the formation of assertions that a proposition is true at a certain actor, we avoid the need for actors to handle each others’ rely-guarantee relations explicitly. Finally, we argue that the approach requires only modest adjustments beyond applying traditional sequential techniques to actors with immutable messages, by implementing most of the logic as a Dafny library. 

   more » « less
4. Modal analysis in curvilinear coordinates

   Parthasarathy, A.; Saxton-Fox, T. (August 2021, International Conference on Theoretical and Applied Mechanics Proceedings)

   Modal analysis techniques have proven useful in understanding and modeling turbulent phenomena. However, these techniques are more efficient in parallel flows where Fourier transforms can be taken along homogeneous directions. We suggest that quasi-1D methods can be applied to mildly non-canonical flows by using a curvilinear coordinate system. For a given base flow, we identify a curvilinear coordinate system that allows the Fourier-transformed equations of motion to be simplified into a quasi-1D system that can be efficiently analyzed. 

   more » « less
5. Multi-Modal Word Synset Induction

   Tomason, J.; Mooney, R.J. (August 2017, Proceedings of the 26th International Joint Conference on Artificial Intelligence (IJCAI-17))

   A word in natural language can be polysemous, having multiple meanings, as well as synonymous, meaning the same thing as other words. Word sense induction attempts to find the senses of pol- ysemous words. Synonymy detection attempts to find when two words are interchangeable. We com- bine these tasks, first inducing word senses and then detecting similar senses to form word-sense synonym sets (synsets) in an unsupervised fashion. Given pairs of images and text with noun phrase labels, we perform synset induction to produce col- lections of underlying concepts described by one or more noun phrases. We find that considering multi- modal features from both visual and textual context yields better induced synsets than using either con- text alone. Human evaluations show that our unsu- pervised, multi-modally induced synsets are com- parable in quality to annotation-assisted ImageNet synsets, achieving about 84% of ImageNet synsets’ approval. 

   more » « less