More Like this

1. Sketching Vocabulary for Crowd Motion

   Mathew, Tharindu; Benes, Bedrich; Aliaga, Daniel (September 2022, ACM SIGGRAPH / Eurographics Symposium on Computer Animation)

   This paper proposes and evaluates a sketching language to author crowd motion. It focuses on the path, speed, thickness, and density parameters of crowd motion. A sketch-based vocabulary is proposed for each parameter and evaluated in a user study against complex crowd scenes. A sketch recognition pipeline converts the sketches into a crowd simulation. The user study results show that 1) participants at various skill levels and can draw accurate crowd motion through sketching, 2) certain sketch styles lead to a more accurate representation of crowd parameters, and 3) sketching allows to produce complex crowd motions in a few seconds. The results show that some styles although accurate actually are less preferred over less accurate ones. 

   more » « less
2. Open Vocabulary Semantic Scene Sketch Understanding

   Bourouis, Ahmed; Fan, Judith E; Gryaditskaya, Yulia (June 2024, Computer Vision Foundation)

   We study the underexplored but fundamental vision problem of machine understanding of abstract freehand scene sketches We introduce a sketch encoder that results in semantically- aware feature space, which we evaluate by testing its performance on a semantic sketch seg- mentation task. To train our model we rely only on the availability of bitmap sketches with their brief captions and do not require any pixel-level annotations. To obtain generalization to a large set of sketches and categories, we build on a vision transformer encoder pretrained with the CLIP model. We freeze the text encoder and perform visual-prompt tuning of the visual encoder branch while introducing a set of critical modifications. Firstly, we augment the classical key-query (k-q) self-attention blocks with value-value (v-v) self-attention blocks. Central to our model is a two-level hierarchical network design that enables efficient semantic disentanglement: The first level ensures holistic scene sketch encoding, and the second level focuses on individual categories. We, then, in the second level of the hierarchy, introduce a cross-attention between textual and visual branches. Our method outperforms zero-shot CLIP pixel accuracy of segmentation results by 37 points, reaching an accuracy of 85.5% on the FS-COCO sketch dataset. Finally, we conduct a user study that allows us to identify further improvements needed over our method to reconcile machine and human understanding of scene sketches. 

   more » « less
3. Interoperable vocabulary for marine microbial flow cytometry

   https://doi.org/10.3389/fmars.2022.975877  

   Thyssen, Melilotus; Grégori, Gérald; Créach, Véronique; Lahbib, Soumaya; Dugenne, Mathilde; Aardema, Hedy M.; Artigas, Luis-Felipe; Huang, Bangqin; Barani, Aude; Beaugeard, Laureen; et al (November 2022, Frontiers in Marine Science)

   The recent development of biological sensors has extended marine plankton studies from conducting laboratory bench work to in vivo and real-time observations. Flow cytometry (FCM) has shed new light on marine microorganisms since the 1980s through its single-cell approach and robust detection of the smallest cells. FCM records valuable optical properties of light scattering and fluorescence from cells passing in a single file in front of a narrow-collimated light source, recording tens of thousands of cells within a few minutes. Depending on the instrument settings, the sampling strategy, and the automation level, it resolves the spatial and temporal distribution of microbial marine prokaryotes and eukaryotes. Cells are usually classified and grouped on cytograms by experts and are still lacking standards, reducing data sharing capacities. Therefore, the need to make FCM data sets FAIR (Findability, Accessibility, Interoperability, and Reusability of digital assets) is becoming critical. In this paper, we present a consensus vocabulary for the 13 most common marine microbial groups observed with FCM using blue and red-light excitation. The authors designed a common layout on two-dimensional log-transformed cytograms reinforced by a decision tree that facilitates the characterization of groups. The proposed vocabulary aims at standardising data analysis and definitions, to promote harmonisation and comparison of data between users and instruments. This represents a much-needed step towards FAIRification of flow cytometric data collected in various marine environments. 

   more » « less
4. AccEar: Accelerometer Acoustic Eavesdropping with Unconstrained Vocabulary

   https://doi.org/10.1109/SP46214.2022.9833716  

   Hu, Pengfei; Zhuang, Hui; Santhalingam, Panneer Selvam; Spolaor, Riccardo; Pathak, Parth; Zhang, Guoming; Cheng, Xiuzhen (May 2022, 2022 IEEE Symposium on Security and Privacy (SP))
5. MasQCLIP for Open-Vocabulary Universal Image Segmentation

   Xu, Xin; Xiong, Tianyi; Ding, Zheng; Tu, Zhuowen (October 2023, IEEE)