Search for: All records

Most research in technologies for the Deaf community have focused on translation using either video or wearable devices. Sensor-augmented gloves have been reported to yield higher gesture recognition rates than camera-based systems; however, they cannot capture information expressed through head and body movement. Gloves are also intrusive and inhibit users in their pursuit of normal daily life, while cameras can raise concerns over privacy and are ineffective in the dark. In contrast, RF sensors are non-contact, non-invasive and do not reveal private information even if hacked. Although RF sensors are unable to measure facial expressions or hand shapes, which would be required for complete translation, this paper aims to exploit near real-time ASL recognition using RF sensors for the design of smart Deaf spaces. In this way, we hope to enable the Deaf community to benefit from advances in technologies that could generate tangible improvements in their quality of life. More specifically, this paper investigates near real-time implementation of machine learning and deep learning architectures for the purpose of sequential ASL signing recognition. We utilize a 60 GHz RF sensor which transmits a frequency modulation continuous wave (FMWC waveform). RF sensors can acquire a unique source of information that is inaccessible to optical or wearable devices: namely, a visual representation of the kinematic patterns of motion via the micro-Doppler signature. Micro-Doppler refers to frequency modulations that appear about the central Doppler shift, which are caused by rotational or vibrational motions that deviate from principle translational motion. In prior work, we showed that fractal complexity computed from RF data could be used to discriminate signing from daily activities and that RF data could reveal linguistic properties, such as coarticulation. We have also shown that machine learning can be used to discriminate with 99% accuracy the signing of native Deaf ASL users from that of copysigning (or imitation signing) by hearing individuals. Therefore, imitation signing data is not effective for directly training deep models. But, adversarial learning can be used to transform imitation signing to resemble native signing, or, alternatively, physics-aware generative models can be used to synthesize ASL micro-Doppler signatures for training deep neural networks. With such approaches, we have achieved over 90% recognition accuracy of 20 ASL signs. In natural environments, however, near real-time implementations of classification algorithms are required, as well as an ability to process data streams in a continuous and sequential fashion. In this work, we focus on extensions of our prior work towards this aim, and compare the efficacy of various approaches for embedding deep neural networks (DNNs) on platforms such as a Raspberry Pi or Jetson board. We examine methods for optimizing the size and computational complexity of DNNs for embedded micro-Doppler analysis, methods for network compression, and their resulting sequential ASL recognition performance. 

Abstract Background Lagoons are common along coastlines worldwide and are important for biogeochemical element cycling, coastal biodiversity, coastal erosion protection and blue carbon sequestration. These ecosystems are frequently disturbed by weather, tides, and human activities. Here, we investigated a shallow lagoon in New England. The brackish ecosystem releases hydrogen sulfide particularly upon physical disturbance, causing blooms of anoxygenic sulfur-oxidizing phototrophs. To study the habitat, microbial community structure, assembly and function we carried out in situ experiments investigating the bloom dynamics over time. Results Phototrophic microbial mats and permanently or seasonally stratified water columns commonly contain multiple phototrophic lineages that coexist based on their light, oxygen and nutrient preferences. We describe similar coexistence patterns and ecological niches in estuarine planktonic blooms of phototrophs. The water column showed steep gradients of oxygen, pH, sulfate, sulfide, and salinity. The upper part of the bloom was dominated by aerobic phototrophic Cyanobacteria , the middle and lower parts by anoxygenic purple sulfur bacteria ( Chromatiales ) and green sulfur bacteria ( Chlorobiales ), respectively. We show stable coexistence of phototrophic lineages from five bacterial phyla and present metagenome-assembled genomes (MAGs) of two uncultured Chlorobaculum and Prosthecochloris species. In addition to genes involved in sulfur oxidation and photopigment biosynthesis the MAGs contained complete operons encoding for terminal oxidases. The metagenomes also contained numerous contigs affiliating with Microviridae viruses, potentially affecting Chlorobi . Our data suggest a short sulfur cycle within the bloom in which elemental sulfur produced by sulfide-oxidizing phototrophs is most likely reduced back to sulfide by Desulfuromonas sp . Conclusions The release of sulfide creates a habitat selecting for anoxygenic sulfur-oxidizing phototrophs, which in turn create a niche for sulfur reducers. Strong syntrophism between these guilds apparently drives a short sulfur cycle that may explain the rapid development of the bloom. The fast growth and high biomass yield of Chlorobi -affiliated organisms implies that the studied lineages of green sulfur bacteria can thrive in hypoxic habitats. This oxygen tolerance is corroborated by oxidases found in MAGs of uncultured Chlorobi . The findings improve our understanding of the ecology and ecophysiology of anoxygenic phototrophs and their impact on the coupled biogeochemical cycles of sulfur and carbon.