Search for: All records

Running with a consistent cadence (number of steps per minute) is important for runners to help reduce risk of injury, improve running form, and enhance overall bio-mechanical efficiency. We introduce CaNRun, a non-contact and acoustic-based system that uses sound captured from a mobile device placed on a treadmill to predict and report running cadence. CaNRun obviates the need for runners to utilize wearable devices or carry a mobile device on their body while running on a treadmill. CaNRun leverages a long short-term memory (LSTM) network to extract steps observed from the microphone to robustly estimate cadence. Through an 8-person study, we demonstrate that CaNRun achieves cadence detection accuracy without calibration for individual users, which is comparable to the accuracy of the Apple Watch despite being non-contact. 

The stethoscope is one of the most important diagnostic tools used by healthcare professionals, through a process called auscultation, to screen patients for abnormalities of the heart and lungs. While there are digital stethoscopes on the market which ease this process, it still takes years of training to properly use these devices to listen for abnormal sounds within the body. We present ARSteth, an intelligent stethoscope platform that improves the accessibility of stethoscopes for the general population, allowing anyone to perform auscultation in the comfort of their own homes. Our platform utilizes a combination of augmented reality (AR), acoustic intelligence, and human-machine interaction to dynamically guide users on where to place the stethoscope on different parts of the body (auscultation points), through visual and audio cues. Through user studies, we show that ARSteth, on average, can guide users within 13.2 mm from optimal auscultation points marked by licensed physicians in 13.09 seconds for each auscultation point. By guiding users towards more effective auscultation points, make preventative health screening more accessible and effective for everyone we are able to achieve higher confidence on classifying heart murmurs. 

Breath monitoring is important for monitoring illnesses, such as sleep apnea, for people of all ages. One cause of concern for parents is sudden infant death syndrome (SIDS), where an infant suddenly passes away during sleep, usually due to complications in breathing. There are a variety of works and products on the market for monitoring breathing, especially for children and infants. Many of these are wearables that require you to attach an accessory onto the child or person, which can be uncomfortable. Other solutions utilize a camera, which can be privacy-intrusive and function poorly during the night, when lighting is poor. In this work, we introduce BuMA, an audio-based, non-intrusive, and contactless, breathing monitoring system. BuMA utilizes a microphone array, beamforming, and audio filtering to enhance the sounds of breathing by filtering out several common noises in or near home environments, such as construction, speech, and music, that could make detection difficult. We show that BuMA improves breathing detection accuracy by up to 12%, within 30cm from a person, over existing audio filtering algorithms or platforms that do not leverage filtering. 

Audio is valuable in many mobile, embedded, and cyber-physical systems. We propose AvA, an acoustic adaptive filtering architecture, configurable to a wide range of applications and systems. By incorporating AvA into their own systems, developers can select which sounds to enhance or filter out depending on their application needs. AvA accomplishes this by using a novel adaptive beamforming algorithm called content-informed adaptive beam-forming (CIBF), that directly uses detectors and sound models that developers have created for their own applications to enhance or filter out sounds. CIBF uses a novel three step approach to prop-agate gradients from a wide range of different model types and signal feature representations to learn filter coefficients. We apply AvA to four scenarios and demonstrate that AvA enhances their respective performances by up to 11.1%. We also integrate AvA into two different mobile/embedded platforms with widely different resource constraints and target sounds/noises to show the boosts in performance and robustness these applications can see using AvA. 

Vehicle accidents are one of the greatest cause of death and injury in urban areas for pedestrians, workers, and police alike. In this work, we present CSafe, a low power audio-wearable platform that detects, localizes, and provides alerts about oncoming vehicles to improve construction worker safety. Construction worker safety is a much more challenging problem than general urban or pedestrian safety in that the sound of construction tools can be up to orders of magnitude greater than that of vehicles, making vehicle detection and localization exceptionally difficult. To overcome these challenges, we develop a novel sound source separation algorithm, called Probabilistic Template Matching (PTM), as well as a novel noise filtering architecture to remove loud construction noises from our observed signals. We show that our architecture can improve vehicle detection by up to 12% over other state-of-art source separation algorithms. We integrate PTM and our noise filtering architecture into CSafe and show through a series of real-world experiments that CSafe can achieve up to an 82% vehicle detection rate and a 6.90° mean localization error in acoustically noisy construction site scenarios, which is 16% higher and almost 30° lower than the state-of-art audio wearable safety works. 

Sound detection and classification are critical in many acoustic-based applications. Existing works generally focus on discovering new features and classifiers to improve detection. However, in many scenarios the presence of other sounds may hinder the performance of these sound classifiers. In this work, we take a sound filtering and enhancement approach to improve sound detection for mobile and embedded applications, regardless of the type of detector used. 

Smartphones and mobile applications have become an integral part of our daily lives. This is reflected by the increase in mobile devices, applications, and revenue generated each year. However, this growth is being met with an increasing concern for user privacy, and there have been many incidents of privacy and data breaches related to smartphones and mobile applications in recent years. In this work, we focus on improving privacy for audio-based mobile systems. These applications will generally listen to all sounds in the environment and may record privacy-sensitive signals, such as speech, that may not be needed for the application. We present PAMS, a software development package for mobile applications. PAMS integrates a novel sound source filtering algorithm called Probabilistic Template Matching to generate a set of privacy-enhancing filters that remove extraneous sounds using learned statistical "templates" of these sounds. We demonstrate the effectiveness of PAMS by integrating it into a sleep monitoring system, with the intent to remove extraneous speech from breathing, snoring, and other sleep sounds that the system is monitoring. By comparing our PAMS enhanced sleep monitoring system with existing mobile systems, we show that PAMS can reduce speech intelligibility by up to 74.3% while maintaining similar performance in detecting sleeping sounds.