Over the last decade, facial landmark tracking and 3D reconstruction have gained considerable attention due to their numerous applications such as human-computer interactions, facial expression analysis, and emotion recognition, etc. Traditional approaches require users to be confined to a particular location and face a camera under constrained recording conditions (e.g., without occlusions and under good lighting conditions). This highly restricted setting prevents them from being deployed in many application scenarios involving human motions. In this paper, we propose the first single-earpiece lightweight biosensing system, BioFace-3D, that can unobtrusively, continuously, and reliably sense the entire facial movements, track 2D facial landmarks, and further render 3D facial animations. Our single-earpiece biosensing system takes advantage of the cross-modal transfer learning model to transfer the knowledge embodied in a high-grade visual facial landmark detection model to the low-grade biosignal domain. After training, our BioFace-3D can directly perform continuous 3D facial reconstruction from the biosignals, without any visual input. Without requiring a camera positioned in front of the user, this paradigm shift from visual sensing to biosensing would introduce new opportunities in many emerging mobile and IoT applications. Extensive experiments involving 16 participants under various settings demonstrate that BioFace-3D can accurately track 53 major facial landmarks with only 1.85 mm average error and 3.38\% normalized mean error, which is comparable with most state-of-the-art camera-based solutions. The rendered 3D facial animations, which are in consistency with the real human facial movements, also validate the system's capability in continuous 3D facial reconstruction.
more »
« less
Recent Advances in Digital Biosensing Technology
Digital biosensing assays demonstrate remarkable advantages over conventional biosensing systems because of their ability to achieve single-molecule detection and absolute quantification. Unlike traditional low-abundance biomarking screening, digital-based biosensing systems reduce sample volumes significantly to the fL-nL level, which vastly reduces overall reagent consumption, improves reaction time and throughput, and enables high sensitivity and single target detection. This review presents the current technology for compartmentalizing reactions and their applications in detecting proteins and nucleic acids. We also analyze existing challenges and future opportunities associated with digital biosensing and research opportunities for developing integrated digital biosensing systems.
more »
« less
- Award ID(s):
- 2144216
- NSF-PAR ID:
- 10407380
- Date Published:
- Journal Name:
- Biosensors
- Volume:
- 12
- Issue:
- 9
- ISSN:
- 2079-6374
- Page Range / eLocation ID:
- 673
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
more » « less
Abstract Wearable sensors that can conveniently detect cytokine levels in human biofluids are essential for assisting hospitals to maximize the benefits of anti‐inflammatory therapies and avoid cytokine storms. Measurement of cytokine levels in biofluids still remains challenging for existing sensors due to high interference from the background. Here, this challenge is overcome through developing a flexible and regenerative aptameric field‐effect transistor biosensor, consisting of a graphene–Nafion composite film, for detecting cytokine storm biomarkers in undiluted human biofluids. The composite film enables the minimization of nonspecific adsorption and empowers the renewability to the biosensor. With these capabilities, the device is capable of consistently and sensitively monitoring cytokines (e.g., IFN‐γ, an inflammatory and cancer biomarker) in undiluted human sweat with a detection range from 0.015 to 250 n
m and limit of detection down to 740 fm . The biosensor is also shown to incur no visible mechanical damage and maintain a consistent sensing response throughout the regenerative (up to 80 cycles) and crumpling (up to 100 cycles) tests. Experimental results demonstrate that the biosensor is expected to offer opportunities for developing wearable biosensing systems for distinguishing acute infectious disease patients and monitoring of patients’ health conditions in daily life. -
Colorimetric sensors offer the prospect for on-demand sensing diagnostics in simple and low-cost form factors, enabling rapid spatiotemporal inspection by digital cameras or the naked eye. However, realizing strong dynamic color variations in response to small changes in sample properties has remained a considerable challenge, which is often pursued through the use of highly responsive materials under broadband illumination. In this work, we demonstrate a general colorimetric sensing technique that overcomes the performance limitations of existing chromatic and luminance-based sensing techniques. Our approach combines structural color optical filters as sensing elements alongside a multichromatic laser illuminant. We experimentally demonstrate our approach in the context of label-free biosensing and achieve ultrasensitive and perceptually enhanced chromatic color changes in response to refractive index changes and small molecule surface attachment. Using structurally enabled chromaticity variations, the human eye is able to resolve ∼0.1-nm spectral shifts with low-quality factor (e.g., Q ∼ 15) structural filters. This enables spatially resolved biosensing in large area (approximately centimeters squared) lithography-free sensing films with a naked eye limit of detection of ∼3 pg/mm 2 , lower than industry standard sensors based on surface plasmon resonance that require spectral or angular interrogation. This work highlights the key roles played by both the choice of illuminant and design of structural color filter, and it offers a promising pathway for colorimetric devices to meet the strong demand for high-performance, rapid, and portable (or point-of-care) diagnostic sensors in applications spanning from biomedicine to environmental/structural monitoring.more » « less
-
more » « less
Abstract 19F magnetic resonance (MR) based detection coupled with well‐designed inorganic systems shows promise in biological investigations. Two proof‐of‐concept inorganic probes that exploit a novel mechanism for19F MR sensing based on converting from low‐spin (
S =0) to high‐spin (S =1) Ni2+are reported. Activation of diamagneticNiL1 andNiL2 by light or β‐galactosidase, respectively, converts them into paramagneticNiL0 , which displays a single19F NMR peak shifted by >35 ppm with accelerated relaxation rates. This spin‐state switch is effective for sensing light or enzyme expression in live cells using19F MR spectroscopy and imaging that differentiate signals based on chemical shift and relaxation times. This general inorganic scaffold has potential for developing agents that can sense analytes ranging from ions to enzymes, opening up diverse possibilities for19F MR based biosensing. -
more » « less
Abstract 19F magnetic resonance (MR) based detection coupled with well‐designed inorganic systems shows promise in biological investigations. Two proof‐of‐concept inorganic probes that exploit a novel mechanism for19F MR sensing based on converting from low‐spin (
S =0) to high‐spin (S =1) Ni2+are reported. Activation of diamagneticNiL1 andNiL2 by light or β‐galactosidase, respectively, converts them into paramagneticNiL0 , which displays a single19F NMR peak shifted by >35 ppm with accelerated relaxation rates. This spin‐state switch is effective for sensing light or enzyme expression in live cells using19F MR spectroscopy and imaging that differentiate signals based on chemical shift and relaxation times. This general inorganic scaffold has potential for developing agents that can sense analytes ranging from ions to enzymes, opening up diverse possibilities for19F MR based biosensing.
- Free Publicly Accessible Full Text
-
Accepted Manuscript1.0
- Journal Article:
- https://doi.org/10.3390/bios12090673
