Search for: All records

Internet-of-Things (IoT) approaches are continually introducing new sensors into the fields of agriculture and animal welfare. The application of multi-sensor data fusion to these domains remains a complex and open-ended challenge that defies straightforward optimization, often requiring iterative testing and refinement. To respond to this need, we have created a new open-source framework as well as a corresponding Python tool which we call the “Data Fusion Explorer (DFE)”. We demonstrated and evaluated the effectiveness of our proposed framework using four early-stage datasets from diverse disciplines, including animal/environmental tracking, agrarian monitoring, and food quality assessment. This included data across multiple common formats including single, array, and image data, as well as classification or regression and temporal or spatial distributions. We compared various pipeline schemes, such as low-level against mid-level fusion, or the placement of dimensional reduction. Based on their space and time complexities, we then highlighted how these pipelines may be used for different purposes depending on the given problem. As an example, we observed that early feature extraction reduced time and space complexity in agrarian data. Additionally, independent component analysis outperformed principal component analysis slightly in a sweet potato imaging dataset. Lastly, we benchmarked the DFE tool with respect to the Vanilla Python3 packages using our four datasets’ pipelines and observed a significant reduction, usually more than 50%, in coding requirements for users in almost every dataset, suggesting the usefulness of this package for interdisciplinary researchers in the field. 

Freshwater mussels are essential parts of our ecosystems to reduce water pollution. As natural bio-filters, they deactivate pollutants such as heavy metals, providing a sustainable method for water decontamination. This project will enable the use of Artificial Intelligence (AI) to monitor mussel behavior, particularly their gaping activity, to use them as bio-indicators for early detection of water contamination. In this paper, we employ advanced 3D reconstruction techniques to create detailed models of mussels to improve the accuracy of AI-based analysis. Specifically, we use a state-of-the-art 3D reconstruction tool, Neural Radiance Fields (NeRF), to create 3D models of mussel valve configurations and behavioral patterns. NeRF enables 3D reconstruction of scenes and objects from a sparse set of 2D images. To capture these images, we developed a data collection system capable of imaging mussels from multiple viewpoints. The system featured a turntable made of foam board with markers around the edges and a designated space in the center for mounting the mussels. The turntable was attached to a servo motor controlled by an ESP32 microcontroller. It rotated in a few degree increments, with the ESP32 camera capturing an image at each step. The images, along with degree information and timestamps, are stored on a Secure Digital (SD) memory card. Several components, such as the camera holder and turntable base, are 3D printed. These images are used to train a NeRF model using the Python-based Nerfstudio framework, and the resulting 3D models were viewed via the Nerfstudio API. The setup was designed to be user-friendly, making it easy for educational outreach engagements and to involve secondary education by replicating and operating 3D reconstructions of their chosen objects. We validated the accessibility and the impact of this platform in a STEM education summer program. A team of high school students from the Juntos Summer Academy at NC State University worked on this platform, gaining hands-on experience in embedded hardware development, basic machine learning principles, and 3D reconstruction from 2D images. We also report on their feedback on the activity. 

Guide dogs play a crucial role in enhancing independence and mobility for people with visual impairment, offering invaluable assistance in navigating daily tasks and environments. However, the extensive training required for these dogs is costly, resulting in a limited availability that does not meet the high demand for such skilled working animals. Towards optimizing the training process and to better understand the challenges these guide dogs may be experiencing in the field, we have created a multi-sensor smart collar system. In this study, we developed and compared two supervised machine learning methods to analyze the data acquired from these sensors. We found that the Convolutional Long Short-Term Memory (Conv-LSTM) network worked much more efficiently on subsampled data and Kernel Principal Component Analysis (KPCA) on interpolated data. Each attained approximately 40% accuracy on a 10-state system. Not needing training, KPCA is a much faster method, but not as efficient with larger datasets. Among various sensors on the collar system, we observed that the inertial measurement units account for the vast majority of predictability, and that the addition of environmental acoustic sensing data slightly improved performance in most datasets. We also created a lexicon of data patterns using an unsupervised autoencoder. We present several regions of relatively higher density in the latent variable space that correspond to more common patterns and our attempt to visualize these patterns. In this preliminary effort, we found that several test states could be combined into larger superstates to simplify the testing procedures. Additionally, environmental sensor data did not carry much weight, as air conditioning units maintained the testing room at standard conditions. 

Canine-assisted interactions (CAIs) have been explored to offer therapeutic benefits to human participants in various contexts, from addressing cancer-related fatigue to treating post-traumatic stress disorder. Despite their widespread adoption, there are still unresolved questions regarding the outcomes for both humans and animals involved in these interactions. Previous attempts to address these questions have suffered from core methodological weaknesses, especially due to absence of tools for an efficient objective evaluation and lack of focus on the canine perspective. In this article, we present a first-of-its-kind system and study to collect simultaneous and continuous physiological data from both of the CAI interactants. Motivated by our extensive field reviews and stakeholder feedback, this comprehensive wearable system is composed of custom-designed and commercially available sensor devices. We performed a repeated-measures pilot study, to combine data collected via this system with a novel dyadic behavioral coding method and short- and long-term surveys. We evaluated these multimodal data streams independently, and we further correlated the psychological, physiological, and behavioral metrics to better elucidate the outcomes and dynamics of CAIs. Confirming previous field results, human electrodermal activity is the measure most strongly distinguished between the dyads’ non-interaction and interaction periods. Valence, arousal, and the positive affect of the human participant significantly increased during interaction with the canine participant. Also, we observed in our pilot study that (a) the canine heart rate was more dynamic than the human’s during interactions, (b) the surveys proved to be the best indicator of the subjects’ affective state, and (c) the behavior coding approaches best tracked the bond quality between the interacting dyads. Notably, we found that most of the interaction sessions were characterized by extended neutral periods with some positive and negative peaks, where the bonded pairs might display decreased behavioral synchrony. We also present three new representations of the internal and overall dynamics of CAIs for adoption by the broader field. Lastly, this paper discusses ongoing options for further dyadic analysis, interspecies emotion prediction, integration of contextually relevant environmental data, and standardization of human–animal interaction equipment and analytical approaches. Altogether, this work takes a significant step forward on a promising path to our better understanding of how CAIs improve well-being and how interspecies psychophysiological states can be appropriately measured. 

The study of plant electrophysiology offers promising techniques to track plant health and stress in vivo for both agricultural and environmental monitoring applications. Use of superficial electrodes on the plant body to record surface potentials may provide new phenotyping insights. Bacterial nanocellulose (BNC) is a flexible, optically translucent, and water-vapor-permeable material with low manufacturing costs, making it an ideal substrate for non-invasive and non-destructive plant electrodes. This work presents BNC electrodes with screen-printed carbon (graphite) ink-based conductive traces and pads. It investigates the potential of these electrodes for plant surface electrophysiology measurements in comparison to commercially available standard wet gel and needle electrodes. The electrochemically active surface area and impedance of the BNC electrodes varied based on the annealing temperature and time over the ranges of 50 °C to 90 °C and 5 to 60 min, respectively. The water vapor transfer rate and optical transmittance of the BNC substrate were measured to estimate the level of occlusion caused by these surface electrodes on the plant tissue. The total reduction in chlorophyll content under the electrodes was measured after the electrodes were placed on maize leaves for up to 300 h, showing that the BNC caused only a 16% reduction. Maize leaf transpiration was reduced by only 20% under the BNC electrodes after 72 h compared to a 60% reduction under wet gel electrodes in 48 h. On three different model plants, BNC–carbon ink surface electrodes and standard invasive needle electrodes were shown to have a comparable signal quality, with a correlation coefficient of >0.9, when measuring surface biopotentials induced by acute environmental stressors. These are strong indications of the superior performance of the BNC substrate with screen-printed graphite ink as an electrode material for plant surface biopotential recordings.