Search for: All records

Freshwater mussels are essential parts of our ecosystems to reduce water pollution. As natural bio-filters, they deactivate pollutants such as heavy metals, providing a sustainable method for water decontamination. This project will enable the use of Artificial Intelligence (AI) to monitor mussel behavior, particularly their gaping activity, to use them as bio-indicators for early detection of water contamination. In this paper, we employ advanced 3D reconstruction techniques to create detailed models of mussels to improve the accuracy of AI-based analysis. Specifically, we use a state-of-the-art 3D reconstruction tool, Neural Radiance Fields (NeRF), to create 3D models of mussel valve configurations and behavioral patterns. NeRF enables 3D reconstruction of scenes and objects from a sparse set of 2D images. To capture these images, we developed a data collection system capable of imaging mussels from multiple viewpoints. The system featured a turntable made of foam board with markers around the edges and a designated space in the center for mounting the mussels. The turntable was attached to a servo motor controlled by an ESP32 microcontroller. It rotated in a few degree increments, with the ESP32 camera capturing an image at each step. The images, along with degree information and timestamps, are stored on a Secure Digital (SD) memory card. Several components, such as the camera holder and turntable base, are 3D printed. These images are used to train a NeRF model using the Python-based Nerfstudio framework, and the resulting 3D models were viewed via the Nerfstudio API. The setup was designed to be user-friendly, making it easy for educational outreach engagements and to involve secondary education by replicating and operating 3D reconstructions of their chosen objects. We validated the accessibility and the impact of this platform in a STEM education summer program. A team of high school students from the Juntos Summer Academy at NC State University worked on this platform, gaining hands-on experience in embedded hardware development, basic machine learning principles, and 3D reconstruction from 2D images. We also report on their feedback on the activity. 

Variational autoencoders (VAEs) optimize an objective that comprises a reconstruction loss (the distortion) and a KL term (the rate). The rate is an upper bound on the mutual information, which is often interpreted as a regularizer that controls the degree of compression. We here examine whether inclusion of the rate term also improves generalization. We perform rate-distortion analyses in which we control the strength of the rate term, the network capacity, and the difficulty of the generalization problem. Lowering the strength of the rate term paradoxically improves generalization in most settings, and reducing the mutual information typically leads to underfitting. Moreover, we show that generalization performance continues to improve even after the mutual information saturates, indicating that the gap on the bound (i.e. the KL divergence relative to the inference marginal) affects generalization. This suggests that the standard spherical Gaussian prior is not an inductive bias that typically improves generalization, prompting further work to understand what choices of priors improve generalization in VAEs.