Search for: All records

Plants are geometrically and topologically complex objects, and methods and devices that produce plant point clouds often miss parts due to self occlusions, making further analysis, such as phenotypic trait extraction or 3D reconstruction, difficult. We introduce A-Occ-Plant, a novel method for point cloud completion. The first novelty of our algorithm is converting point clouds into a set of images, which are then completed using 2D amodal segmentation. The images are then converted into a complete point cloud by using view-consistent Gaussian splats. The second novelty is the use of a coarse-to-fine hierarchical Transformer with cross-scale attention. The completed soft masks are fused into a continuous 3D density field using Gaussian splatting, removing the need for external pose estimation or fixed-size inputs. We introduce a synthetic dataset using a procedural model and a real-world plant reconstruction benchmark with artificially generated occlusions. We further benchmark A-Occ-Plant against representative 3D point-cloud completion methods, demonstrate that it recovers downstream phenotypic traits (leaf count, leaf angle, plant height), and show that it generalizes to another crops (soybean). AOcc-Plant achieves a 264.8% improvement in LPIPS and an 8.3% gain in SSIM compared to the current state of the art, while using only 2.3% of the parameters and running 39.4× faster. We release our code at https://github.com/JaeLee18/PlantPhenomics_Occlusion. 

Fungal wood decay is a complex biophysical phenomenon that involves the degradation of a variety of structural wood components, ranging from lignin and carbohydrates to defensive chemical agents. All these substrates serve as varying resources with different material properties that determine the rate of fungal propagation and the structural integrity and color of decaying wood. We propose a novel approach to simulate the dynamic interactions between the biological and mechanical components of wood decay, including fungal colonization, chemical defense, and moisture-driven fracture. We propose a novel volumetric representation of trees that includes grain-aligned mesh generation, internal moisture dynamics, and tissue-specific health states. Furthermore, we model the anisotropic diffusion, consumption, and resulting material failure caused by white and brown rot fungi. This allows simulating and rendering 3D volumetric decaying trees that realistically capture key aspects of the process, such as the progression of cuboid fracture patterns, the hollowing of trunks, and the effects of environmental moisture on structural stability. 

To enhance learning and training, VR haptic devices must provide safe and physically accurate feedback that helps users build intuitions about force and motion. We present TrueForce, a method for rendering force feedback by capturing user input and reconfiguring a compliant mechanism in real time to align perceived with expected forces. We implemented TrueForce in a grounded Encountered-Type Haptic Display (ETHD) and demonstrated its versatility through three representative physics scenarios: pushing a stationary object, experiencing electrostatic repulsion, and bouncing a moving object. A user study with 29 participants showed that users adapted to new interaction profiles within 2–3 trials, and that rendered forces remained within perceptual thresholds except in the heaviest and most dynamic conditions. These results suggest that users can rapidly adapt to the system’s force-motion mappings, providing the consistent sensorimotor feedback necessary for building physical intuitions when interacting with moving virtual objects. 

Amodal segmentation is an image-based algorithm that aims to predict masks for both visible and occluded parts of objects. Existing methods typically rely on supervised learning with annotated amodal masks or synthetic data. The effectiveness of these methods relies heavily on the quality of the datasets. This dependence can unintentionally restrict their generalization capabilities due to insufficient diversity and size. Although existing zero-shot methods perform well on their reported datasets, their performance does not necessarily transfer to other datasets. We propose a tuning-free approach that re-purposes diffusion-based inpainting foundation models for amodal segmentation. Our approach is motivated by the “occlusion-free bias” of inpainting models, i.e., the inpainted objects tend to be complete and without occlusions. We reconstruct the occluded regions of an object via inpainting and then apply segmentation, all without additional training or fine-tuning. Experiments on five datasets, three previously unreported, demonstrate the generalizability of our approach. On average, our approach achieves 5.3% more accurate masks in mIoU compared to the publicly available state-of-the-art, pix2gestalt. 

We introduce, implement, and test VR BioTalk, a hands-free, immersive, voice-controlled visual analytics system for phenotypic data. Our system does not require any programming knowledge. Yet, it enables users to receive an interactive solution to complex tasks involving large datasets through simple verbal commands, such as “Show me all leaves smaller than the average and calculate their leaf area index.” We claim three main contributions: 1) preprocessing and feature extraction of point cloud data for interactive visual analytics, 2) development of a novel interface that converts user speech into commands, and 3) an immersive VR visualization that executes the commands and displays the results in VR. The speech recognition system’s precision has been validated on 416 spoken commands across 13 English accents, with an accuracy of around 99.7% for transcription and 94% for command recognition. The visualization averages 63 FPS, and the system’s response time is approximately 1.25 seconds. We tested VR BioTalk on several tasks that would otherwise require extensive programming knowledge. We tested our system with 9 participants, and the results show that VR BioTalk is highly usable, engaging, and easy to use, enabling experts with no programming background to explore large phenotyping datasets and generate hypotheses in natural language.