Search for: All records

Animals and robots must self-right on the ground after overturning. Biology research has described various strategies and motor patterns in many species. Robotics research has devised many strategies. However, we do not well understand the physical principles of how the need to generate mechanical energy to overcome the potential energy barrier governs behavioral strategies and 3D body rotations given the morphology. Here, I review progress on this which I led studying cockroaches self-righting on level, flat, solid, low-friction ground, by integrating biology experiments, robotic modeling, and physics modeling. Animal experiments using three species (Madagascar hissing, American, and discoid cockroaches) found that ground self-righting is strenuous and often requires multiple attempts to succeed. Two species (American and discoid cockroaches) often self-right dynamically, using kinetic energy to overcome the barrier. All three species use and often stochastically transition across diverse strategies. In these strategies, propelling motions are often accompanied by perturbing motions. All three species often display complex yet stereotyped body rotation. They all roll more in successful attempts than in failed ones, which lowers the barrier, as revealed by a simplistic 3D potential energy landscape of a rigid body self-righting. Experiments of an initial robot self-righting via rotation about a fixed axis revealed that the longer and faster appendages push, the more mechanical energy can be gained to overcome the barrier. However, the cockroaches rarely achieve this. To further understand the physical principles of strenuous ground self-righting, we focused on the discoid cockroach’s leg-assisted winged self-righting. In this strategy, wings propel against the ground to pitch the body up but are unable to overcome the highest pitch barrier. Meanwhile, legs flail in the air to perturb the body sideways to self-right via rolling. Experiments using a refined robot and an evolving 3D potential energy landscape revealed that, although wing propelling cannot generate sufficient kinetic energy to overcome the highest pitch barrier, it reduces the barrier to allow small kinetic energy from the perturbing legs to probabilistically overcome the barrier to self-right via rolling. Thus, only by combining propelling and perturbing can self-righting be achieved when it is so strenuous; this physical constraint leads to the stereotyped body rotation. Finally, multi-body dynamics simulation and template modeling revealed that the animal’s substantial randomness in wing and leg motions helps it, by chance, to find good coordination, which accumulates more mechanical energy to overcome the barrier, thus increasing the likelihood of self-righting.

 

This paper presents Multi-View Attentive Contextualization (MvACon), a simple yet effective method for improving 2D- to-3D feature lifting in query-based multi-view 3D (MV3D) object detection. Despite remarkable progress witnessed in the field of query-based MV3D object detection, prior art often suffers from either the lack of exploiting high- resolution 2D features in dense attention-based lifting, due to high computational costs, or from insufficiently dense grounding of 3D queries to multi-scale 2D features in sparse attention-based lifting. Our proposed MvACon hits the two birds with one stone using a representationally dense yet computationally sparse attentive feature contextualization scheme that is agnostic to specific 2D-to-3D feature lifting approaches. In experiments, the proposed MvACon is thoroughly tested on the nuScenes benchmark, using both the BEVFormer and its recent 3D deformable attention (DFA3D) variant, as well as the PETR, showing consistent detection performance improvement, especially in enhancing performance in location, orientation, and velocity prediction. It is also tested on the Waymo-mini benchmark using BEVFormer with similar improvement. We qualitatively and quantitatively show that global cluster-based contexts effectively encode dense scene-level contexts for MV3D object detection. The promising results of our proposed MvACon reinforces the adage in computer vision – “(contextualized) feature matters”. 

In the realm of neuroscience, mapping the three-dimensional (3D) neural circuitry and architecture of the brain is important for advancing our understanding of neural circuit organization and function. This study presents a novel pipeline that transforms mouse brain samples into detailed 3D brain models using a collaborative data analytics platform called “Texera.” The user-friendly Texera platform allows for effective interdisciplinary collaboration between team members in neuroscience, computer vision, and data processing. Our pipeline utilizes the tile images from a serial two-photon tomography/TissueCyte system, then stitches tile images into brain section images, and constructs 3D whole-brain image datasets. The resulting 3D data supports downstream analyses, including 3D whole-brain registration, atlas-based segmentation, cell counting, and high-resolution volumetric visualization. Using this platform, we implemented specialized optimization methods and obtained significant performance enhancement in workflow operations. We expect the neuroscience community can adopt our approach for large-scale image-based data processing and analysis.

 

Introducing interpretability and reasoning into Multiple Instance Learning (MIL) methods for Whole Slide Image (WSI) analysis is challenging given the complexity of gigapixel slides. Traditionally MIL interpretability is limited to identifying salient regions deemed pertinent for downstream tasks offering little insight to the end-user (pathologist) regarding the rationale behind these selections. To address this we propose Self-Interpretable MIL (SI-MIL) a method intrinsically designed for interpretability from the very outset. SI-MIL employs a deep MIL framework to guide an interpretable branch grounded on handcrafted pathological features facilitating linear predictions. Beyond identifying salient regions SI-MIL uniquely provides feature-level interpretations rooted in pathological insights for WSIs. Notably SI-MIL with its linear prediction constraints challenges the prevalent myth of an inevitable trade-off between model interpretability and performance demonstrating competitive results compared to state-of-the-art methods on WSI-level prediction tasks across three cancer types. In addition we thoroughly benchmark the local- and global-interpretability of SI-MIL in terms of statistical analysis a domain expert study and desiderata of interpretability namely user-friendliness and faithfulness.