skip to main content


Title: Kit-Net: Self-Supervised Learning to Kit Novel 3D Objects into Novel 3D Cavities
Award ID(s):
1734633
NSF-PAR ID:
10314372
Author(s) / Creator(s):
; ; ; ; ; ; ; ;
Date Published:
Journal Name:
Conference on Automation Science and Engineering (CASE) 2021
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract

    Realistic 3D indoor scene datasets have enabled significant recent progress in computer vision, scene understanding, autonomous navigation, and 3D reconstruction. But the scale, diversity, and customizability of existing datasets is limited, and it is time‐consuming and expensive to scan and annotate more. Fortunately, combinatorics is on our side: there are enough individualroomsin existing 3D scene datasets, if there was but a way to recombine them into new layouts. In this paper, we propose the task of generating novel 3D floor plans from existing 3D rooms. We identify three sub‐tasks of this problem: generation of 2D layout, retrieval of compatible 3D rooms, and deformation of 3D rooms to fit the layout. We then discuss different strategies for solving the problem, and design two representative pipelines: one uses available 2D floor plans to guide selection and deformation of 3D rooms; the other learns to retrieve a set of compatible 3D rooms and combine them into novel layouts. We design a set of metrics that evaluate the generated results with respect to each of the three subtasks and show that different methods trade off performance on these subtasks. Finally, we survey downstream tasks that benefit from generated 3D scenes and discuss strategies in selecting the methods most appropriate for the demands of these tasks.

     
    more » « less
  2. We developed a novel whisker-follicle sensor that measures three mechanical signals at the whisker base. The first two signals are closely related to the two bending moments, and the third is an approximation to the axial force. Previous simulation studies have shown that these three signals are sufficient to determine the three-dimensional (3D) location at which the whisker makes contact with an object. Here we demonstrate hardware implementation of 3D contact point determination and then use continuous sweeps of the whisker to show proof-of principle 3D contour extraction. We begin by using simulations to confirm the uniqueness of the mapping between the mechanical signals at the whisker base and the 3D contact point location for the specific dimensions of the hardware whisker. Multi-output random forest regression is then used to predict the contact point locations of objects based on observed mechanical signals. When calibrated to the simulated data, signals from the hardware whisker can correctly predict contact point locations to within 1.5 cm about 74% of the time. However, if normalized output voltages from the hardware whiskers are used to train the algorithm (without calibrating to simulation), predictions improve to within 1.5 cm for about 96% of contact points and to within 0.6 cm for about 78% of contact points. This improvement suggests that as long as three appropriate predictor signals are chosen, calibrating to simulations may not be required. The sensor was next used to perform contour extraction on a cylinder and a cone. We show that basic contour extraction can be obtained with just two sweeps of the sensor. With further sweeps, it is expected that full 3D shape reconstruction could be achieved. 
    more » « less
  3. Current methods for assessing cell proliferation in 3D scaffolds rely on changes in metabolic activity or total DNA, however, direct quantification of cell number in 3D scaffolds remains a challenge. To address this issue, we developed an unbiased stereology approach that uses systematic-random sampling and thin focal-plane optical sectioning of the scaffolds followed by estimation of total cell number (StereoCount). This approach was validated against an indirect method for measuring the total DNA (DNA content); and the Bürker counting chamber, the current reference method for quantifying cell number. We assessed the total cell number for cell seeding density (cells per unit volume) across four values and compared the methods in terms of accuracy, ease-of-use and time demands. The accuracy of StereoCount markedly outperformed the DNA content for cases with ~ 10,000 and ~ 125,000 cells/scaffold. For cases with ~ 250,000 and ~ 375,000 cells/scaffold both StereoCount and DNA content showed lower accuracy than the Bürker but did not differ from each other. In terms of ease-of-use, there was a strong advantage for the StereoCount due to output in terms of absolute cell numbers along with the possibility for an overview of cell distribution and future use of automation for high throughput analysis. Taking together, the StereoCount method is an efficient approach for direct cell quantification in 3D collagen scaffolds. Its major benefit is that automated StereoCount could accelerate research using 3D scaffolds focused on drug discovery for a wide variety of human diseases. 
    more » « less