More Like this

1. Semantic Gaze Labeling for Human-Robot Shared Manipulation

   Aronson, Reuben M ; Admoni, Henny ( June 2019 , ACM Symposium on Eye Tracking Research & Applications)

   Human-robot collaboration systems benefit from recognizing people’s intentions. This capability is especially useful for collaborative manipulation applications, in which users operate robot arms to manipulate objects. For collaborative manipulation, systems can determine users’ intentions by tracking eye gaze and identifying gaze fixations on particular objects in the scene (i.e., semantic gaze labeling). Translating 2D fixation locations (from eye trackers) into 3D fixation locations (in the real world) is a technical challenge. One approach is to assign each fixation to the object closest to it. However, calibration drift, head motion, and the extra dimension required for real-world interactions make this position matching approach inaccurate. In this work, we introduce velocity features that compare the relative motion between subsequent gaze fixations and a nite set of known points and assign fixation position to one of those known points. We validate our approach on synthetic data to demonstrate that classifying using velocity features is more robust than a position matching approach. In addition, we show that a classifier using velocity features improves semantic labeling on a real-world dataset of human-robot assistive manipulation interactions. 

   more » « less
2. Semantic gaze labeling for human-robot shared manipulation

   https://doi.org/10.1145/3314111.3319840  

   Aronson, Reuben M. ; Admoni, Henny ( June 2019 , Symposium on Eye Tracking Research and Applications (ETRA))

   Human-robot collaboration systems benefit from recognizing people’s intentions. This capability is especially useful for collaborative manipulation applications, in which users operate robot arms to manipulate objects. For collaborative manipulation, systems can determine users’ intentions by tracking eye gaze and identifying gaze fixations on particular objects in the scene (i.e., semantic gaze labeling). Translating 2D fixation locations (from eye trackers) into 3D fixation locations (in the real world) is a technical challenge. One approach is to assign each fixation to the object closest to it. However, calibration drift, head motion, and the extra dimension required for real-world interactions make this position matching approach inaccurate. In this work, we introduce velocity features that compare the relative motion between subsequent gaze fixations and a finite set of known points and assign fixation position to one of those known points. We validate our approach on synthetic data to demonstrate that classifying using velocity features is more robust than a position matching approach. In addition, we show that a classifier using velocity features improves semantic labeling on a real-world dataset of human-robot assistive manipulation interactions. 

   more » « less
3. Inverse Error Function Trajectories for Image Reconstruction*This material is based upon work supported by the National Science Foundation under Grant No. 1662029

   https://doi.org/10.1109/IROS.2018.8594315  

   Katoch, Rohan ; Fusaro, Beatriz ; Ueda, Jun ( October 2018 , 2018 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS))

   Capturing clear images while a camera is moving fast, is integral to the development of mobile robots that can respond quickly and effectively to visual stimuli. This paper proposes to generate camera trajectories, with position and time constraints, that result in higher reconstructed image quality. The degradation in of an image captured during motion is known as motion blur. Three main methods exist for mitigating the effects of motion blur: (i) controlling optical parameters, (ii) controlling camera motion, and (iii) image reconstruction. Given control of a camera's motion, trajectories can be generated that result in an expected blur kernel or point-spread function. This work compares the motion blur effects and reconstructed image quality of three trajectories: (i) linear, (ii) polynomial, and (iii) inverse error. Where inverse error trajectories result in Gaussian blur kernels. Residence time analysis provides a basis for characterizing the motion blur effects of the trajectories 

   more » « less
4. A Deep Learning Approach to Localization for Navigation on a Miniature Autonomous Blimp

   https://doi.org/10.1109/ICCA51439.2020.9264514  

   Seguin, Landan ; Zheng, Justin ; Li, Alberto ; Tao, Qiuyang ; Zhang, Fumin ( October 2020 , IEEE International Conference on Control and Automation (ICCA))

   null (Ed.)

   The Georgia Tech Miniature Autonomous Blimp (GT-MAB) needs localization algorithms to navigate to way-points in an indoor environment without leveraging an external motion capture system. Indoor aerial robots often require a motion capture system for localization or employ simultaneous localization and mapping (SLAM) algorithms for navigation. The proposed strategy for GT-MAB localization can be accomplished using lightweight sensors on a weight-constrained platform like the GT-MAB. We train an end-to-end convolutional neural network (CNN) that predicts the horizontal position and heading of the GT-MAB using video collected by an onboard monocular RGB camera. On the other hand, the height of the GT-MAB is estimated from measurements through a time-of-flight (ToF) single-beam laser sensor. The monocular camera and the single-beam laser sensor are sufficient for the localization algorithm to localize the GT-MAB in real time, achieving the averaged 3D positioning errors to be less than 20 cm, and the averaged heading errors to be less than 3 degrees. With the accuracy of our proposed localization method, we are able to use simple proportional-integral-derivative controllers to control the GT-MAB for waypoint navigation. Experimental results on the waypoint following are provided, which demonstrates the use of a CNN as the primary localization method for estimating the pose of an indoor robot that successfully enables navigation to specified waypoints. 

   more » « less
5. NudgeSeg: Zero-Shot Object Segmentation by Repeated Physical Interaction

   https://doi.org/10.1109/IROS51168.2021.9635832  

   Singh, Chahat Deep ; Sanket, Nitin J. ; Parameshwara, Chethan M. ; Fermuller, Cornelia ; Aloimonos, Yiannis ( September 2021 , 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS))

   Recent advances in object segmentation have demonstrated that deep neural networks excel at object segmentation for specific classes in color and depth images. However, their performance is dictated by the number of classes and objects used for training, thereby hindering generalization to never seen objects or zero-shot samples. To exacerbate the problem further, object segmentation using image frames rely on recognition and pattern matching cues. Instead, we utilize the ‘active’ nature of a robot and their ability to ‘interact’ with the environment to induce additional geometric constraints for segmenting zero-shot samples. In this paper, we present the first framework to segment unknown objects in a cluttered scene by repeatedly ‘nudging’ at the objects and moving them to obtain additional motion cues at every step using only a monochrome monocular camera. We call our framework NudgeSeg. These motion cues are used to refine the segmentation masks. We successfully test our approach to segment novel objects in various cluttered scenes and provide an extensive study with image and motion segmentation methods. We show an impressive average detection rate of over 86% on zero-shot objects. 

   more » « less