skip to main content


Title: Deep Coverage: Motion Synthesis In The Data-Driven Era
Effective robotic systems must be able to produce desired motion in a sufficiently broad variety of robot states and environmental contexts. Classic control and planning methods achieve such coverage through the synthesis of model-based components. New applications and platforms, such as soft robots, present novel challenges, ranging from richer dynamical behaviors to increasingly unstructured environments. In these setups, derived models frequently fail to express important real-world subtleties. An increasingly popular approach to deal with this issue corresponds to end-to-end machine learning architectures, which adapt to such complexities through a data-driven process. Unfortunately, however, data are not always available for all regions of the operational space, which complicates the extensibility of these solutions. In light of these issues, this paper proposes a reconciliation of classic motion synthesis with modern data-driven tools towards the objective of ``deep coverage''. This notion utilizes the concept of composability, a feature of traditional control and planning methods, over data-derived ``motion elements'', towards generalizable and scalable solutions that adapt to real-world experience.  more » « less
Award ID(s):
1734492 1723869
NSF-PAR ID:
10086341
Author(s) / Creator(s):
;
Date Published:
Journal Name:
International Symposium on Robotics Research (ISRR)
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Recently, much attention has been devoted to the development of generative network traces and their potential use in supplementing real-world data for a variety of data-driven networking tasks. Yet, the utility of existing synthetic traffic approaches are limited by their low fidelity: low feature granularity, insufficient adherence to task constraints, and subpar class coverage. As effective network tasks are increasingly reliant on raw packet captures, we advocate for a paradigm shift from coarse-grained to fine-grained traffic generation compliant to constraints. We explore this path employing controllable diffusion-based methods. Our preliminary results suggest its effectiveness in generating realistic and fine-grained network traces that mirror the complexity and variety of real network traffic required for accurate service recognition. We further outline the challenges and opportunities of this approach, and discuss a research agenda towards text-to-traffic synthesis. 
    more » « less
  2. null (Ed.)
    The objective of this work is to augment the basic abilities of a robot by learning to use sensorimotor primitives to solve complex long-horizon manipulation problems. This requires flexible generative planning that can combine primitive abilities in novel combinations and, thus, generalize across a wide variety of problems. In order to plan with primitive actions, we must have models of the actions: under what circumstances will executing this primitive successfully achieve some particular effect in the world? We use, and develop novel improvements to, state-of-the-art methods for active learning and sampling. We use Gaussian process methods for learning the constraints on skill effectiveness from small numbers of expensive-to-collect training examples. In addition, we develop efficient adaptive sampling methods for generating a comprehensive and diverse sequence of continuous candidate control parameter values (such as pouring waypoints for a cup) during planning. These values become end-effector goals for traditional motion planners that then solve for a full robot motion that performs the skill. By using learning and planning methods in conjunction, we take advantage of the strengths of each and plan for a wide variety of complex dynamic manipulation tasks. We demonstrate our approach in an integrated system, combining traditional robotics primitives with our newly learned models using an efficient robot task and motion planner. We evaluate our approach both in simulation and in the real world through measuring the quality of the selected primitive actions. Finally, we apply our integrated system to a variety of long-horizon simulated and real-world manipulation problems. 
    more » « less
  3. It is increasingly acknowledged that urban and landscape planning processes need to incorporate stakeholder input and feedback. To this end, decision-makers have been implementing a range of decision support systems (DSSs), such as using geographic information systems (GIS) or 3D renderings of designs to help better explain the advantages and disadvantages of proposed designs. In addition, urban and landscape planning DSSs have also incorporated gamification (the use of game features and mechanics in non-game environments) to provide interactivity whilst providing an engaging experience. In these contexts, using 3D renderings of real-world environments can be a powerful tool for aiding in the democratisation of planning decisions. However, the creation of large-scale 3D models representing real cities or landscapes is limited by time-intensive manual methods. This is compounded by the fact that under our current rapidly changing environment, landscapes and urban areas are likely to alter in appearance within short periods of time. It is therefore imperative that 3D renderings of real-world environments can adapt to these changes. Here, we propose methods of using GIS datasets to automatically generate in-game worlds reflective of the real-world and how these 3D models can be used to engage citizens in planning decisions. 
    more » « less
  4. null (Ed.)
    Theoretical and Empirical Modeling of Identity and Sentiments in Collaborative Groups (THEMIS.COG) was an interdisciplinary research collaboration of computer scientists and social scientists from the University of Waterloo (Canada), Potsdam University of Applied Sciences (Germany), and Dartmouth College (USA). This white paper summarizes the results of our research at the end of the grant term. Funded by the Trans-Atlantic Platform’s Digging Into Data initiative, the project aimed at theoretical and empirical modeling of identity and sentiments in collaborative groups. Understanding the social forces behind self-organized collaboration is important because technological and social innovations are increasingly generated through informal, distributed processes of collaboration, rather than in formal organizational hierarchies or through market forces. Our work used a data-driven approach to explore the social psychological mechanisms that motivate such collaborations and determine their success or failure. We focused on the example of GitHub, the world’s current largest digital platform for open, collaborative software development. In contrast to most, purely inductive contemporary approaches leveraging computational techniques for social science, THEMIS.COG followed a deductive, theory-driven approach. We capitalized on affect control theory, a mathematically formalized theory of symbolic interaction originated by sociologist David R. Heise and further advanced in previous work by some of the THEMIS.COG collaborators, among others. Affect control theory states that people control their social behaviours by intuitively attempting to verify culturally shared feelings about identities, social roles, and behaviour settings. From this principle, implemented in computational simulation models, precise predictions about group dynamics can be derived. It was the goal of THEMIS.COG to adapt and apply this approach to study the GitHub collaboration ecosystem through a symbolic interactionist lens. The project contributed substantially to the novel endeavor of theory development in social science based on large amounts of naturally occurring digital data. 
    more » « less
  5. The evolution of Mechatronics and Robotics Engineering (MRE) has enabled numerous technological advancements since the early 20th century. Professionals in this field are reshaping the world by designing smart and autonomous systems aiming to improve human well-being. Recognizing the need for preparing highly-educated MRE professionals, many universities and colleges are adopting MRE as a distinct degree program. One of the cornerstones of MRE education is laboratory- and project-based learning to provide a hands-on and engaging experience for the students. To this end, numerous software and hardware platforms have been developed and utilized in MRE courses and laboratories. Commercial products can provide a rich hands-on experience for the students, but they can be cost-prohibitive. On the other hand, open-source platforms are low-cost alternatives to their commercial counterparts and are being increasingly used in industry. Developing open-source laboratory platforms will be a more feasible option for a wider range of institutions and would enable familiarizing the students with recent technological trends in industry and exposing them to the development details of a real-world system. However, adoption of open-source platforms in MRE courses can be lengthy and time consuming. Educators who wish to utilize such systems typically lack the expertise in all aspects of their implementation which can make them difficult to troubleshoot. Debugging open-source systems can also be challenging because most of the troubleshooting is done through forum discussions which appear to be very noisy and unfocused. The flip side of this chaotic nature of the open-source world is that there is a vast amount of information available, including tutorials, examples, and commentary and, with some focused searching, debugging and usage questions can often get answered. There is also a disconnect between the forum participants, typically computer scientists and hobbyists, and MRE educators and students. Finally, the available resources and documentation for utilizing open-source platforms in MRE education are insufficient and incomprehensive. Therefore, the main goal of this paper is to increase awareness and familiarity with the use of open-source software and hardware packages in MRE education and practice towards accelerating their adoption. To this end, open-source software packages such as Python, GNU Octave, OpenFOAM, Java, Modelica, Gazebo, SPICE, Scilab, and Gnuplot, which have the potential to be useful in the modeling and analysis of MRE systems are introduced. Furthermore, low-cost and powerful open-source hardware packages such as Arduino, Raspberry Pi, and BeagleBone which can be used as the main processing unit for data acquisition and control implementation in a wide range of MRE systems are reviewed and their limitations and potentials are investigated. This paper provides a valuable resource for MRE students and faculty who would like to utilize open-source hardware and software platforms in their education and research. 
    more » « less