skip to main content

This content will become publicly available on July 7, 2023

Title: CoFrame: A System for Training Novice Cobot Programmers
The introduction of collaborative robots (cobots) into the workplace has presented both opportunities and challenges for those seeking to utilize their functionality. Prior research has shown that despite the capabilities afforded by cobots, there is a disconnect between those capabilities and the applications that they currently are deployed in, partially due to a lack of effective cobot-focused instruction in the field. Experts who work successfully within this collaborative domain could offer insight into the considerations and process they use to more effectively capture this cobot capability. Using an analysis of expert insights in the collaborative interaction design space, we developed a set of Expert Frames based on these insights and integrated these Expert Frames into a new training and programming system that can be used to teach novice operators to think, program, and troubleshoot in ways that experts do. We present our system and case studies that demonstrate how Expert Frames provide novice users with the ability to analyze and learn from complex cobot application scenarios.
; ; ; ; ;
Award ID(s):
1822872 1925043
Publication Date:
Journal Name:
HRI '22: Proceedings of the 2022 ACM/IEEE International Conference on Human-Robot Interaction
Page Range or eLocation-ID:
Sponsoring Org:
National Science Foundation
More Like this
  1. Ruis, Andrew R. ; Lee, Seung B. (Ed.)
    Rapid advances in technology also come with increased training needs for people who engineer and interact with these technologies. One such technology is collaborative robots, cobots, which are designed to be safer and easier to use than their traditional robotic counterparts. However, there have been few studies of how people use cobots and even fewer identifying what a user must know to properly set up and effectively use cobots for their manufacturing processes. In this study, we interviewed nine experts in robots and automation in manufacturing settings. We employ a quantitative ethnographic approach to gain qualitative insights into the cultural practices of robotics experts and corroborate these stories with quantitative warrants. Both quantitative and qualitative analyses revealed that experts put safety first when designing and monitoring cobot applications. This study improves our understanding of expert problem-solving in collaborative robotics, defines an expert model that can serve as a basis for the development of an authentic learning technology, and illustrates a useful method for modeling expertise in vocational settings.
  2. Collaborative robots, or cobots, represent a breakthrough technology designed for high-level (e.g., collaborative) interactions between workers and robots with capabilities for flexible deployment in industries such as manufacturing. Understanding how workers and companies use and integrate cobots is important to inform the future design of cobot systems and educational technologies that facilitate effective worker-cobot interaction. Yet, little is known about typical training for collaboration and the application of cobots in manufacturing. To close this gap, we interviewed nine experts in manufacturing about their experience with cobots. Our thematic analysis revealed that, contrary to the envisioned use, experts described most cobot applications as only low-level (e.g., pressing start/stop buttons) interactions with little flexible deployment, and experts felt traditional robotics skills were needed for collaborative and flexible interaction with cobots. We conclude with design recommendations for improved future robots, including programming and interface designs, and educational technologies to support collaborative use.
  3. Objective

    Trade-offs between productivity, physical workload (PWL), and mental workload (MWL) were studied when integrating collaborative robots (cobots) into existing manual work by optimizing the allocation of tasks.


    As cobots become more widely introduced in the workplace and their capabilities greatly improved, there is a need to consider how they can best help their human partners.


    A theoretical data-driven analysis was conducted using the O*NET Content Model to evaluate 16 selected jobs for associated work context, skills, and constraints. Associated work activities were ranked by potential for substitution by a cobot. PWL and MWL were estimated using variables from the O*Net database that represent variables for the Strain Index and NASA-TLX. An algorithm was developed to optimize work activity assignment to cobots and human workers according to their most suited abilities.


    Human workload for some jobs decreased while workload for some jobs increased after cobots were reassigned tasks, and residual human capacity was used to perform job activities designated the most important to increase productivity. The human workload for other jobs remained unchanged.


    The changes in human workload from the introduction of cobots may not always be beneficial for the human worker unless trade-offs are considered. Application: The framework of thismore »study may be applied to existing jobs to identify the relationship between productivity and worker tolerances that integrate cobots into specific tasks.

    « less
  4. It is hard for experts to create good instructional resources due to a phenomenon known as the expert blind spot: They forget what it was like to be a novice, so they cannot pinpoint exactly where novices commonly struggle and how to best phrase their explanations. To help overcome these expert blind spots for computer programming topics, we created a learnersourcing system that elicits explanations of misconceptions directly from learners while they are coding. We have deployed this system for the past three years to the widely-used Python Tutor coding website ( and collected 16,791 learner-written explanations. To our knowledge, this is the largest dataset of explanations for programming misconceptions. By inspecting this dataset, we found surprising insights that we did not originally think of due to our own expert blind spots as programming instructors. We are now using these insights to improve compiler and run-time error messages to explain common novice misconceptions.
  5. Digital games featuring programmable agents are popular tools for teaching coding and computational thinking skills. However, today's games perpetuate an arguably obsolete relationship between programmable agents and human operators. Borrowing from the field of human-robotics interaction, we argue that collaborative robots, or cobots, are a better model for thinking about computational agents, working directly with humans rather than in place of or at arm's length from them. In this paper, we describe an initial design inquiry into the design of “cobot games”, programmable agent scenarios in which players program an in-game ally to assist them in accomplishing gameplay objectives. We detail three questions that emerged out of this exploration, our present thinking on them, and plans for deepening inquiry into cobot game design moving forward.