Search for: All records

We introduce Visual Inverse Kinematics (VIK), which finds kinematically feasible joint configurations that satisfy vision-based constraints, bridging the gap between inverse kinematics (IK) and visual servoing (VS). Unlike IK, no explicit end-effector pose is given, and unlike VS, exact image measurements may not be available. In this work, we develop a formulation of the VIK problem with a field of view (FoV) constraint, enforcing the visibility of an object from a camera on the robot. Our proposed solution introduces a virtual kinematic chain that connects the physical robot and the object, transforming the FoV constraint into a joint angle kinematic constraint. Along the way, we introduce multiple vision-based cost functions to fulfill different objectives. We solve this formulation of the VIK problem using a method that involves a semidefinite program (SDP) constraint followed by a rank minimization algorithm. The performance of this method for solving the VIK problem is validated through simulations. 

While humans can successfully navigate using abstractions, ignoring details that are irrelevant to the task at hand, most of the existing approaches in robotics require detailed environment representations which consume a significant amount of sensing, computing, and storage; these issues become particularly important in resource-constrained settings with limited power budgets. Deep learning methods can learn from prior experience to abstract knowledge from novel environments, and use it to more efficiently execute tasks such as frontier exploration, object search, or scene understanding. We propose BoxMap, a Detection-Transformer-based architecture that takes advantage of the structure of the sensed partial environment to update a topological graph of the environment as a set of semantic entities (rooms and doors) and their relations (connectivity). The predictions from low-level measurements can be leveraged to achieve high-level goals with lower computational costs than methods based on detailed representations. As an example application, we consider a robot equipped with a 2-D laser scanner tasked with exploring a residential building. Our BoxMap representation scales quadratically with the number of rooms (with a small constant), resulting in significant savings over a full geometric map. Moreover, our high-level topological representation results in 30.9% shorter trajectories in the exploration task with respect to a standard method. Code is available at: bit.ly/3F6w2Yl. 

Deep learning methods have been widely used in robotic applications, making learning-enabled control design for complex nonlinear systems a promising direction. Although deep reinforcement learning methods have demonstrated impressive empirical performance, they lack the stability guarantees that are important in safety-critical situations. One way to provide these guarantees is to learn Lyapunov certificates alongside control policies. There are three related problems: 1) verify that a given Lyapunov function candidate satisfies the conditions for a given controller on a region, 2) find a valid Lyapunov function and controller on a given region, and 3) find a valid Lyapunov function and a controller such that the region of attraction is as large as possible. Previous work has shown that if the dynamics are piecewise linear, it is possible to solve problem 1) and 2) by solving a Mixed-Integer Linear Program (MILP). In this work, we build upon this method by proposing a Lyapunov neural network that considers monotonicity over half spaces in different directions. We 1) propose a specific choice of Lyapunov function architecture that ensures non-negativity and a unique global minimum by construction, and 2) show that this can be leveraged to find the controller and Lyapunov certificates faster and with a larger valid region by maximizing the size of a square inscribed in a given level set. We apply our method to a 2D inverted pendulum, unicycle path following, a 3-D feedback system, and a 4-D cart pole system, and demonstrate it can shorten the training time by half compared to the baseline, as well as find a larger ROA. 

Inverse kinematics (IK) is an important problem in robot control and motion planning; however, the nonlinearity of the map from joint angles to robot configurations makes the problem nonconvex. In this paper, we propose an inverse kinematics solver that works in the space of rotation matrices of the link reference frames rather than joint angles. To overcome the nonlinearity of the manifold of rotation matrices $$\mathbf{SO}(3)$$, we propose a semidefinite programming (SDP) relaxation of the kinematic constraints followed by a fixed-trace rank minimization via maximization of a convex function. Along the way, we show that the feasible set of an IK problem is exactly the intersection of a convex set and fixed-trace rank-1 matrices. Thanks to the use of matrices with fixed trace, our algorithm to obtain rank-1 solutions has guaranteed local convergence. Unlike some traditional solvers, our method does not require an initial guess, and can be applied to robots with closed kinematic chains without ad-hoc modifications such as splitting the kinematic chain. Compared to other work that performs SDP relaxation for IK problems, our formulation is simpler, and uses variables with smaller sizes. We validate our approach via simulations on a closed kinematic chain constituted by two robotic arms holding a box, comparing against a standard IK method. 

Deep learning methods are widely used in robotic applications. By learning from prior experience, the robot can abstract knowledge of the environment, and use this knowledge to accomplish different goals, such as object search, frontier exploration, or scene understanding, with a smaller amount of resources than might be needed without that knowledge. Most existing methods typically require a significant amount of sensing, which in turn has significant costs in terms of power consumption for acquisition and processing, and typically focus on models that are tuned for each specific goal, leading to the need to train, store and run each one separately. These issues are particularly important in a resource-constrained setting, such as with small-scale robots or during long-duration missions. We propose a single, multi-task deep learning architecture that takes advantage of the structure of the partial environment to predict different abstractions of the environment (thus reducing the need for rich sensing), and to leverage these predictions to simultaneously achieve different high-level goals (thus sharing computation between goals). As an example application of the proposed architecture, we consider the specific example of a robot equipped with a 2-D laser scanner and an object detector, tasked with searching for an object (such as an exit) in a residential building while constructing a topological map that can be used for future missions. The prior knowledge of the environment is encoded using a U-Net deep network architecture. In this context, our work leads to an object search algorithm that is complete, and that outperforms a more traditional frontier-based approach. The topological map we produce uses scene trees to qualitatively represent the environment as a graph at a fraction of the cost of existing SLAM-based solutions. Our results demonstrate that it is possible to extract multi-task semantic information that is useful for navigation and mapping directly from bare-bone, non-semantic measurements. 

The Weighted-Mean Subsequence Reduced (W-MSR) algorithm, the state-of-the-art method for Byzantine-resilient design of decentralized multi-robot systems, is based on discarding outliers received over Linear Consensus Protocol (LCP). Although W-MSR provides theoretical guarantees relating network connectivity to the convergence of the underlying consensus, W-MSR comes with several limitations: the number of Byzantine robots, 𝐹 , to tolerate should be known a priori, each robot needs to maintain 2𝐹 + 1 neighbors, 𝐹 + 1 robots must independently make local measurements of the consensus property in order for the swarm’s decision to change, and W-MSR is specific to LCP and does not generalize to applications not implemented over LCP. In this work, we pro- pose a Decentralized Blocklist Protocol (DBP) based on inter-robot accusations. Accusations are made on the basis of locally-made observations of misbehavior, and once shared by cooperative robots across the network are used as input to a graph matching algorithm that computes a blocklist. DBP generalizes to applications not implemented via LCP, is adaptive to the number of Byzantine robots, and allows for fast information propagation through the multi- robot system while simultaneously reducing the required network connectivity relative to W-MSR. On LCP-type applications, DBP reduces the worst-case connectivity requirement of W-MSR from (2𝐹 + 1)-connected to (𝐹 + 1)-connected and the minimum number of cooperative observers required to propagate new information from 𝐹 + 1 to just 1 observer. We demonstrate that our approach to Byzantine resilience scales to hundreds of robots on target tracking, time synchronization, and localization case studies. 

In centralized multi-robot systems, a central entity (CE) checks that robots follow their assigned motion plans by comparing their expected location to the location they self-report. We show that this self-reporting monitoring mechanism is vulnerable to plan- deviation attacks where compromised robots don’t follow their assigned plans while trying to conceal their movement by misreporting their location. We propose a two-pronged mitigation for plan-deviation attacks: (1) an attack detection technique leveraging both the robots’ local sensing capabilities to report observations of other robots and co-observation schedules generated by the CE, and (2) a prevention technique where the CE issues horizon-limiting announcements to the robots, reducing their instantaneous knowledge of forward lookahead steps in the global motion plan. On a large-scale automated warehouse benchmark, we show that our solution enables attack prevention guarantees from a stealthy attacker that has compromised multiple robots.