An official website of the United States government
Inverse kinematics (IK) is an important problem in robot control and motion planning; however, the nonlinearity of the map from joint angles to robot configurations makes the problem nonconvex. In this paper, we propose an inverse kinematics solver that works in the space of rotation matrices of the link reference frames rather than joint angles. To overcome the nonlinearity of the manifold of rotation matrices $$\mathbf{SO}(3)$$, we propose a semidefinite programming (SDP) relaxation of the kinematic constraints followed by a fixed-trace rank minimization via maximization of a convex function. Along the way, we show that the feasible set of an IK problem is exactly the intersection of a convex set and fixed-trace rank-1 matrices. Thanks to the use of matrices with fixed trace, our algorithm to obtain rank-1 solutions has guaranteed local convergence. Unlike some traditional solvers, our method does not require an initial guess, and can be applied to robots with closed kinematic chains without ad-hoc modifications such as splitting the kinematic chain. Compared to other work that performs SDP relaxation for IK problems, our formulation is simpler, and uses variables with smaller sizes. We validate our approach via simulations on a closed kinematic chain constituted by two robotic arms holding a box, comparing against a standard IK method.
more »
« less
This content will become publicly available on July 10, 2026
Visual Inverse Kinematics: Finding Feasible Robot Poses under Kinematic and Vision Constraints
We introduce Visual Inverse Kinematics (VIK), which finds kinematically feasible joint configurations that satisfy vision-based constraints, bridging the gap between inverse kinematics (IK) and visual servoing (VS). Unlike IK, no explicit end-effector pose is given, and unlike VS, exact image measurements may not be available. In this work, we develop a formulation of the VIK problem with a field of view (FoV) constraint, enforcing the visibility of an object from a camera on the robot. Our proposed solution introduces a virtual kinematic chain that connects the physical robot and the object, transforming the FoV constraint into a joint angle kinematic constraint. Along the way, we introduce multiple vision-based cost functions to fulfill different objectives. We solve this formulation of the VIK problem using a method that involves a semidefinite program (SDP) constraint followed by a rank minimization algorithm. The performance of this method for solving the VIK problem is validated through simulations.
more »
« less
- Award ID(s):
- 2212051
- PAR ID:
- 10621339
- Publisher / Repository:
- IEEE American Control Conference
- Date Published:
- Journal Name:
- W Telekomunikacji
- Format(s):
- Medium: X
- Sponsoring Org:
- National Science Foundation
More Like this
-
-
This paper presents an inverse kinematic controller using neural networks for trajectory controlling of a delta robot in real-time. The developed control scheme is purely data-driven and does not require prior knowledge of the delta robot kinematics. Moreover, it can adapt to the changes in the kinematics of the robot. For developing the controller, the kinematic model of the delta robot is estimated by using neural networks. Then, the trained neural networks are configured as a controller in the system. The parameters of the neural networks are updated while the robot follows a path to adaptively compensate for modeling uncertainties and external disturbances of the control system. One of the main contributions of this paper is to show that updating the parameters of neural networks offers a smaller tracking error in inverse kinematic control of a delta robot with consideration of joint backlash. Different simulations and experiments are conducted to verify the proposed controller. The results show that in the presence of external disturbance, the error in trajectory tracking is bounded, and the negative effect of joint backlash in trajectory tracking is reduced. The developed method provides a new approach to the inverse kinematic control of a delta robot.more » « less
-
This paper presents a visual servoing method for controlling a robot in the configuration space by purely using its natural features. We first created a data collection pipeline that uses camera intrinsics, extrinsics, and forward kinematics to generate 2D projections of a robot's joint locations (keypoints) in image space. Using this pipeline, we are able to collect large sets of real-robot data, which we use to train realtime keypoint detectors. The inferred keypoints from the trained model are used as control features in an adaptive visual servoing scheme that estimates, in runtime, the Jacobian relating the changes of the keypoints and joint velocities. We compared the 2D configuration control performance of this method to the skeleton-based visual servoing method (the only other algorithm for purely vision-based configuration space visual servoing), and demonstrated that the keypoints provide more robust and less noisy features, which result in better transient response. We also demonstrate the first vision-based 3D configuration space control results in the literature, and discuss its limitations. Our data collection pipeline is available at https://github.com/JaniC-WPI/KPDataGenerator.git which can be utilized to collect image datasets and train realtime keypoint detectors for various robots and environments.more » « less
-
We introduce a system that exploits the screen and front-facing camera of a mobile device to perform three-dimensional deflectometry-based surface measurements. In contrast to current mobile deflectometry systems, our method can capture surfaces with large normal variation and wide field of view (FoV). We achieve this by applying automated multi-view panoramic stitching algorithms to produce a large FoV normal map from a hand-guided capture process without the need for external tracking systems, like robot arms or fiducials. The presented work enables 3D surface measurements of specular objects ’in the wild’ with a system accessible to users with little to no technical imaging experience. We demonstrate high-quality 3D surface measurements without the need for a calibration procedure. We provide experimental results with our prototype Deflectometry system and discuss applications for computer vision tasks such as object detection and recognition.more » « less
-
null (Ed.)Continuum robots have strong potential for application in Space environments. However, their modeling is challenging in comparison with traditional rigid-link robots. The Kinematic-Model-Free (KMF) robot control method has been shown to be extremely effective in permitting a rigid-link robot to learn approximations of local kinematics and dynamics (“kinodynamics”) at various points in the robot’s task space. These approximations enable the robot to follow various trajectories and even adapt to changes in the robot’s kinematic structure. In this paper, we present the adaptation of the KMF method to a three-section, nine degrees-of-freedom continuum manipulator for both planar and spatial task spaces. Using only an external 3D camera, we show that the KMF method allows the continuum robot to converge to various desired set points in the robot’s task space, avoiding the complexities inherent in solving this problem using traditional inverse kinematics. The success of the method shows that a continuum robot can “learn” enough information from an external camera to reach and track desired points and trajectories, without needing knowledge of exact shape or position of the robot. We similarly apply the method in a simulated example of a continuum robot performing an inspection task on board the ISS.more » « less
