Robots can collect information about their environments through on-board sensors, and then leverage this information through a feedback control law. A map can be viewed as an idealized representation of the environment that provides guidance for such intelligent behaviors. In addition to their sensor measurements, robots can retrieve additional information from the map to achieve more sophisticated behaviors than just using local measurements. By intentionally manipulating the map and allowing the robot to react to such changes, we may be able to produce a variety of robot behaviors without changing the control law. This approach is especially attractive to controlling a large swarm of robots because the map can be shared by the swarm, hence avoiding the interactions with each individual robot.
We have investigated one situation where the map is specified as a scalar field distributed over the 2D plane. A robot can measure this scalar field through a noisy sensor. A swarm of such robots can share their local measurements to achieve at least several intelligent behaviors. Our previous works [1]- [6] have developed control laws and filtering algorithms that generate source seeking or level curve tracking behaviors. We have also shown that even when the map (scalar field) is unknown to the robots, these behaviors can be successfully enabled. Yet most of the existing results in the literature [7]- [14], including our previous works, rely on the assumption that the robots are supported by localization services. This assumption is needed because the map is given as a spatially distributed scalar field. To retrieve information from the map, robots need to provide their locations.
In this paper, we consider the situation where no localization service is available to the robots. The denial of localization makes it difficult for a robot to use the scalar map. But this case is practically useful because an increasing number of service robots are being deployed in environments where localization is difficult, such as underwater, underground, or polar areas. Our problem may be viewed as a simplified problem that connects to more practical applications.
We propose a level curve tracking strategy for robots without localization service. Our strategy is enabled by recent advancements in deep recurrent neural networks (RNN) [15]. In particular, we leverage the long short-term memory (LSTM) networks [16] to process training data collected from a given map or from an emulated environment when localization is available. As a deep neural network, the LSTM is able to memorize long term dependencies in sequential data. This memory can be trained to produce the correct sequence of data that can be retrieved by a robot without location information.
The key component of our proposed strategy is the con-version from a scalar field map to trained memory. For each given map, two LSTM networks are trained, one for learning the information dynamics along level curves of the scalar field, and the other for learning the uncertainty in the information states. The information states consist of the field value and gradient along level curve. This information is required by the level curve tracking control law for a robot to follow a level curve. The uncertainty is represented by the covariance matrix for the information states.
The trained memory will then be used to retrieve the information states when the robot is moving through the scalar field. While a robot is moving, its sensor generates a sequence of scalar field measurements, which can be provided to the trained memory in real-time to retrieve information about the field value and gradient. The key point here is that the robot does NOT provide its location to retrieve this information.
We have formulated the information retrieving process as an online filtering problem. The trained memory, such as an RNN, is used for the predictions of further information state sequence. Then a Kalman-like filter is designed to fuse the measurements from the robots with this prediction. This leads to an LSTM enabled Kalman filter (LSTM-KF) that is able to capture long term dependence in the data sequence. Previous works have leveraged LSTM-KF for modeling human motion data [17] as well as spatial-temporal data [18].
There is an increasing number of papers related to control systems where LSTM or other types of RNNs are leveraged. One typical reason to use RNNs is to model system dynamics that are too complex or completely unknown [19]- [23]. Another important idea is to use RNNs to generalize a predesigned controller to increase robustness against data that are not included in the training process [24]- [27]. The LSTM-KF [17], [18] is closely related to these approaches. This paper provides a new perspective on the application of LSTM to robot motion control without localization.
The problem for level curve tracking without localization service is formulated in Section II. Section III introduces level curve tracking control laws for a single robot and a multi-robot group. Section IV discusses the learning of information dynamics using LSTM. The LSTM-KF is introduced in Section V. Simulation results are illustrated in Section VI. Conclusions and future work follow in Section VII.
In this section, we formulate the problem of level curve tracking in scalar fields using one or multiple mobile robots without localization.
Consider a 2D scalar field described by a function z(r) : R 2 → R where r ∈ R 2 denotes the position. The field function z is assumed to be smooth, time-invariant, and bounded function, i.e., 0 ≤ z min ≤ z(r) ≤ z max < ∞.
Suppose we have mobile robots and each is located at r i,k where i = 1, • • • , N is the index of the robot and k is the index of time step t k . The robots are equipped with environmental sensors and are deployed into the field to track the level curve {r|z(r) = z d } where z d is a desired level curve value.
Assume that each robot takes a discrete measurement of the field at each time step t k at its current position r i,k , then the measurement is written as
where r i,k is the position of i-th robot at time t k and n i,k ∈ R is an independently and identically distributed Gaussian noise. Assumption II.1 Each robot does not know its current location r i,k . The velocity of each robot is described by
where u is a control law that relies on the measurement z(r i ) and gradient ∇z(r i ) of the field function.
The objectives of this paper are summarized as (1) Train two LSTM networks that can collectively provide estimates of field values and gradients of known scalar fields. ( 2) Design an LSTM-KF that fuses the real-time measurements with the estimates from the two LSTM networks and update the estimates. (3) Develop a level curve tracking controller to control robot(s) to detect and track a level curve in a 2D space based on the estimates from the LSTM-KF. Remark II.2 Known or estimated field information is often represented in terms of the spatial locations. Our problem is challenging as each robot lacks a localization capability as restricted by Assumption II.1. This problem is practically useful especially in harsh environments where localization is difficult, such as underwater, underground, or polar areas.
In this section, we introduce the level curve tracking control design for a single robot and multiple robots. The control law will produce robot trajectories that follow a desired level curve. Along the robot trajectories, the field value and gradient of the field will evolve over time, which can be modeled by the information dynamics.
Define a level curve {r|z(r) = z d } where z d ∈ R is a desired field value. Define
to be a unit length direction vector along the gradient direction ∇z perturbed by some error estimation e. Let the velocity of the robot be described by
where p is the noisy field measurement given by ( 1). The constants a, b are tuning parameters. Intuitively, the first term in (4) acts to stabilize the agent on desired level curve while the second term acts to drive the agent along that level curve.
Compared with single robot level curve tracking control (4), a formation term is added to the velocity control design ṙi = -a(p i -
where c is a tuning parameter. The formation term f i is defined as
where d ij is a desired separation distance between agents i and j.
Using the control law (5) and the fact that 1 N N i=1 f i = 0 for an undirected connectivity graph, then the dynamics of the center of the formation,
where p a = 1 N N i=1 p(r i ) is the average noisy measurement. This implies that the trajectory of the center of formation is tracking the level curve {r c |p a = z d }. Remark III.1 The formation (6) requires the knowledge of local displacements (r j -r i ) but does not require the knowledge of absolute positions. We choose this formation controller to simplify the analysis. In the future we may use other controllers that may depends only on distances or bearing angles [28].
When robots move along a level curve, the data collected by the robots satisfy the information dynamics. These data are used as training data for the LSTM. Due to the capability to capture long term trends in data, the trained LSTM is able to memorize the sequence of the information states along the level curves. Since no localization information is used during the training process, the trained memory can be retrieved by a sequence of robot measurements without location information.
The long short-term memory (LSTM) is often used to model long term dependencies in sequential data, which suits environmental monitoring applications.
The equations for a general LSTM block are as follows,
where σ(•) denotes sigmoid function and • denotes elementwise multiplication. x k is the input, h k is the hidden state, C k is the cell state. f k , i k and o k are outputs of forget gate layer, input gate layer, and output gate layer respectively. The weight matrices in (8) can be learned through a supervised training process that reduces mismatch between predicted output and true output. After the network has been trained, it is capable of modeling the dynamics between the input and the output. In particular, the LSTM is able to relate the prediction to a long history in the data sequence, which is difficult for state-space models.
Without using robot locations, we assume that the information dynamics of state s k = [z k , ∇z k ] can be written as,
where f s (•) is an unknown time-varying function and l is the length of time window that describes time dependency. LSTM networks are employed to learn the state dynamics f s (•) through sequential data s 1 , • • • , s K , in order to build an input-output map between previous l states s k-1 , • • • , s k- l and current state s k using LST M s as follows,
where l is the estimated length of time window. The uncertainty in the information states is captured by the covariance matrix P k . We employ another network LST M P to learn the sequence of covariance matrices as follows,
If a simulation program is available for the scalar field, the training data can be obtained through the simulation.
In the case when no such simulation is available, the training data can be collected using a group of robots that can be deployed to explore a portion of the unknown scalar field. Our previous work [29] has developed a cooperative Kalman filter to provide the estimates of the information states. Collection of training data might require localization, but we often do not need to collect training data everywhere in the workspace, hence the requirement on localization can be mild.
With the trained LSTM networks, mobile robots are able to retrieve the predicted field values and gradients from the LSTM models at their current locations without explicitly knowing their locations. The pre-trained LSTM networks produce fixed outputs corresponding to given inputs without considering the real-time measurements from robots. On the contrary, Kalman filter and its variants are capable of estimating system state iteratively every time a new measurement becomes available. However, the system dynamics is required to run a Kalman filter. To obtain more accurate state predictions by fusing the real-time measurements without knowing the system dynamics, in this section, we design an LSTM-KF to update the state predictions from the trained LSTM networks. We introduce the LSTM-KF filter design for both single-robot and multi-robot level curve tracking.
The measurement p k taken by a mobile robot at position r k at time t k can be modeled as
where z k is the field value at r k , n k ∼ N (0, R s k ) and C s k = [1 0 0]. Note that the robot has no location information, r k .
In the case of using multiple robots with relative position information, the LSTM-KF is applied to estimate the field value and gradient at formation center. The state is defined as s k = [z c,k , ∇z c,k ] , where z c,k is the field value and ∇z c,k is the gradient at formation center r c,k = 1
Then the measurement equation can be written as
where
Combining the LSTM predictions with the general structure of Kalman filter, the LSTM-KF can be written as
where s k(-) is the state prediction, P k(-) is the covariance matrix prediction, K k is the Kalman gain, s k(+) is the updated state estimation, and P k(+) is the updated covariance matrix estimation.
k for single robot and multi-robot level curve tracking, respectively.
After the two LSTM networks are trained, a mobile robot or a group of mobile robots without localization are employed to perform level tracking based on the estimated information from the LSTM-KF, using the control law ( 4) or (7). Remark V.1 The trained LSTM networks serve as "memory" for the robots, allowing robots to obtain gradient information without knowing their location. This is an important capability that is useful when localization service is denied in practice. Remark V.2 In the case of multi-robot level curve tracking, only relative positions among the robots are needed when implementing the LSTM-KF. Neither the location of the formation center or the absolute locations of each robot are needed to obtain the gradient information.
In this section, we present the simulation results that demonstrate level curve tracking behaviors for both a single robot and a multi-robot team enabled by the LSTM-KF and the feedback control law using MATLAB.
The training data is collected by simulating four mobile robots (considered as points) deployed in a simulated field to perform cooperative exploration along a desired level curve
x 2 k + y 2 k = 6 at t k . Two single-hidden-layer LSTM networks each with 100 hidden units are used for state and covariance prediction, respectively. The estimated time window length l is 4 time steps. The two LSTM networks are trained using 9996 historical data points collected by the robots.
After the two LSTMs have been trained, a single robot without localization is employed to track desired level curve using the LSTM-KF and gradient-based control law. At each time step, the robot takes a new measurement, which is used by the LSTM-KF to predict the field gradient needed in (4).
We now demonstrate that the single robot can track timevarying level curves. Note that data for time-varying level curves is not contained in the training process. In this case, the desired level curve is x 2 k + y 2 k -sin(k/500) = 6 at t k . The trajectory of the robot in 2D plane is shown in Fig. 1, where black curve represents robot trajectory and red dots represent robot positions. As shown in Fig. 2, the level curve value z along trajectory is very close to the desired value z d with mean squared error 0.0012, which demonstrates that the robot is capable of tracking the time-varying level curve.
In the simulation for multi-robot level curve tracking, we want to track a time-invariant level curve, x 2 k + y 2 k = 6 at time t k using three robots only with relative position information among robots. At each time step, new measurements will be taken by the three robots and are used to retrieve the gradient information at the formation center. The velocity control for each robot will be updated according to (5) using the predicted gradient and relative positions. As shown in Fig. 3, the black curve is the trajectory of the formation center and the colored dots represent the positions of the three robots. The formation center is capable of tracking the desired level curve with mean squared error of 0.0272. The normalized x-component and y-component of gradient predicted by LSTM-KF of the three robots at the formation center are shown in Fig. 4. They are almost identical to that of the true gradient. This means that the LSTM-KF is capable to make accurate gradient prediction based on previous state information and measurement information.
LSTM can be leveraged to memorize a long training sequence. This property can be exploited to model the information states along robot trajectories. The memory can be retrieved through the sensor measurements collected by the robots. We discover that the trained memory can be leveraged by feedback control laws to achieve the level curve tracking behavior when no localization is available to the robot. In addition, the LSTM can be combined with the Kalman filter. Future work includes the exploration of other types of behaviors such as source seeking and feature tracking without localization, as well as generalization of the LSTM-KF to other situations.
978-1-7281-9888-0/20/$31.00 ©2020 IEEE
Authorized licensed use limited to: Georgia Institute of Technology. Downloaded on November 02,2020 at 19:26:04 UTC from IEEE Xplore. Restrictions apply.