Search for: All records

We propose a geometric reinforcement learning algorithm for real-time path planning for mobile sensor networks (MSNs) in the problem of reconstructing a spatial-temporal varying field described by the advection-diffusion partial differential equation. A Luenberger state estimator is provided to reconstruct the concentration field, which uses the collected measurements from a MSN along its trajectory. Since the path of the MSN is critical in reconstructing the field, a novel geometric reinforcement learning (GRL) algorithm is developed for real-time path planning. The basic idea of GRL is to divide the whole area into a series of lattice to employ a specific time-varying reward matrix, which contains the information of the length of the path and the mapping error. Thus, the proposed GRL can balance the performance of the field reconstruction and the efficiency of the path. By updating the reward matrix, the real-time path planning problem can be converted to the shortest path problem in a weighted graph, which can be solved efficiently using dynamic programming. The convergence of calculating the reward matrix is theoretically proven. Simulation results serve to demonstrate the effectiveness and feasibility of the proposed GRL for an MSN. 

We investigate the problem of simultaneous parameter identification and mapping of a spatially distributed field using a mobile sensor network. We first develop a parametrized model that represents the spatially distributed field. Based on the model, a recursive least squares algorithm is developed to achieve online parameter identification. Next, we design a global state observer, which uses the estimated parameters, together with data collected by the mobile sensor network, to real-timely reconstruct the whole spatial-temporal varying field. Since the performance of the parameter identification and map reconstruction algorithms depends on the trajectories of the mobile sensors, we further develop a Lyapunov redesign based online trajectory planning algorithm for the mobile sensor network so that the mobile sensors can use local real-time information to guide them to move along information-rich paths that can improve the performance of the parameter identification and map construction. Lastly, a cooperative filtering scheme is developed to provide the state estimates of the spatially distributed field, which enables the recursive least squares method. To test the proposed algorithms in realistic scenarios, we first build a CO2 diffusion field in a lab and construct a sensor network to measure the field concentration over time. We then validate the algorithms in the reconstructed CO2 field in simulation. Simulation results demonstrate the efficiency of the proposed method.