Search for: All records

Current studies that apply reinforcement learning (RL) to dynamic spectrum access (DSA) problems in wireless communications systems are mainly focusing on model-free RL. However, in practice model-free RL requires large number of samples to achieve good performance making it impractical in real time applications such as DSA. Combining model-free and model-based RL can potentially reduce the sample complexity while achieving similar level of performance as model-free RL as long as the learned model is accurate enough. However, in complex environment the learned model is never perfect. In this paper we combine model-free and model-based reinforcement learning, introduce an algorithm that can work with an imperfectly learned model to accelerate the model-free reinforcement learning. Results show our algorithm achieves higher sample efficiency than standard model-free RL algorithm and Dyna algorithm (a standard algorithm that integrating model-based and model-free RL) with much lower computation complexity than the Dyna algorithm. For the extreme case where the learned model is highly inaccurate, the Dyna algorithm performs even worse than the model-free RL algorithm while our algorithm can still outperform the model-free RL algorithm. 

Mobile edge and vehicle-based depth sending and real-time point cloud communication is an essential subtask enabling autonomous driving. In this paper, we propose a framework for point cloud multicast in VANETs using vehicle to infrastructure (V2I) communication. We employ a scalable Binary Tree embedded Quad Tree (BTQT) point cloud source encoder with bitrate elasticity to match with an adaptive random network coding (ARNC) to multicast different layers to the vehicles. The scalability of our BTQT encoded point cloud provides a trade-off in the received voxel size/quality vs channel condition whereas the ARNC helps maximize the throughput under a hard delay constraint. The solution is tested with the outdoor 3D point cloud dataset from MERL for autonomous driving. The users with good channel conditions receive a near lossless point cloud whereas users with bad channel conditions are still able to receive at least the base layer point cloud.