Search for: All records

Abstract This paper develops a hybrid deep reinforcement learning approach to manage an insurance portfolio for diffusion models. To address the model uncertainty, we adopt the recently developed modelling of exploration and exploitation strategies in a continuous-time decision-making process with reinforcement learning. We consider an insurance portfolio management problem in which an entropy-regularized reward function and corresponding relaxed stochastic controls are formulated. To obtain the optimal relaxed stochastic controls, we develop a Markov chain approximation and stochastic approximation-based iterative deep reinforcement learning algorithm where the probability distribution of the optimal stochastic controls is approximated by neural networks. In our hybrid algorithm, both Markov chain approximation and stochastic approximation are adopted in the learning processes. The idea of using the Markov chain approximation method to find initial guesses is proposed. A stochastic approximation is adopted to estimate the parameters of neural networks. Convergence analysis of the algorithm is presented. Numerical examples are provided to illustrate the performance of the algorithm. 

A swimming microorganism stirs the surrounding fluid, creating a flow field that governs not only its locomotion and nutrient uptake, but also its interactions with other microorganisms and the environment. Despite its fundamental importance, capturing this flow field and unraveling its biological implications remains a challenge. Here, we report direct, time-resolved measurements of the three-dimensional (3D) flow field generated by a single, free-swimming microalga, , a model organism for microbial locomotion and flagellar dynamics. Supported by hydrodynamic modeling and simulations, our measurements resolve how established two-dimensional (2D) flow features such as in-plane vortices and the stagnation point emerge from and shape the full algal flow in 3D. Moreover, we reveal unexpected low-Reynolds-number flow phenomena including micron-sized vortex rings and periodically recurring translating vortices and uncover topological changes in the underlying flow structure associated with the puller-to-pusher transition of an alga. Biologically, access to the 3D flow field enables rigorous quantification of the alga’s energy expenditure, as well as its swimming and feeding efficiency, improving the precision of these physiological metrics. Taken together, our study demonstrates rich vortex dynamics in inertialess flows and shows their influence on microbial motility. The work also introduces an experimental method for mapping the fluid environment sculpted by beating flagella. 

We investigate the dynamics of a pair of rigid rotating helices in a viscous fluid, as a model for bacterial flagellar bundle and a prototype of microfluidic pumps. Combining experiments with hydrodynamic modelling, we examine how spacing and phase difference between the two helices affect their torque, flow field and fluid transport capacity at low Reynolds numbers. Hydrodynamic coupling reduces the torque when the helices rotate in phase at constant angular speed, but increases the torque when they rotate out of phase. We identify a critical phase difference, at which the hydrodynamic coupling vanishes despite the close spacing between the helices. A simple model, based on the flow characteristics and positioning of a single helix, is constructed, which quantitatively predicts the torque of the helical pair in both unbounded and confined systems. Finally, we show the influence of spacing and phase difference on the axial flux and the pump efficiency of the helices. Our findings shed light on the function of bacterial flagella and provide design principles for efficient low-Reynolds-number pumps.