skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Optimal Control of Gene Regulatory Networks with Unknown Cost Function
Most of the existing methodologies for control of Gene Regulatory Networks (GRNs) assume that the immediate cost function at each state and time point is fully known. In this paper, we introduce an optimal control strategy for control of GRNs with unknown or partially-known immediate cost function. Toward this, we adopt a partially-observed Boolean dynamical system (POBDS) model for the GRN and propose an Inverse Reinforcement Learning (IRL) methodology for quantifying the imperfect behavior of experts, obtained via prior biological knowledge or experimental data. The constructed cost function then is used in finding the optimal infinite-horizon control strategy for the POBDS. The application of the proposed method using a single sequence of experimental data is investigated through numerical experiments using a melanoma gene regulatory network.  more » « less
Award ID(s):
1718924
PAR ID:
10110101
Author(s) / Creator(s):
;
Date Published:
Journal Name:
2018 Annual American Control Conference (ACC)
Page Range / eLocation ID:
3939 to 3944
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Control of gene regulatory networks (GRNs) to shift gene expression from undesirable states to desirable ones has received much attention in recent years. Most of the existing methods assume that the cost of intervention at each state and time point, referred to as the immediate cost function, is fully known. In this paper, we employ the Partially-Observed Boolean Dynamical System (POBDS) signal model for a time sequence of noisy expression measurement from a Boolean GRN and develop a Bayesian Inverse Reinforcement Learning (BIRL) approach to address the realistic case in which the only available knowledge regarding the immediate cost function is provided by the sequence of measurements and interventions recorded in an experimental setting by an expert. The Boolean Kalman Smoother (BKS) algorithm is used for optimally mapping the available gene-expression data into a sequence of Boolean states, and then the BIRL method is efficiently combined with the Q-learning algorithm for quantification of the immediate cost function. The performance of the proposed methodology is investigated by applying a state-feedback controller to two GRN models: a melanoma WNT5A Boolean network and a p53-MDM2 negative feedback loop Boolean network, when the cost of the undesirable states, and thus the identity of the undesirable genes, is learned using the proposed methodology. 
    more » « less
  2. Background: Single-cell gene expression measurements offer opportunities in deriving mechanistic understanding of complex diseases, including cancer. However, due to the complex regulatory machinery of the cell, gene regulatory network (GRN) model inference based on such data still manifests significant uncertainty. Results:The goal of this paper is to develop optimal classification of single-cell trajectories accounting for potential model uncertainty. Partially-observed Boolean dynamical systems (POBDS) are used for modeling gene regulatory networks observed through noisy gene-expression data. We derive the exact optimal Bayesian classifier (OBC) for binary classification of single-cell trajectories. The application of the OBC becomes impractical for large GRNs, due to computational and memory requirements. To address this, we introduce a particle-based single-cell classification method that is highly scalable for large GRNs with much lower complexity than the optimal solution. Conclusion:The performance of the proposed particle-based method is demonstrated through numerical experiments using a POBDS model of the well-known T-cell large granular lymphocyte (T-LGL) leukemia network with noisy time-series gene-expression 
    more » « less
  3. null (Ed.)
    We present PALLAS, a practical method for gene regulatory network (GRN) inference from time series data, which employs penalized maximum likelihood and particle swarms for optimization. PALLAS is based on the Partially-Observable Boolean Dynamical System (POBDS) model and thus does not require ad-hoc binarization of the data. The penalty in the likelihood is a LASSO regularization term, which encourages the resulting network to be sparse. PALLAS is able to scale to networks of realistic size under no prior knowledge, by virtue of a novel continuous-discrete Fish School Search particle swarm algorithm for efficient simultaneous maximization of the penalized likelihood over the discrete space of networks and the continuous space of observational parameters. The performance of PALLAS is demonstrated by a comprehensive set of experiments using synthetic data generated from real and artificial networks, as well as real time series microarray and RNA-seq data, where it is compared to several other well-known methods for gene regulatory network inference. The results show that PALLAS can infer GRNs more accurately than other methods, while being capable of working directly on gene expression data, without need of ad-hoc binarization. PALLAS is a fully-fledged program, written in python, and available on GitHub (https://github.com/yukuntan92/PALLAS). 
    more » « less
  4. Partially-observed Boolean dynamical systems (POBDS) are large and complex dynamical systems capable of being monitored through various sensors. However, time, storage, and economical constraints may impede the use of all sensors for estimation purposes. Thus, developing a procedure for selecting a subset of sensors is essential. The optimal minimum mean-square error (MMSE) POBDS state estimator is the Boolean Kalman Filter (BKF) and Smoother (BKS). Naturally, the performance of these estimators strongly depends on the choice of sensors. Given a finite subsets of sensors, for a POBDS with a finite observation space, we introduce the optimal procedure to select the best subset which leads to the smallest expected mean-square error (MSE) of the BKF over a finite horizon. The performance of the proposed sensor selection methodology is demonstrated by numerical experiments with a p53-MDM2 negative-feedback loop gene regulatory network observed through Bernoulli noise. 
    more » « less
  5. We propose a new algorithm for inference of gene regulatory networks (GRN) from noisy gene expression data based on maximum-likelihood (ML) adaptive filtering and the discrete fish school search algorithm (DFSS). The approach is based on the general partially-observed Boolean dynamical system (POBDS) model, and as such can be used for simultaneous state and parameter estimation for any Boolean dynamical system observed in noise. The proposed DFSS-ML-BKF algorithm combines the ML adaptive Boolean Kalman Filter (ML-BKF) with DFSS, a version of the Fish School Search algorithm tailored for discrete parameter spaces. Results based on synthetic gene expression time-series data using the well-known p53-MDM2 negative-feedback loop GRN demonstrate that DFSS-ML-BKF can infer the network topology accurately and efficiently. 
    more » « less