skip to main content
US FlagAn official website of the United States government
dot gov icon
Official websites use .gov
A .gov website belongs to an official government organization in the United States.
https lock icon
Secure .gov websites use HTTPS
A lock ( lock ) or https:// means you've safely connected to the .gov website. Share sensitive information only on official, secure websites.


Title: Deep reinforcement learning for personalized treatment recommendation
In precision medicine, the ultimate goal is to recommend the most effective treatment to an individual patient based on patient‐specific molecular and clinical profiles, possibly high‐dimensional. To advance cancer treatment, large‐scale screenings of cancer cell lines against chemical compounds have been performed to help better understand the relationship between genomic features and drug response; existing machine learning approaches use exclusively supervised learning, including penalized regression and recommender systems. However, it would be more efficient to apply reinforcement learning to sequentially learn as data accrue, including selecting the most promising therapy for a patient given individual molecular and clinical features and then collecting and learning from the corresponding data. In this article, we propose a novel personalized ranking system called Proximal Policy Optimization Ranking (PPORank), which ranks the drugs based on their predicted effects per cell line (or patient) in the framework of deep reinforcement learning (DRL). Modeled as a Markov decision process, the proposed method learns to recommend the most suitable drugs sequentially and continuously over time. As a proof‐of‐concept, we conduct experiments on two large‐scale cancer cell line data sets in addition to simulated data. The results demonstrate that the proposed DRL‐based PPORank outperforms the state‐of‐the‐art competitors based on supervised learning. Taken together, we conclude that novel methods in the framework of DRL have great potential for precision medicine and should be further studied.  more » « less
Award ID(s):
1952539
PAR ID:
10370578
Author(s) / Creator(s):
 ;  ;  
Publisher / Repository:
Wiley Blackwell (John Wiley & Sons)
Date Published:
Journal Name:
Statistics in Medicine
Volume:
41
Issue:
20
ISSN:
0277-6715
Page Range / eLocation ID:
p. 4034-4056
Format(s):
Medium: X
Sponsoring Org:
National Science Foundation
More Like this
  1. Abstract Functional precision medicine offers a promising complement to genomics-based cancer therapy guidance by testing drug efficacy directly on a patient’s tumor cells. Here, we describe a workflow that utilizes single-cell mass measurements with inline brightfield imaging and machine-learning based image classification to broaden the clinical utility of such functional testing for cancer. Using these image-curated mass measurements, we characterize mass response signals for 60 different drugs with various mechanisms of action across twelve different cell types, demonstrating an improved ability to detect response for several slow acting drugs as compared with standard cell viability assays. Furthermore, we use this workflow to assess drug responses for various primary tumor specimen formats including blood, bone marrow, fine needle aspirates (FNA), and malignant fluids, all with reports generated within two days and with results consistent with patient clinical responses. The combination of high-resolution measurement, broad drug and malignancy applicability, and rapid return of results offered by this workflow suggests that it is well-suited to performing clinically relevant functional assessment of cancer drug response. 
    more » « less
  2. Despite significant advances in cancer diagnosis and treatment, osteosarcoma (OSA), an aggressive primary bone tumor, has eluded attempts at improving patient survival for many decades. The difficulty in managing OSA lies in its extreme genetic complexity, drug resistance, and heterogeneity, making it improbable that a single-target treatment would be beneficial for the majority of affected individuals. Precision medicine seeks to fill this gap by addressing the intra- and inter-tumoral heterogeneity to improve patient outcome and survival. The characterization of differentially expressed genes (DEGs) unique to the tumor provides insight into the phenotype and can be useful for informing appropriate therapies as well as the development of novel treatments. Traditional DEG analysis combines patient data to derive statistically inferred genes that are dysregulated in the group; however, the results from this approach are not necessarily consistent across individual patients, thus contradicting the basis of precision medicine. Spontaneously occurring OSA in the dog shares remarkably similar clinical, histological, and molecular characteristics to the human disease and therefore serves as an excellent model. In this study, we use transcriptomic sequencing of RNA isolated from primary OSA tumor and patient-matched normal bone from seven dogs prior to chemotherapy to identify DEGs in the group. We then evaluate the universality of these changes in transcript levels across patients to identify DEGs at the individual level. These results can be useful for reframing our perspective of transcriptomic analysis from a precision medicine perspective by identifying variations in DEGs among individuals. 
    more » « less
  3. As the most lethal major cancer, pancreatic cancer is a global healthcare challenge. Personalized medicine utilizing cutting-edge multi-omics data holds potential for major breakthroughs in tackling this critical problem. Radiomics and deep learning, two trendy quantitative imaging methods that take advantage of data science and modern medical imaging, have shown increasing promise in advancing the precision management of pancreatic cancer via diagnosing of precursor diseases, early detection, accurate diagnosis, and treatment personalization and optimization. Radiomics employs manually-crafted features, while deep learning applies computer-generated automatic features. These two methods aim to mine hidden information in medical images that is missed by conventional radiology and gain insights by systematically comparing the quantitative image information across different patients in order to characterize unique imaging phenotypes. Both methods have been studied and applied in various pancreatic cancer clinical applications. In this review, we begin with an introduction to the clinical problems and the technology. After providing technical overviews of the two methods, this review focuses on the current progress of clinical applications in precancerous lesion diagnosis, pancreatic cancer detection and diagnosis, prognosis prediction, treatment stratification, and radiogenomics. The limitations of current studies and methods are discussed, along with future directions. With better standardization and optimization of the workflow from image acquisition to analysis and with larger and especially prospective high-quality datasets, radiomics and deep learning methods could show real hope in the battle against pancreatic cancer through big data-based high-precision personalization. 
    more » « less
  4. There is tremendous interest in precision medicine as a means to improve patient out- comes by tailoring treatment to individual characteristics. An individualized treatment rule formalizes precision medicine as a map from patient information to a recommended treatment. A treatment rule is defined to be optimal if it maximizes the mean of a scalar outcome in a population of interest, e.g., symptom reduction. However, clinical and intervention scientists often seek to balance multiple and possibly competing outcomes, e.g., symptom reduction and the risk of an adverse event. One approach to precision medicine in this setting is to elicit a composite outcome which balances all competing outcomes; unfortunately, eliciting a composite outcome directly from patients is difficult without a high-quality instrument, and an expert-derived composite outcome may not account for heterogeneity in patient preferences. We propose a new paradigm for the study of precision medicine using observational data that relies solely on the assumption that clinicians are approximately (i.e., imperfectly) making decisions to maximize individual patient utility. Estimated composite outcomes are subsequently used to construct an estimator of an individualized treatment rule which maximizes the mean of patient-specific composite out- comes. The estimated composite outcomes and estimated optimal individualized treatment rule provide new insights into patient preference heterogeneity, clinician behavior, and the value of precision medicine in a given domain. We derive inference procedures for the pro- posed estimators under mild conditions and demonstrate their finite sample performance through a suite of simulation experiments and an illustrative application to data from a study of bipolar depression. 
    more » « less
  5. null (Ed.)
    Abstract The ability to predict the efficacy of cancer treatments is a longstanding goal of precision medicine that requires improved understanding of molecular interactions with drugs and the discovery of biomarkers of drug response. Identifying genes whose expression influences drug sensitivity can help address both of these needs, elucidating the molecular pathways involved in drug efficacy and providing potential ways to predict new patients’ response to available therapies. In this study, we integrated cancer type, drug treatment, and survival data with RNA-seq gene expression data from The Cancer Genome Atlas to identify genes and gene sets whose expression levels in patient tumor biopsies are associated with drug-specific patient survival using a log-rank test comparing survival of patients with low vs. high expression for each gene. This analysis was successful in identifying thousands of such gene–drug relationships across 20 drugs in 14 cancers, several of which have been previously implicated in the respective drug’s efficacy. We then clustered significant genes based on their expression patterns across patients and defined gene sets that are more robust predictors of patient outcome, many of which were significantly enriched for target genes of one or more transcription factors, indicating several upstream regulatory mechanisms that may be involved in drug efficacy. We identified a large number of genes and gene sets that were potentially useful as transcript-level biomarkers for predicting drug-specific patient survival outcome. Our gene sets were robust predictors of drug-specific survival and our results included both novel and previously reported findings, suggesting that the drug-specific survival marker genes reported herein warrant further investigation for insights into drug mechanisms and for validation as biomarkers to aid cancer therapy decisions. 
    more » « less