More Like this

1. Predicting Geothermal Favorability in the Western United States by Using Machine Learning: Addressing Challenges and Developing Solutions

   Mordensky, Stanley P.; Lipor, John J.; DeAngelo, J.; Burns, Erick R.; Lindsey, Cary R. (February 2022, Forty Seventh Workshop on Geothermal Reservoir Engineering)

   Previous moderate- and high-temperature geothermal resource assessments of the western United States utilized weight-of-evidence and logistic regression methodstoestimateresourcefavorability,buttheseanalyses relied uponsomeexpert decisions.Whileexpert decisions can add confidence to aspects of the modeling process by ensuring only reasonable models are employed, expert decisions also introduce human bias into assessments. This bias presents a source of error that may affect the performance of the models and resulting resource estimates. Our study aims to reduce expert input through robust data-driven analyses and better-suited data science techniques, with the goals of saving time, reducing bias, and improving predictive ability. We present six favorability maps for geothermal resources in the western United States created using two strategies applied to three modern machine learning algorithms (logistic regression, support- vector machines, and XGBoost). To provide a direct comparison to previous assessments, we use the same input data as the 2008 U.S. Geological Survey (USGS) conventional moderate- to high-temperature geothermal resource assessment. The six new favorability maps required far less expert decision-making, but broadly agree with the previous assessment. Despite the fact that the 2008 assessment results employed linear methods, the non-linear machine learning algorithms (i.e., support-vector machines and XGBoost) produced greater agreement with the previous assessment than the linear machine learning algorithm (i.e., logistic regression). It is not surprising that geothermal systems depend on non-linear combinations of features, and we postulate that the expert decisions during the 2008 assessment accounted for system non-linearities. Substantial challenges to applying machine learning algorithms to predict geothermal resource favorability include severe class imbalance (i.e., there are very few known geothermal systems compared to the large area considered), and while there are known geothermal systems (i.e., positive labels), all other sites have an unknown status (i.e., they are unlabeled), instead of receiving a negative label (i.e., the known/proven absence of a geothermal resource). We address both challenges through a custom undersampling strategy that can be used with any algorithm and then evaluated using F1 scores. 

   more » « less
2. Don’t Let Negatives Hold You Back: Accounting for Underlying Physics and Natural Distributions of Hydrothermal Systems When Selecting Negative Training Sites Leads to Better Machine Learning Predictions

   Caraccioli, Pascal D; Mordensky, Stanley Paul; Lindsey, Cary R; DeAngelo, Jacob; Burns, Erick; Lipor, John (October 2023, 2023 Geothermal Rising Conference)

   Selecting negative training sites is an important challenge to resolve when utilizing machine learning (ML) for predicting hydrothermal resource favorability because ideal models would discriminate between hydrothermal systems (positives) and all types of locations without hydrothermal systems (negatives). The Nevada Machine Learning project (NVML) fit an artificial neural network to identify areas favorable for hydrothermal systems by selecting 62 negative sites where the research team had confidence that no hydrothermal resource exists. Herein, we compare the implications of the expert selection of negatives (i.e., the NVML strategy) with a random sample strategy, where it is assumed that areas outside the favorable structural ellipses defined by NVML are negative. Because hydrothermal systems are sparse, it is highly probable that, in the absence of a favorable geological structure, hydrothermal favorability is low. We compare three training strategies: 1) the positive and negative labeled examples from NVML; 2) the positive examples from NVML with randomly selected negatives in equal frequency as NVML; and 3) the positive examples from NVML with randomly selected negatives reflecting the expected natural distribution of hydrothermal systems relative to the total area. We apply these training strategies to the NVML feature data (input data) using two ML algorithms (XGBoost and logistic regression) to create six favorability maps for hydrothermal resources. When accounting for the expected natural distribution of hydrothermal systems, we find that XGBoost performs better than the NVML neural network and its negatives. Model validation was less reliable using F1 scores, a common performance metric, than comparing probability estimates at known positives, likely because of the extreme natural class imbalance and the lack of negatively labeled sites. This work demonstrates that expert selection of negatives for training in NVML likely imparted modeling bias. Accounting for the sparsity of hydrothermal systems and all the types of locations without hydrothermal systems allows us to create better models for predicting hydrothermal resource favorability. 

   more » « less
3. Cursed? Why one does not simply add new data sets to supervised geothermal machine learning models

   Mordensky, Stanley Paul; Burns, Erick; Lipor, John; DeAngelo, Jacob (October 2023, 2023 Geothermal Rising Conference)

   Recent advances in machine learning (ML) identifying areas favorable to hydrothermal systems indicate that the resolution of feature data remains a subject of necessary improvement before ML can reliably produce better models. Herein, we consider the value of adding new features or replacing other, low-value features with new input features in existing ML pipelines. Our previous work identified stress and seismicity as having less value than the other feature types (i.e., heat flow, distance to faults, and distance to magmatic activity) for the 2008 USGS hydrothermal energy assessment; hence, a fundamental question regards if the addition of new but partially correlated features will improve resulting models for hydrothermal favorability. Therefore, we add new maps for shear strain rate and dilation strain rate to fit logistic regression and XGBoost models, resulting in new 7-feature models that are compared to the old 5-feature models. Because these new features share a degree of correlation with the original relatively uninformative stress and seismicity features, we also consider replacement of the two lower-value features with the two new features, creating new 5-feature models. Adding the new features improves the predictive skill of the new 7-feature model over that of the old 5-feature model; albeit, that improvement is not statistically significant because the new features are correlated with the old features and, consequently, the new features do not present considerable new information. However, the new 5-feature XGBoost model has a statistically significant increase in predictive skill for known positives over the old 5-feature model at p = 0.06. This improved performance is due to the lower-dimensional feature space of the former than that of the latter. In higher-dimensional feature space, relationships between features and the presence or absence of hydrothermal systems are harder to discern (i.e., the 7-feature model likely suffers from the “curse of dimensionality”). 

   more » « less
4. Automated Analysis of the US Drought Monitor Maps With Machine Learning and Multiple Drought Indicators

   https://doi.org/10.3389/fdata.2021.750536  

   Hatami Bahman Beiglou, Pouyan; Luo, Lifeng; Tan, Pang-Ning; Pei, Lisi (October 2021, Frontiers in Big Data)

   The US Drought Monitor (USDM) is a hallmark in real time drought monitoring and assessment as it was developed by multiple agencies to provide an accurate and timely assessment of drought conditions in the US on a weekly basis. The map is built based on multiple physical indicators as well as reported observations from local contributors before human analysts combine the information and produce the drought map using their best judgement. Since human subjectivity is included in the production of the USDM maps, it is not an entirely clear quantitative procedure for other entities to reproduce the maps. In this study, we developed a framework to automatically generate the maps through a machine learning approach by predicting the drought categories across the domain of study. A persistence model served as the baseline model for comparison in the framework. Three machine learning algorithms, logistic regression, random forests, and support vector machines, with four different groups of input data, which formed an overall of 12 different configurations, were used for the prediction of drought categories. Finally, all the configurations were evaluated against the baseline model to select the best performing option. The results showed that our proposed framework could reproduce the drought maps to a near-perfect level with the support vector machines algorithm and the group 4 data. The rest of the findings of this study can be highlighted as: 1) employing the past week drought data as a predictor in the models played an important role in achieving high prediction scores, 2) the nonlinear models, random forest, and support vector machines had a better overall performance compared to the logistic regression models, and 3) with borrowing the neighboring grid cells information, we could compensate the lack of training data in the grid cells with insufficient historical USDM data particularly for extreme and exceptional drought conditions. 

   more » « less
5. Objective estimation of m-CTSIB balance test scores using wearable sensors and machine learning

   https://doi.org/10.3389/fdgth.2024.1366176  

   Nassajpour, Marjan; Shuqair, Mustafa; Rosenfeld, Amie; Tolea, Magdalena I; Galvin, James E; Ghoraani, Behnaz (April 2024, Frontiers in Digital Health)

   Accurate balance assessment is important in healthcare for identifying and managing conditions affecting stability and coordination. It plays a key role in preventing falls, understanding movement disorders, and designing appropriate therapeutic interventions across various age groups and medical conditions. However, traditional balance assessment methods often suffer from subjectivity, lack of comprehensive balance assessments and remote assessment capabilities, and reliance on specialized equipment and expert analysis. In response to these challenges, our study introduces an innovative approach for estimating scores on the Modified Clinical Test of Sensory Interaction on Balance (m-CTSIB). Utilizing wearable sensors and advanced machine learning algorithms, we offer an objective, accessible, and efficient method for balance assessment. We collected comprehensive movement data from 34 participants under four different sensory conditions using an array of inertial measurement unit (IMU) sensors coupled with a specialized system to evaluate ground truth m-CTSIB balance scores for our analysis. This data was then preprocessed, and an extensive array of features was extracted for analysis. To estimate the m-CTSIB scores, we applied Multiple Linear Regression (MLR), Support Vector Regression (SVR), and XGBOOST algorithms. Our subject-wise Leave-One-Out and 5-Fold cross-validation analysis demonstrated high accuracy and a strong correlation with ground truth balance scores, validating the effectiveness and reliability of our approach. Key insights were gained regarding the significance of specific movements, feature selection, and sensor placement in balance estimation. Notably, the XGBOOST model, utilizing the lumbar sensor data, achieved outstanding results in both methods, with Leave-One-Out cross-validation showing a correlation of 0.96 and a Mean Absolute Error (MAE) of 0.23 and 5-fold cross-validation showing comparable results with a correlation of 0.92 and an MAE of 0.23, confirming the model’s consistent performance. This finding underlines the potential of our method to revolutionize balance assessment practices, particularly in settings where traditional methods are impractical or inaccessible. 

   more » « less