research contributes to the understanding of how these models can improve solar power forecasting, thereby benefiting solar energy operators and utility companies in making informed decisions about adopting ML-based solar power forecasting models [8]. Furthermore, the research seeks to enhance the theoretical discussion in the field of ML-based solar power forecasting by providing an in-depth analysis of the various ML algorithms used, the types of data sources employed, and the measures employed to assess their performance. By addressing this debate, the study aims to fill a knowledge gap and contribute to the ongoing advancements in accurate solar power forecasting techniques.

Recently, there have been numerous research studies in machine learning (ML)-based solar power forecasting to evaluate various ML algorithms. In this literature review, an overview of ML-based solar power forecasting is provided, including the different ML algorithms used, the types of data sources employed, and the measures employed to show how well these algorithms performed. In Kim et al. [9], the authors used various time-series techniques, such as deep learning and machine learning algorithms, to forecast the generation of PV power and promptly spot equipment and panel flaws. AI models were created using the South Korean data that was gathered between January 2017 and June 2021. Based on the findings, it was determined that the LSTM model had the greatest level of accuracy for predicting the hourly PV power generation in this 1.5 MW PV system. Elsaraiti & Merabet [10] suggested a deep learning-based approach for short-term PV power generation forecasting. Various performance criteria are used to analyze and contrast the Long Short-Term Memory (LSTM) algorithm and the Multi-Layer Perceptron (MLP) network. The outcomes demonstrate that the LSTM network works better than the MLP network and offers trustworthy data for the effective operation of PV power facilities. Deep learning and energy efficiency work well together to advance sustainability in the electrical industry.

In Li et al.'s [11] study, the author proposes a novel hybrid model for short-term PV energy prediction that combines WPD and LSTM networks. The model was validated using an actual solar system and outperformed other models in different seasons and meteorological conditions. The proposed approach may enhance the operating efficiency of regional power systems. Cabezón et al. [12] explore the implementation of ML techniques for short-term power forecasting in photovoltaic production plants. The study concludes that the tree-based XGB model provided the most accurate results in predicting the next hour's power demand. Bajpai & Duchon [13] present a hybrid approach using machine learning techniques for forecasting solar power generation by PV cells. According to the study, the model based on random forests outperformed other models, and it also suggested a unique method by merging several models according to certain weather circumstances. However, the hybrid approach requires a longer time to train the models. An RNN variant known as a gated recurrent unit (GRU) and LSTM have been employed by Sorkun et al. [14] to assess the effectiveness of the suggested model in terms of accurate solar irradiance prediction. Results show that the GRU and LSTM are more effective in forecasting time-series irradiance than a straightforward RNN.

Miraftabzadeh et al. [15] propose a framework that uses transfer learning to forecast the day ahead of energy production for recently constructed solar power stations. The framework involves training four predictive models based on different network architectures with a dataset, which are then transferred to the second phase to be retrained with the recently constructed PV dataset. The findings demonstrate that the transferred model outperforms other models and has the highest precision. Polasek & Čadík [16] propose a deep-learning (DL) model that predicts solar power plant production by combining elements of UNet with residual aggregation modules, using weather forecasts for training and augmentation. The dataset includes data for power stations throughout the year. By accurately assessing prediction uncertainty, the proposed architecture outperforms the base model by an extra 28.27%. Also, transfer learning is shown to be viable, enabling the provision of forecasts with just a few days ahead. In Mystakidis et al. [17], the author investigates the use of ML and DL algorithms for time-series data for energy generation forecasting. The study employs various metrics to evaluate the models' predicting capabilities and concludes that an ensemble method that integrates multiple ML and DL algorithms is the most accurate approach. The dynamic weighted average ensemble model outperformed all other standalone models, including Random Forest, LSTM, and XGBoost.

The main contribution of this paper is the successful utilization of machine learning techniques to accurately predict solar photovoltaic (PV) power generation in Lubbock, Texas. The study achieved a low Mean Squared Error (MSE) and high R² values, indicating the effectiveness of the RFR and LSTM models in capturing complex patterns and relationships in solar power generation data. These findings have practical implications for solar PV investors, providing valuable insights and aiding in the improvement of planning and decision-making processes for producing solar energy. In order to provide a clear and organized analysis, the remainder of this paper is structured into four main sections. Section 2 outlines the methodology and the processing of the data source utilized in this study, while Section 3 delves into a comprehensive analysis of the various machine learning models employed. Section 4 showcases the simulation and results generated from the analysis using Python. Finally, the paper concludes with a detailed discussion in Section 5, highlighting the final thoughts, implications, and future prospects of this study.

The required processing steps, as shown in Figure 1, must be implemented to the dataset in order to estimate solar power with ML/DL models.

Figure 1 summarizes the key procedures that must be followed and illustrates a methodical process that must be used to guarantee precise estimates of solar electricity. These essential actions are described as follows:

This study used the West Texas Mesonet dataset for Lubbock, Texas. The West Texas Mesonet is a network of weather stations that provides high-quality meteorological data for various locations in West Texas. The dataset includes 5-minute measurements of humidity, temperature, wind direction, wind speed, and solar radiation for the years 2012-2022. The dataset was pre-processed and cleaned to remove any missing or erroneous data [18]. The solar radiation data were processed to calculate the solar irradiance on a horizontal surface, which was used as an input variable for the ML models. The West Texas Mesonet dataset is localized and provides more accurate and relevant meteorological data for Lubbock than other US-wide resources.

Before training the ML models, the West Texas Mesonet dataset was pre-processed and cleaned to remove any missing or erroneous data. This step is crucial to ensuring the accuracy of the results and preventing errors in the models. The preprocessing steps included removing any NAN values, which are missing values in the dataset, and removing negative values for solar power, which is physically impossible [19]. After preprocessing, the dataset has been split into two sets: a training set and a test set. The training set was used to train the ML models, while the testing set was used to evaluate their performance. The training set contained a large portion of the dataset (80%), while the testing set contained a smaller portion (20%) to ensure that the models were not overfitting the data [20].

Feature selection can play a crucial role in optimizing solar power generation. Feature selection is the process of determining the most important variables or features that contribute to a given outcome while ignoring irrelevant or redundant ones. Using feature selection techniques, researchers can identify the subset of variables that have the greatest impact on solar power output, thus reducing the dimensionality of the problem and improving the accuracy and efficiency of solar power generation models [21,22]. This can also help investors reduce costs by focusing on the most influential factors rather than wasting resources on less significant ones. Therefore, feature selection is an important tool for optimizing solar power generation and achieving greater efficiency in renewable energy production [23]. In order to optimize solar power generation, it is important to understand the factors that affect solar power output. This can include variables such as solar radiation, wind speed, temperature, humidity, and so on. Feature selection for extracting solar power can be done by examining the correlations between these variables and solar power output. Therefore, investors can gain insights into the relative importance of each factor and potentially improve the efficiency of solar power generation [24]. In this context, the correlation coefficients between solar power and the most related variables are shown in Table 1. The above table shows that solar radiation has the highest positive correlation with solar output, while wind direction, temperature, and wind speed have moderate positive correlations. The humidity has a moderately negative correlation with solar power output. These findings suggest that solar radiation is the most important factor affecting solar power output. The information can be used to inform future research and practical applications in solar power generation [25]. The correlation matrix is depicted in Figure 2.

According to Figure 2, solar radiation has the most effect, and humidity has a low effect on solar power output. Furthermore, the scatter plot of the solar power regarding the correlated variables is depicted in Figure 3.

Based on Figure 3, the scatter plot matrix is utilized to visualize the pairwise relationship between the solar power output and the other weather-related variables like solar radiation, wind direction, temperature, wind speed, and humidity. The diagonal histograms show the distribution of each variable, while the off-diagonal scatter plots show the pairwise relationship between each pair of variables. For example, the scatter plot between solar power output and solar radiation shows a positive linear relationship, suggesting that solar radiation has the highest positive correlation with solar power output. The scatter plot between solar power and humidity shows a negative relationship, suggesting that humidity has a moderately negative correlation with solar power output [26].

The following evaluation metrics were used to assess the performance of the hybrid RFR-LSTM approach and compare it with standalone RFR, LSTM, and other Machine Learning (ML) and Deep Learning (DL) models.

Mean Squared Error (MSE): MSE measures the average squared difference between the predicted and actual solar power values. It quantifies the overall accuracy of the predictions. The formula for MSE is:

R-squared (R 2 ) Score: The R 2 score evaluates the proportion of variance in the target variable (solar power generation) that can be explained by the predictions. It indicates the goodness of fit of the model. The formula for the R 2 score is:

(2)

These evaluation metrics provide insights into the accuracy and performance of the models in predicting solar power generation. A lower MSE value indicates better accuracy, while a higher R 2 score indicates a better fit of the predictions to the actual data.

In this study, eight ML models were used to forecast solar PV power generation: Linear Regression (LR), Polynomial Regression (PR), Decision Tree Regression (DTR), Artificial Neural Network (ANN), Convolutional Neural Network (CNN), Random Forest Regression (RFR), Gradient Boosting Regression (GBR), and Long Short-Term Memory (LSTM).

Linear regression (LR) is a statistical approach for modeling the relationship between a target and features by fitting a linear equation to the observed data. In solar power forecasting, LR can be used to model the relationship of weather features and generated solar power. LR is a simple and interpretable model that can provide useful insights into the relationship among a target and some features [27].

PR is a type of LR in which the relationship between the target and the features is modeled as an nth-degree function. Polynomial regression is a useful technique when the relationship between the target and the features is non-linear. In solar power forecasting, polynomial regression can be used to identify the nonlinear relationship between weather variables and solar power generation. Polynomial regression can be more flexible than linear regression, but it can also be prone to overfitting, particularly when the degree of the polynomial is high. Overfitting occurs when the model fits the training very closely, and as a result, it may not generalize well to new data. To avoid overfitting, it is important to use cross-validation and regularization techniques when fitting polynomial regression models [28].

DTR is a non-parametric supervised learning technique implemented for either regression and classification applications. In the context of solar power forecasting, decision trees can be used to model the relationship of weather features and extracted solar power. The basic idea behind decision trees is to recursively partition the data into sub datasets regarding the input feature values. Each subset represents a node of the tree, and the tree is grown until a stopping point, like the lowest number of samples per loaf or the maximum node depth. Decision trees have several advantages, including their simplicity, interpretability, and ability to handle both continuous and categorical input variables. However, decision trees can be prone to overfitting, particularly when the tree is allowed to grow too deep. To avoid overfitting, it is important to use pruning techniques, such as minimum sample split or minimum impurity decrease [29].

ANN is an ML model inspired by the function of the human brain. In the context of solar power forecasting, ANN can be used to model the relationship of weather features and generated solar output. ANN works by using a set of interconnected nodes (neurons) that are organized in layers. The output layer generates the output forecast after receiving the input data from the input layer. The hidden layers process the input data using a series of mathematical operations that transform the input data into a useful representation for the output layer. ANN can be trained using a variety of techniques, including supervised and unsupervised learning. ANN is capable of discovering patterns and relationships in input datasets autonomously. These networks possess several advantages, including their ability to handle non-linear relationships among input and target variables, as well as their flexibility in modeling complex data. However, training ANN requires careful hyperparameter tuning to achieve optimal performance. One potential challenge is overfitting, where the model fails to generalize well. To address this issue, techniques like regularization and early stopping are essential to prevent overfitting [30].

CNN is a DL model that performs well for image and time series data. In solar power forecasting, CNN can be used to identify the spatial and temporal dependencies in the input data, making it a powerful tool for solar power prediction. CNN works by applying convolutional filters to the input data, which extract features that are useful for the prediction task. The output of the convolutional layer is passed through a series of nonlinear activation functions and pooling layers, which help to lower the dimensionality and increase the computational efficiency of the model. The resulting features are then passed through fully connected layers, which produce the final prediction. One of the main advantages of CNN is its ability to automatically learn spatial and temporal patterns in the input data, which makes it well-suited for solar power forecasting. Additionally, CNN can handle large amounts of data and can generalize well to new data, making it a powerful tool for prediction tasks. However, CNN can be computationally expensive to train and requires large amounts of data to achieve good performance. It also requires careful tuning of hyperparameters to achieve optimal performance, and it can be sensitive to changes in the input data, such as changes in weather patterns or solar panel configurations [31].

RFR is an ensemble learning technique that combines multiple decision trees to create a strong learner. In the context of solar power forecasting, RFR can be used to model the relationship between weather features and the extracted solar power. RFR works by training multiple DR, with each tree using a random subset of the input features. The forecasts for the trees are then combined to make a final forecast. This approach can reduce the risk of overfitting and improve the generalization performance of the model. RFR has several advantages, including its ability to handle non-linear relationships among the target and output features, its robustness to outliers and noisy data, and its ability to capture complex interactions between variables. Additionally, RFR can handle missing data and categorical variables without requiring pre-processing. However, RFR can be computationally expensive and requires careful tuning of hyperparameters to achieve good performance. To overcome these challenges, it is important to use techniques such as regularization, early stopping, and cross-validation to prevent overfit [32].

GBR is another ensemble learning technique, and in the context of solar power forecasting, GBR can be used to model the relationship among weather features and generated solar power. GBR works by iteratively training decision trees to correct the errors of the previous trees. Each new tree is fitted to the residual errors of the previous trees, so that the overall model gradually improves in its ability to predict the target variable. GBR has several advantages, including its ability to handle non-linear relationships, its robustness to outliers and noisy data, and its ability to capture complex interactions between variables. However, like RFR, GBR can be computationally expensive and requires careful tuning of hyperparameters to achieve good performance [33].

LSTM is a recurrent neural network (RNN) that performs well for sequential data such as time series. LSTM networks are designed to overcome the vanishing gradient problem. LSTM networks use memory cells and gating mechanisms to selectively recall or forget details from earlier phases, allowing them to capture long-term dependencies more effectively. LSTM networks have several advantages, including their potential to handle input sequences of variable length and to learn complex relationships. In addition, it is important to use techniques such as dropout, early stopping, and regularization to prevent overfit problems [34].

In this study, the performance of eight machine learning models for solar power forecasting was evaluated using two commonly used metrics for evaluating the performance of ML models: Mean Squared Error (MSE) and R-squared (R²). MSE calculates the mean squared variation in the expected and true values. It is commonly used to evaluate the accuracy of regression models. A smaller MSE indicates that the model is better at predicting the outcome variable [35]. R 2 is a metric that shows how much of the variance in the target is accounted for by other variables. It gauges how well a regression model fits the data. R 2 has a 0 to 1 scale, with 1 denoting a perfect fit [36]. Figure 4 shows the performance of the Random Forest Model for one day, four days, ten days, and one month. Figure 4 shows the performance of a Random Forest Regression Model used for forecasting solar power for different times, ranging from one day to one month. Time is shown on the x-axis, while actual and expected solar power output are shown on the y-axis. The blue line represents the true solar power output by the model, while the red line represents the predicted solar power output observed. The model's performance can be assessed by looking at the overall pattern and trend in the graph. If the red line consistently aligns closely with the blue line, it suggests the model performs well in accurately forecasting solar power output over different timeframes. On the other hand, significant disparities between the two lines indicate that the model's predictions may not be reliable enough. Figure 3 shows that the predicted solar power output (red line) generally follows the trend of the true solar power output (blue line). This indicates that the Random Forest Regression Model is able to capture the underlying patterns and variations in the solar power output to some extent. Furthermore, Table 2 shows the performance of each model. According to Table 2, the ML/DL models chosen for the paper come from different categories, including traditional regression models, ensemble models, and neural network models. The results show that all eight ML models were able to predict solar PV power generation with good accuracy, as evidenced by the high R² values and low MSE values. The RFR and LSTM models performed well, achieving lower MSE and higher R² values than others. The MSE for RFR is 2.06, which is the lowest, and the R² value is 0.97%, which shows very high accuracy. Lower MSE indicates less deviation between predicted and actual values, and higher R² values indicate a better fit to the data compared to the other models. Overall, the results suggest that ML models, particularly ensemble methods such as RFR and GBR and neural network models such as LSTM, CNN, and ANN, are well-suited for forecasting solar PV power generation. Ensemble methods combine multiple ML models to improve predictive accuracy, while neural network models, including LSTM, CNN, and ANN, are designed to capture complex patterns and relationships in data.

Predicting solar energy generation is a challenging task due to the intermittent nature of weather conditions. Factors such as cloudy days, variations in wind, and the time of year can significantly impact the performance of solar panels and the amount of energy they generate. These dynamic elements introduce uncertainty and make accurate predictions more complex. Therefore, accurate ML/DL models are crucial for predicting solar power with precision. This paper has demonstrated the effectiveness of using various machine learning models to predict PV generation in Lubbock, Texas. The results indicate that the RFR and LSTM models performed better among other models, with the lowest MSE of 2.06% and 2.23%, respectively, and the highest R² values of 0.977 and 0.975, indicating their potential for helping solar energy operators optimize their operations and plan better for solar PV power generation. According to the research results, ensemble approaches like RFR and GBR and neural network models like LSTM, CNN, and ANN are optimal for predicting solar PV power generation. However, it is important to note that the dataset used in this study is limited to a specific location (Lubbock, Texas) and a particular period, and the models may perform differently in other locations or under different weather conditions. Future research endeavors could expand upon this study by investigating the performance of the models under varying weather conditions and in different geographical locations. This would provide valuable insights into the generalizability and robustness of the developed models. Additionally, exploring the impact of data pre-processing techniques on the models' performance would be a worthwhile avenue to pursue. By examining the effectiveness of different data preprocessing methods, researchers could potentially enhance the accuracy and reliability of PV generation predictions.

The data presented in this study are available on request from the corresponding author.

In this work, the financial support of the National Science Foundation (NSF) under Award Number: 2115427 is gratefully acknowledged. SRS RN: Sustainable Transportation Electrification for an Equitable and Resilient Society (STEERS).

Not applicable.

Not applicable.

The authors declare that there is no conflict of interest regarding the publication of this manuscript. In addition, the ethical issues, including plagiarism, informed consent, misconduct, data fabrication and/or falsification, double publication and/or submission, and redundancies have been completely observed by the authors.