This article is published as: Chuangye Wang, Wei Zhong, Ji-Cheng Zhao, "Insights on Phase Formation from Thermodynamic Calculations and Machine Learning of 2436 Experimentally Measured High Entropy Alloys", Journal of Alloys and Compounds, vol. 915, 165173, 2022. https://doi.org/10.1016/j.jallcom.2022.165173

High entropy alloys (HEAs) typically consist of four or more principal elements with concentration between 5 and 35 at.% [1][2][3][4][5][6][7][8][9][10] in contrast to traditional alloys such as copper, iron, magnesium and aluminum alloys with low concentrations of other alloying elements. HEAs open up vast composition spaces for designing and discovering new alloys for ever-increasing demands of new materials for energy, environment, and human well-beings.

There are many experimental methods for preparation of HEAs, including mechanical alloying, induction melting, vacuum arc melting, vapor deposition method, and sputtering [3,5,[11][12][13][14][15]. To accelerate the discovery of novel and advanced materials, the CALculation of PHAse Diagrams (CALPHAD) approach has often been employed to predict and investigate phase formation of HEAs [16][17][18][19][20][21][22][23][24]. In addition to CALPHAD, phase selection rules were developed by various research groups to help predict phases of HEAs via studying the existing experimental data. Such empirical rules are built by presenting the formation of phases in onedimensional histograms or two-dimensional plots whose axes are usually thermodynamic and physical parameters. For instance, the Hume-Rothery rules depict the effects of atomic size ratio and electrochemical properties on solid solution (SS) formation [25]. Guo et al. revealed that single-phase FCC forms at valency electron concentration (VEC) > 8 and single-phase BCC forms at VEC < 6.87, while Jiang's work added some constraints based on their designed alloys and indicated that this formation rule applies under the conditions of Δ𝑆𝑆 𝑚𝑚𝑚𝑚𝑚𝑚 > 12.47, 7.27kJ/mol < Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 < 4 kJ/mol, and δ < 4.27%, where Δ𝑆𝑆 𝑚𝑚𝑚𝑚𝑚𝑚 , Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 , and δ are the mixing entropy, mixing enthalpy, and difference in atomic sizes, respectively [26,27]. Recently, Yang et al. developed a new VEC selection rule for predicting SS phases based on high-throughput CALPHAD calculations in the Al-Co-Cr-Fe-Ni system [28]. Other parameters such as -22 ≤ Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 7 kJ/mol, 0 ≤ δ ≤ 8.5, and 11 ≤ Δ𝑆𝑆 𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 19.5 kJ/mol are found to be conditions for the formation of SS [29]. According to the evaluation of Senkov et al. over 130,000 alloys, the increasing number of alloying elements in a system beyond 3 results in a reduced likelihood of occurrence of SS [30]. Inoue summarized the conditions to form bulk metallic glasses (BMGs) in terms of the number of elements, atomic size ratios, and mixing heat of atomic pairs [31]. Both Ω ≥ 1.1 and δ ≤ 6.6% were shown by Yang and Zhang to be conditions for the formation of SS phases, and BMGs form in regions of smaller Ω and larger δ as compared to HEAs, where Ω is a parameter related to the Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 , Δ𝑆𝑆 𝑚𝑚𝑚𝑚𝑚𝑚 , and melting temperature (𝑇𝑇 𝑚𝑚 ) of constituent elements [32]. A single dimensionless thermodynamic parameter ϕ which is correlated with Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 , Δ𝑆𝑆 𝑚𝑚𝑚𝑚𝑚𝑚 , 𝑇𝑇 𝑚𝑚 , and excessive entropy was defined and HEAs were found to be single-phase SS (SPSS) at ϕ > 20 based on the analysis of nearly 50 types of HEAs [33]. All these phase selection rules can be used to guide future design of HEAs; however, most of the rules were developed from small experimental datasets. Li and Tsai collected 100 selected HEAs from the literature and assessed the accuracies of 8 published formation rules; they found that the overall accuracy is only ~ 72% in predicting SPSSs and intermetallics (IM) [34]. Therefore, it is highly desirable to test and expand these rules using very large, consistent datasets.

Machine learning (ML) holds great promise for future materials design and discovery [35,36]. Unlike CALPHAD that was built upon semi-empirical physical models, ML makes prediction using data-driven strategies with unique algorithms [6,37,38] that learn from training datasets with input patterns and a optimization target, and then extract the implicit insights hidden in the datasets. For phase prediction using ML, the properties of alloys or constituent atoms will be taken as input features, and the crystal structures of the phases are the corresponding targets. Since ML can explore different features simultaneously, it is able to overcome the limitations of traditional strategies in studying phase selection rules [39][40][41]. For example, phase selection rules was explored via support vector machine (SVM) and artificial neural network (ANN) on two separate group of features -compositions of HEAs and physical parameters of HEAs, respectively, and showed that the accuracies of learning obtained from these two groups of input features are similar and high for HEAs when 4 or 5 physical parameters were employed [42,43]. The Gaussian process statistical analysis was performed on 322 alloys based on a combination of 9 physical parameters, which provided a robust predictions for the formation of SPSSs [44]. Krishna et al. utilized six ML approaches with 5 input features to study the classification of SS and SS + IM with a dataset of 636 compositions, and they found that the trained ANN reached 80% prediction accuracy and can correctly predict the crystal structures of newly designed alloys [45]. Zhao et al. recently studied 3 categories (SS, IM, SS + IM) using five ML models and achieved 87% prediction accuracy through ANN with 5 selected features [46]. Pei et al. put forward a new parameter λ related to bulk modulus, melting temperature, volume, and configurational entropy with the help of performing ML and found that SPSS forms at λ ≥ 1 with 73% accuracy and it can increase to 81% with a constraint of δ ≤ 6%. They further applied their new formation rule to new compositions that are SPSSs predicted by CALPHAD and the consistency reached 94% [47]. In addition to studying phase formation separately, a recent study applied the eXtreme Gradient Boosting (XGBoost) model to explore > 300,000 equilibrium data of HEAs generated by CALPHAD calculations, and built more comprehensive and superior phase selection rules for single-phase FCC and BCC based on the 5 ML-selected features [48]. Machaka applied 6 ML models to 896 SPSSs and 101 dualphase SSs and did feature selection using 36 dataset features [49]. The accuracy reaches 95% with the top-most identified 13 features. Zhang et al used a model of SVM combined Kernel Principal Component Analysis to classify a dataset of 556 entries including SS, amorphous (AM), the mixture of SS and IM, and IM, and obtained 97% accuracy with 4 selected features [50]. Zhou et al. performed various experiments on the (FeCrNi) 10-x (ZrCu) x system to verify the ability of the trained ML models, and they achieved good agreements for bulk alloys made by arc melting and ribbon samples through vacuum melt spinning. The thin films made via cosputtering presented a transition from a crystalline to amorphous structure as x in (FeCrNi) 10- x (ZrCu) x increases, which is consistent with their ML predictions [51]. [15] were employed in this study to perform a more extensive test of CALPHAD predictions and ML capabilities. The phase categories and their corresponding numbers of alloys are listed in Table 1. CALPHAD was utilized to calculate the phase equilibrium of 1,761 SS HEAs from a full dataset of 2,436 HEAs, and the CALPHAD predictions were analyzed and compared with experimental data. Phase selection rules based on different parameters were also investigated using both traditional methods and ML models.

A program is written to predict the phase equilibrium of the 1,761 SS alloys using TC-Python, a python language-based software development kit (SDK) that allows Thermo-Calc to perform high throughput calculations. The computation settings were: TCHEA4 database, only BCC and FCC phases were entered (explained in the Results section), and 1100 °C which is the temperature at which all high-throughput thin film libraries were made.

Eight parameters are selected for studying the phase selection rules among 2,436 alloys: the mixing entropy (Δ𝑆𝑆 𝑚𝑚𝑚𝑚𝑚𝑚 ), the mixing enthalpy (Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 ), atomic size difference (δ) [52], Ω that links mixing entropy, mixing enthalpy and melting temperature [32,53], valence electron concentration (VEC) [25,26], the FCC-BCC-index (FBI) [15], number of itinerant electrons (e/a) [25,54], and the difference in Pauling electronegativity Δχ [4,55]. These parameters are defined as followings. χ 𝑚𝑚 ) are the mole fraction, atomic radius, valency electron concentration, number of itinerant electrons, electronegativity of element 𝑖𝑖, average atomic radius, average electronegativity of an alloy. N is the number of elements in an alloy system, and R is the gas constant. Ω 𝑚𝑚𝑖𝑖 (= 4ΔH 𝑚𝑚𝑚𝑚𝑚𝑚 𝐴𝐴𝐴𝐴 ) is the regular solution interaction parameter between elements 𝑖𝑖 and 𝑗𝑗 [56], and ΔH 𝑚𝑚𝑚𝑚𝑚𝑚 𝐴𝐴𝐴𝐴 is the mixing enthalpy of A-B binary liquid alloys whose values can be found in Refs. [57,58].

, where (𝑇𝑇 𝑚𝑚 ) 𝑚𝑚 is the melting point of element 𝑖𝑖. Φ 𝑚𝑚 is +1 for element 𝑖𝑖 with either the FCC or HCP crystal structure, and φ 𝑚𝑚 is -1 for elements 𝑖𝑖 with the BCC crystal structure.

With these eight parameters as input features, four supervised ML algorithms, decision tree (DT), k-nearest neighbor (KNN), SVM, and ANN were employed to predict the phase formation of the HEAs. These four algorithms are implemented using the scikit-learn package, a machine learning library in python [59]. Each algorithm has different tuning parameters that are used to adjust the training model. Five-fold cross-validation (CV) was employed to judge the performance of predictions, which can avoid the overfitting and underfitting phenomena in ML. The dataset is split randomly into five disjoint subsets with nearly the same size. The models were trained with 4 subsets and tested 5 times with the remaining subset. The 5-fold CV accuracy is the average of the 5 test accuracies, which is called CV accuracy for simplicity throughout this article.

The DT model divides the data at each parent node into the left child node and the right child node in a binary tree [60,61] represent the data at node m, the left child of node m, and the right child of m, respectively. 𝑙𝑙 𝑙𝑙𝑙𝑙𝑙𝑙𝑙𝑙 and 𝑙𝑙 𝑟𝑟𝑚𝑚𝑟𝑟ℎ𝑙𝑙 are the number of samples in the left child and the right child of node m, respectively. 𝑁𝑁 𝑚𝑚 is the number of samples at node m and θ is a candidate split for the 𝑄𝑄 𝑚𝑚 . This algorithm achieves most information gain at the best split θ * that minimizes G. However, using DT can overfit the data by generating overly complex trees, resulting in failure of predictions. Therefore, two parameters, the min_samples_leaf (the minimal number of samples at a node) and max_depth (the maximal depth of a tree) were controlled during our ML to regularize the size of trees to avoid overfitting. KNN utilizes a voting mechanism by k nearest neighbors to a query sample [62,63]. In this way, the query sample is assigned to the class with major votes among k nearest neighbors. When the weight of voting for every nearest neighbor is the same, then KNN uses a uniform weight. However, the voting weight is often different for every voter since the voting power of every voter is not same. The distance weights are proportional to the inverse of distance between training sample and query sample and are used here because the contributions of nearer neighbors are generally more than that of farer neighbors, which is based on the , where 𝑝𝑝 is a training vector and 𝑞𝑞 is the query vector. Not only does the weight of a neighbor matter, but also the number of participating nearest neighbors. Thus, it is meaningful to control the number of nearest neighbors for obtaining high CV accuracy, which can be adjusted by the parameter n_neighbors.

The SVM model implements one-versus-one method for multi-class classification [65]. It is chosen with radical basis function 𝑘𝑘�𝑥𝑥 𝑚𝑚 , 𝑥𝑥 𝑖𝑖 � = 𝑒𝑒 -γ�𝑚𝑚 𝑚𝑚 -𝑚𝑚 𝑗𝑗 � 2 as the kernel function, where 𝑥𝑥 𝑚𝑚 and 𝑥𝑥 𝑖𝑖 are the features of 𝑖𝑖th and 𝑗𝑗th samples, and γ is the kernel coefficient in this work. Γ is also an indicator of the degree of effect of a training sample on other samples. A sample with larger γ has lower influence on other samples. Another penalty parameter C is also critical to control the tradeoff between the overfitting and simplicity of the decision surface. A large C could increase misclassification of samples in the training process and cause underfitting, while a small C could improve the training accuracy but may lead to overfitting for the training samples. Hence the appropriate parameters γ and C of the SVM model should be determined to obtain reasonable train and test outcomes.

ANN simulates the decision-making process of human mind by constructing artificial neurons in the model [66,67], whose architecture is illustrated in Fig. 1. It usually contains an input layer of the exploring features that are fed into separate neurons, some hidden layers whose quantity and number of neurons are affected by the complexity of data and number of classes, an output layer including the classified outcomes in each neuron. The input features propagate forward to each neuron in each hidden layer with fitted weights, then the fitting values are transformed by an activation function. The rectified linear unit activation function was employed here. At each hidden layer, a bias is introduced and propagates with the transformed fitting values to the next layer together. The propagating process stops till it arrives at the final output layer. When number of neurons is too large or too small, it has a negative influence on predictions, thus we search out the best set by using three hidden layers with the number of neurons n 1 , n 2 , n 3 in each layer ranged from 5 to 50 with an interval 5 -there are 1,000 total combinations.

Since the effects of the initial 8 input features on phase prediction are likely very different, it is very valuable to perform a systematic screening of the features and find those that are more critical phase selection criteria. The forward selection is applied with a three-step procedure for each model [36,[68][69][70]. First step is training these features separately and creating an initial empty feature pool. The feature with the highest CV accuracy is added to the initial pool of features. The second step is adding next feature that increases the CV accuracy most to the previous pool of features from remaining features until all the features are stored in the feature pool. Therefore, a pool of features is ranked by their importance to ML models in datasets. The last step is selecting a cutoff point from the ordered optimal pool for each ML model, where the increase in the CV accuracy by adding a new feature to the pool becomes smaller than the standard deviation of the CV accuracy.

We first performed unconstrained high-throughput thermodynamic calculations of the equilibrium phases at the experimental temperature (1100 °C) for each of the 2,436 HEAs using the Thermo-Calc software and its associated TCHEA4 thermodynamic database, and the calculations are presented Figs. S1 and S2 in the Supplementary Information. The results are very different from the experimental observations reported by Kube et al., including the formation of IMs in far more alloys than experimental observations as well as the formation of liquid in a large number of alloys. Since the liquid phase was not observed during the experimental co-sputtering process of the combinatorial films, it is thus excluded in subsequent thermodynamic calculations in this study. The experimental IM consists of Sigma and other possible unidentified phases, which makes it hard to compare with thermodynamic calculations. However, we also tried to perform constrained calculations for the 233 alloys with the IM phase from experiments after excluding the liquid phase and it predicts 146 SS and 87 Sigma + SS alloys, which deviates significantly from the experimental observations, as shown in Fig. S3(a) and (b) in the Supplementary Information. The prediction of the IM formation is also beyond the capabilities of the current TCHEA4 database, similar to the situation in Ni-based superalloys [71], thus IMs are also excluded in further calculations; and we focused on calculating and analyzing the phase formation of the SS alloys. Calculations using theTCHEA4 database predicted far more alloys with the B2 phase and B2 associated two-phase cases (B2+FCC and BCC+B2) as well as the BCC+B2+FCC threephase cases than the experimental observations, as shown in Fig. 2(a). Far few single-phase BCC alloys were predicted at the expense of forming the B2 phase in a very large number of alloys. The TCHEA4 database calculations predicted the number of single-phase FCC alloys close to experimental observations, but the overall consistency with experimental observations (including all the matched predictions of BCC, B2, BCC+FCC, and B2+FCC phases) is only 28.2%. When the single-phase B2 and single-phase BCC phases are regarded as one BCCbased single phase (BCC/B2), the agreement between the Thermo-Calc results and experimental observations is much better and reaches 61.4%, Fig. 2(b), showing less singlephase BCC/B2 and single-phase FCC, and more BCC/B2 + FCC two-phase cases. It is very likely that the Thermo-Calc database TCHEA4 does not have accurate enough thermodynamic parameters for the B2 phase whose Gibbs free energy is very close to the disordered BCC phase and thus hard to be modeled accurately, especially for multicomponent HEAs. The disparity between the CALPHAD predictions (representing the phases of bulk alloys) and the sputtering experimental data may also be the results of some non-equilibrium nature of the sputtering process [15]. calculations. Only a few alloys as represented by the blue part of the middle panel of Fig. 3(c) of the experimentally observed 2-phase (BCC/B2 + FCC) alloys were computed as single-phase BCC/B2. Similarly, the green part of the middle panel in Fig. 3(c) represents the experimentally observed 2-phase (BCC/B2 + FCC) alloys which were predicted to be single-phase FCC. Fig. 3 also shows a clear trend that alloys with higher VEC prefer FCC over BCC/B2 both in experimental observations and CALPHAD predictions. A previous study has found that the FCC and BCC phases tend to be stable at higher VEC (> 8) and lower VEC (< 6.87), respectively [26]. The rule of VEC < 6.87 for single-phase BCC/B2 formation holds well with respect to the 1,761 SS HEAs, as shown in Fig. 3. The VEC > 8 rule also holds well for the formation of single-phase FCC, but at 8 < VEC < 9.16, there is also substantial alloys with BCC/B2 + FCC two-phase structure. Only when VEC > 9.16, the single-phase FCC is formed with only a few exceptions in the CALPHAD predictions. At VEC < 6.87 and VEC > 9.16, the predicted phase formation is in good agreement with experimental observations, which illustrates the good ability of CALPHAD in calculating phase equilibrium of HEAs, especially for single-phase solid solution SPSS. Yang et al. found that more than 90% compositions have single-phase BCC structures at 5.7 ≤ VEC ≤ 7.2, and it has pure FCC structure at VEC > 8.4 in Al-Co-Cr-Fe-Ni system by highthroughput calculations using CALPHAD [28]. The new rules revealed from this study and that of Kube et al. [15] should be more reliable due to the very large number of experimental HEAs that consist of more elements.

The crystal structure, atomic radius, melting temperature, VEC, φ, e/a, and electronegativity of Al, Cr, Mn, Fe, Co, Ni, Cu in Table 2 are used to calculate those aforementioned 8 parameters for HEAs [4]. Representative 2D plots are shown in Fig. 4 in which the left-hand side and right-hand side columns plot the experimental results and the CALPHAD results, respectively, for comparison.

One can see from Fig. 4(a) and 4(b) that at δ < 5 based the experimental data and δ < 4 based on the computed data, the FCC phase forms at higher FBI values, which is the direct consequence of the definition of FBI, Eq. 6. For experimental data at δ > 6, the FBI effect This article is published as: Chuangye Wang, Wei Zhong, Ji-Cheng Zhao, "Insights on Phase Formation from Thermodynamic Calculations and Machine Learning of 2436 Experimentally Measured High Entropy Alloys", Journal of Alloys and Compounds, vol. 915, 165173, 2022. https://doi.org/10.1016/j.jallcom.2022.165173 breaks down and almost all alloys form single-phase BCC irrespective of the FBI values, showing the overwhelming effect of δ in dictating the formation of BCC when δ value is high. This phenomenon is also verified and is more obvious in the CALPHAD calculations. Formation of BCC/B2 over FCC with increasing size difference δ was attributable to the lower packing density of BCC/B2 and higher ability to accommodate atoms with different sizes than FCC structure [15]. Guo and Liu [29] found that Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 and δ for the formation of SS satisfy -22 ≤ Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 ≤ 7 0 ≤ δ ≤ 8.5, and 11 ≤ ΔS mix ≤ 8.5 simultaneously. Our study, covering far more alloy compositions than those of Guo and Liu, shows that the Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 range for the formation of FCC is similar to that of Guo and Liu, but it can be expanded to -33.6kJ/mole to 17.5 kJ/mol for forming single-phase BCC/B2, Fig. 4(c) and(d). Single-phase BCC can form at a negative Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 and larger δ values, which agrees with the work of Raghaven et al [72] and Agarwal and Rao [43]. Our results also agree with the observation of Agarwal and Rao that BCC can coexist with FCC at a lower δ and FCC preferentially forms at a negative Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 while BCC prefer to form at a positive Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 . The CALPHAD results show similar trend but more pronounced separations of the phase formation regions, Fig. 4(d). These phase selection rules are useful for predicting alloy crystal structures and compositions, but more effective rules are still highly desired; thus we will subsequently explore the capabilities of ML in improving the phase predictions.

Different HEA libraries cover different regions of the multicomponent HEA composition space. The agreements between experimental and computational results can be very different. Two examples are shown in Fig. 5 to illustrate the degree of agreements for both the Al-Cr-Co-Ni-Fe and Al-Cr-Co-Ni-Cu systems. Fig. 5(a) and(b) show that for the compositions covered by a pseudo-ternary Fe -Co 51.4 Ni 48.6 -Al 42.7 Cr 57.3 system, the agreement is good except for a smaller single-phase FCC region and slightly larger BCC/B2 + FCC two-phase region from CALPHAD. Much pronounced difference was observed in Fig. 5(c) and(d) for the compositions covered by the pseudo-ternary Cu -Co 54.8 Ni 45.2 -Al 48.3 Cr 51.7 system. Such comparisons may also help identify the specific systems where CALPHAD assessments need to be improved.

Two groups of datasets are analyzed using ML, one is the SS only dataset with 1,761 HEAs, the other is the full dataset with 2,436 HEAs. Similar to the analysis using CALPHAD, the SS is divided into three categories, BCC/B2, BCC/B2 + FCC, FCC, which are the targets of the ML models. The predictions from ML in the SS dataset are further compared with the experimental observations. The full dataset is classified into four classes, SS, AM, SS + AM, and IM as the output of the ML models. It is noted that IM is used here to refer to alloys with an intermetallic compound (mostly the σ phase) identified from XRD and some alloys contain additional phases in addition to IM. The ML models are optimized by adjusting their parameters first. Then the best feature sets are screened by their contributions towards improving the CV accuracy of predictions for each ML model, which provides information for the relative importance among these input features.

Before training the ML model and making predictions, it is important to look into the correlation among these features because adding collinear features will not provide more valuable information and will increase the time cost of training and could even degrade the performance of the models. The Pearson correlation coefficient is used for the quantitative assessment of the correlation between any pair of features, and it is calculated with the following formula,

where 𝑥𝑥, 𝑦𝑦 are the two variables/features, 𝑥𝑥 𝑚𝑚 and 𝑦𝑦 𝑚𝑚 are the 𝑥𝑥 and 𝑦𝑦 values of ith sample, 𝑥𝑥̅ and 𝑦𝑦 � are the mean values of 𝑥𝑥 and 𝑦𝑦, and 𝑙𝑙 is the sample size of both 𝑥𝑥 and 𝑦𝑦 [73]. Using this formula, the correlation between any pair of features is shown in Fig. features can be kept and further used in the ML models since there is no high correlation coefficient between any two features.

The ML models can be modified by tuning parameters and then judged by the CV accuracy. Here, the grid search method was employed to adjust parameters for each model with all the eight features, and the optimized results are shown in Fig. 7. The max_depth is identified as the main factor in determining the quality of the DT model and the min_samples_leaf has negligible effect, Fig. 7(a). The CV accuracy of the DT model reaches highest when max_depth is 10 and stay stable after 10; thus min_samples_leaf = 2 and max_depth = 10 were selected for the final DT model. The n_neighbors = 2 is best for the KNN model in Fig. 7(b). For SVM, a parameter set of 𝛾𝛾 = 10 𝑝𝑝 and 𝑉𝑉 = 2 𝑞𝑞 is created, where -6 ≤ 𝑝𝑝 ≤ 4 and 1 ≤ 𝑞𝑞 ≤ 10. Both 𝛾𝛾 and 𝑉𝑉 affect the accuracy of the SVM model, and both cannot be too large or too small; otherwise, they would lead to inefficiency. The pair of 𝛾𝛾 = 0.01 and 𝑉𝑉 = 64 (𝑝𝑝 = -2, 𝑞𝑞 = 6) achieved a high CV accuracy from the parameter sets, Fig. 7(c). The neural network was built with three hidden layers that contain n 1 , n 2 , n 3 neurons in each hidden layer, and the corresponding CV result is shown in Fig. 7(d). For each third hidden layer with neurons changing from 5 to 50 (interval 5), the combination of three layers' neurons with highest CV accuracy is marked in black dot, so there are 10 highest CV accuracies shown in 10 black dots. Finally, the combination n 1 = 35, n 2 = 25, n 3 = 10 is chosen from these 10 CV accuracies. The results of feature selections for the four ML models are shown in Fig. 8. The CV accuracy increases rapidly first, then stays stable, and finally decreases when more input features are employed for the ML models, Fig. 8(a), indicating that more features are not always helpful in ML because they could lead to overfitting in the training process and decreasing CV test accuracy. The order of precedence of the features in each ML model are shown in Fig. 8(b), and the best feature sets for different ML models are colored for easier visualization. The {VEC, Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 , e/a, δ} set is the best features for the DT model, which renders a 91.4% CV accuracy. Adding other features does not improve the performance of learning. The {VEC, δ, FBI, e/a}, and { VEC, Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 , e/a, δ, ΔS mix }, {VEC, Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 , e/a, δ, ΔS mix } sets are the best three feature combinations for KNN, SVM, and ANN, which achieves 93.1%, 90.2%, and 89.1% CV accuracy, respectively. In all four feature sets, the VEC feature owns the highest order of precedence, thus is the most influential among these eight features in determining SS formation. It is noted that the CV accuracy obtained by using only VEC feature is comparable to or even better than the prediction accuracy from CALPHAD; and the CV accuracy is higher than This article is published as: Chuangye Wang, Wei Zhong, Ji-Cheng Zhao, "Insights on Phase Formation from Thermodynamic Calculations and Machine Learning of 2436 Experimentally Measured High Entropy Alloys", Journal of Alloys and Compounds, vol. 915, 165173, 2022. https://doi.org/10.1016/j.jallcom.2022.165173 85% after training with the first three best features in each ML model, showing advantages of phase predictions over current state of thermodynamic calculations. The other features that are helpful for enhancing prediction accuracies are Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 , e/a, δ. The two worst features are Ω and Δχ that degrade the performance of the ML models in predicting SS formation. Li and Guo also found that VEC is the most important feature in classifying BCC, FCC, and NSP (not forming SPSS) with the SVM model and the CV accuracy reaches 90.7% under the input of {VEC, δ, T m , Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 , Δ𝑆𝑆 𝑚𝑚𝑚𝑚𝑚𝑚 } feature set [68]. A study by Zeng et al. explored >300,000 phase equilibrium generated by CALPHAD calculations in 3 classes (single-phase FCC, single-phase BCC, and other phases) using XGBoost [48]. They achieved > 99% accuracy on both the training set and the test set and predicted 155 experimental phases with 81% accuracy through the selection of 5 most important features {equilibrium temperature, average atomic radius, VEC difference, VEC, Δχ}. A feature set of 13 top-ranked features was screened in classifying BCC, FCC, and BCC + FCC by Machaka, and VEC was found to have the highest relative importance [49]. His models can reach 97.5%, 95.8%, 94.35%, and 94.0% using random forest (RF), SVM, KNN, and ANN, which is slightly higher than the current study possibly due to the larger set of features used in his study. Zhang et al. utilized a genetic algorithm with ANN and also found that the VEC criterion is important in classifying BCC, FCC, and dual-phase HEAs [70]. Their classification accuracy achieved 91.3% with a feature set containing VEC, the core electron distance, Δχ, the mismatch in compression modulus. Their CV accuracy is comparable to ours in ANN learning. Overall, the accuracy could reach higher than 89% for these four models, making the application of ML in phase prediction significant. Since ML has found both VEC and Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 are the two most significant features to predict the SS phases, VEC-vs-Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 plots are made based on experimental results in Fig. 9(a ML predictions in Fig. 9(b). All alloys with BCC/B2 + FCC two phases are located in the shading red region in Fig. 9(a). Towards the left side of this region is single-phase BCC/B2 and singlephase FCC on the right side. By counting the correct phases predicted by this division strategy, the accuracy is 67.1% in Fig. 9(a), which is much more efficient than relying on the VEC only (VEC < 6.87 for BCC/B2 and VEC > 9.16 for FCC). The colored areas in Fig. 9(b) are the predicted phases by SVM, and the boundaries for classification are curves rather than straight lines. When experimental data points are in the area with same color, then the predictions for phases agree with experiments (Note some of the red color points are outside the red region). The CV accuracy achieves 81.9% by SVM based on VEC and Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 , exhibiting the advantages of ML in predicting right phases over traditional strategies. We also used the ML models to study the classification on BCC, B2, FCC, BCC + FCC, and B2 + FCC phases (having B2 and BCC separately) and found that VEC is still the most important feature in every ML model as shown in Fig. S6 in the Supplementary Information.

The VEC-vs-Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 plots are far more accurate in predicting the single-phase BCC/B2 and single-phase FCC solid solutions without considering the BCC/B2 + FCC two-phase alloys, as shown in Fig. 9(c) and(d). Even a simple plot of experimental data only, a simple straight line of Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 = 28.97 × 𝑉𝑉𝑉𝑉𝑉𝑉 -246.77 can achieve 96.2% accuracy, which is simply remarkable, Fig. 9(c). The CV accuracy of 96.4%, 97.0%, 97.4%, and 96.5% is achieved for DT, KNN, SVM, and ANN, respectively, not significantly better than the predicted results determined by the straight line, Fig. 9(d) which shows SVM results only. Even though the simply plots of experimental data achieve excellent abilities to predict the right phases, it is ML that helps identify these two key parameters for the plots. With these well-trained models, we applied them to predict the selected alloy systems (containing 13 elements) that were tested in the study of Zeng et al. [49]. The compositions, experimentally determined phases, and ML-predicted phases are shown in the Supplementary Information Table S1. ANN beats other ML models with highest overall prediction accuracy 81.8%, demonstrating the feasibility of using trained ANN with thin-film data to predict the phase formation of bulk alloys. The CALPHAD prediction accuracy is 78.8% without a constrained limit and 86.4% with constrained input of only BCC and FCC phases. The prediction accuracy of ANN lies in between the two CALPHAD predictions, exhibiting a comparable prediction ability of ANN with CALPHAD. There is an advantage of using ANN over CALPHAD in predicting the phase equilibria of bulk alloys whose elements are not available in the TCHEA4 or other HEA CALPHAD database.

In addition to analyzing the classification of SS into BCC/B2, BCC/B2 + FCC, and FCC, ML was applied to distinguish SS from other phases, such as AM, SS + AM, and IM. Such information is very beneficial for the design of new materials, especially when it is necessary to avoid the additional phases. The ML process of tuning parameters for SS, AM, SS + AM, IM phases are the same as that for the SS process in the above discussions and the corresponding plots are presented in Fig. S4 in the Supplementary Information. The best four feature sets are {Δχ, FBI, Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 , VEC}, { Δχ, FBI, e/a, VEC, δ, Δ𝑆𝑆 𝑚𝑚𝑚𝑚𝑚𝑚 }, { Δχ, FBI, Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 , VEC, δ, Δ𝑆𝑆 𝑚𝑚𝑚𝑚𝑚𝑚 }, { Δχ, FBI, Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 , δ} with a CV accuracy of 93.6%, 93.3%, 95.5%, 92.7% for DT, KNN, SVM, ANN, respectively. The CV accuracies of these ML models are very high and very close to one another. The two best features for predicting the phases here are Δχ and FBI, which is different with VEC as the best feature in classifying BCC/B2, BCC/B2 + FCC, and FCC phases. Islam et al. found that VEC and Δ𝑆𝑆 𝑚𝑚𝑚𝑚𝑚𝑚 are the most and the least important features respectively in classifying AM, SS, and IM, by performing singular value decomposition on the weight matrix in the first hidden layer of ANN [39]. Our results also show the positive effect of VEC in helping classify phases, but it is not the most important feature. The VEC feature has the fourth priority in the feature set of DT, KNN, and SVM. It can also be seen that Ω is not helpful in predictions in both the full dataset and SS dataset from Fig. 10(b) since it decreases the CV accuracy when it is added into the input features. To separate SS and IM, King et al. proposed a new parameter Φ related to mixing enthalpy of both SS and IM by means of Miedema's model [74]. They found that Φ > 1 favors SS and almost all IM forms when Φ < 1, which forecasts more accurately than using Ω. Zhou et al. reported that their trained ANN, convolutional neural network (CNN), and SVM can reach around 95% accuracy with 13 features and reach around 89% with 4 features {Δ𝑆𝑆 𝑚𝑚𝑚𝑚𝑚𝑚 , δ, Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 , Δχ} in predicting AM, IM, and SS. The 4 top-ranked features they selected are amongst our selected feature sets. The CV accuracy with only Δχ and FBI as the two input features can reach more than 80% for the four ML models, which motivates us explore the Δχ-vs-FBI plot as a potential useful phase selection rule.

In the Δχ-vs-FBI plot shown in Fig. 11, all amorphous alloys are in the purple shading area, where -0.065 ≤ FBI ≤ 0.5 and 0.13 ≤ Δχ ≤ 0.165. All intermetallics with only one exception are in the black shading area, in which FBI ≤ 0.015 and Δχ ≥ 0.09. These two shading areas only have a small, overlapped area of which the most SS alloys form in the lower right area. Therefore, these two areas could be used as phase selection rules for screening amorphous alloys and intermetallics, respectively. The ML methods are also applied to classify BCC, B2, FCC, BCC+FCC, B2+FCC, BCC+AM, FCC+AM, BCC+FCC+AM, AM, and IM phases (having B2 and BCC separately) and FBI and δ are found to be the two best features in improving the CV accuracies rather than Δχ and FBI as shown in Fig. S8. The corresponding FBI-vs-δ plot is shown in Fig. S9 in the Supplementary Information.

High-throughput CALPHAD calculations were performed on 2,436 experimental HEAs and compared the results with experimental observations. Both CALPHAD predictions and experimental results show that alloys prefer BCC/B2 over FCC at larger atomic size difference which is understandable since BCC is less constrained than FCC, and alloys tend to form FCC structure at higher FBI, negative Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 , and small δ. Alloys exhibit single-phase BCC/B2 at VEC This article is published as: Chuangye Wang, Wei Zhong, Ji-Cheng Zhao, "Insights on Phase Formation from Thermodynamic Calculations and Machine Learning of 2436 Experimentally Measured High Entropy Alloys", Journal of Alloys and Compounds, vol. 915, 165173, 2022. https://doi.org/10.1016/j.jallcom.2022.165173 20 < 6.87 and FCC at VEC > 9.16, which agrees with prior findings for BCC/B2, but the range has been expanded beyond VEC > 8 for FCC as reported in the literature.

Four ML models were applied to two different datasets, one with the 1,761 solid solution only HEA compositions and the other with the entire 2,436 HEAs with alloys with form amorphous (AM) and/or intermetallic compounds (IMs). The best feature sets were identified using the five-fold cross validation method and achieve 91.4%, 93.1%, 90.2%, 89.1% for classifying BCC/B2, BCC/B2 + FCC, FCC with DT, KNN, SVM, and ANN models, respectively. These trained models were applied on predicting the phase structures of 66 selected bulk alloys, and the ANN model achieved 81.8% prediction accuracy, which is comparable to CALPHAD predictions. For classification of SS, AM, SS + AM, IM phases, the CV accuracy can reach 93.6%, 93.3%, 95.5%, and 92.7% in DT, KNN, SVM, and ANN models, respectively. The CV accuracies for both datasets are very high, which shows the effectiveness of ML in phase predictions of HEAs. In the ranked feature sets, VEC is identified as the most important feature in determining phase formation of SS. ML identified both Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 and 𝑉𝑉𝑉𝑉𝑉𝑉 as two dominating factors for predicting the solid solution phase; and based on this learning, a simple straight line of Δ𝐻𝐻 𝑚𝑚𝑚𝑚𝑚𝑚 = 28.97 × 𝑉𝑉𝑉𝑉𝑉𝑉 -246.77 was found to be able to predict single-phase BCC/B2 and single-phase FCC at 96.2% accuracy, which is simply remarkable, Fig. 8(c Both Δχ and FBI are the two best features that improve the performance of ML models in predicting the formation of SS, AM, SS + AM, and IM phases. In the Δχ-vs-FBI plot shown in Fig. 10, all amorphous alloys are in the purple shading area, where -0.065 ≤ FBI ≤ 0.5 and 0.13 ≤ Δχ ≤ 0.165. All intermetallic-containing alloys with only one exception are in the black shading area, in which FBI ≤ 0.015 and Δχ ≥ 0.09. All these phase selection rules will be very valuable for future design of advanced HEAs for challenging applications.