MFB. Noor et al. /Future Sustainability February 2024| Volume 02 | Issue 01 | Pages 47-58 47 Article Machine learning in high-entropy alloys: phase formation predictions with artificial neural networks Md Fahel Bin Noor*, Nusrat Yasmin, Tiglet Besara Department of Physics, Astronomy and Materials Science, Missouri State University, USA A R T I C L E I N F O Article history: Received 01 November 2023 Received in revised form 30 November 2023 Accepted 06 December 2023 Keywords: Phase formation prediction, High entropy alloys, Artificial neural networks, Machine learning *Corresponding author Email address: mdfahelbin1@missouristate.edu DOI: 10.55670/fpll.fusus.2.1.5 A B S T R A C T Due to their complex compositions, high entropy alloys (HEAs) offer a diverse range of material properties, making them highly adaptable for various applications, including those crucial for future sustainability. Phase engineering in HEAs presents a unique opportunity to tailor materials for environmentally friendly technologies and energy-efficient solutions. However, the challenge of predicting phase selection, a key aspect in harnessing the full potential of HEAs for sustainable applications, is compounded by the limited availability of HEA data. This study presents a distinctive approach by using a precisely produced and selected dataset to train an artificial neural network (ANN) model. This dataset, unlike prior studies, is uniquely constructed to contain an equal amount of training data for each phase in HEAs, which includes single-phase solid solutions (SS), amorphous (AM), and intermetallic compounds (IM). This methodology is relatively unexplored in the field and addresses the imbalanced data issue common in HEA research. To accurately assess the model's performance, rigorous cross-validation was employed to systematically adapt the model's hyperparameters for phase formation prediction. The assessment includes metrics such as phase-wise accuracy (AM 86.67% SS 81.25% & IM 82.35%), confusion matrix, and Micro-F1 score (0.83), all of which collectively demonstrate the effectiveness of this approach. The study highlights the importance of feature parameters in phase prediction for HEAs, shedding light on the factors influencing phase selection. Its balanced dataset and training method notably advance machine learning in HEA phase prediction, providing valuable insights for material design amidst challenges and data scarcity in the field. 1. Introduction Recently, Multi-principle element alloys (MPEAs) have been different from conventional metal alloys, as these alloys consist of an equal proportion of individual principal elements [1]. MPEA is commonly mentioned interchangeably with high entropy alloy (HEA) in the literature [2–4]. Due to its remarkable properties, high entropy alloys are characterized as novel and promising materials class. These alloys tend to have complex chemical compositions containing several components [5–8]. Nonetheless, the HEA definition limits the number of species to a minimum of four. In comparison, only two species of identical atomic concentrations can comprise an MPEA. We opt to constantly use the term HEA in this paper because of its broader classification [3, 9]. Phase engineering is a strategic approach that employs various phase structures found in HEAs to achieve remarkable performance configuration [10]. This approach offers an abundance of potential to modify HEAs for specific applications, producing materials that are precisely tuned to meet various technological requirements. HEAs can exhibit a wide range of desirable characteristics, including elevated strength for high load-bearing capacity, increased hardness for improved durability, heightened ductility for improved deformability, robust wear resistance against Future Sustainability Open Access Journal https://doi.org/10.55670/fpll.fusus.2.1.5 February 2024| Volume 02 | Issue 01 | Pages 47- 58 Journal homepage: https://fupubco.com/fusus ISSN 2995-0473 mailto:mdfahelbin1@missouristate.edu https://doi.org/10.55670/fpll.fusus.2.1.5 https://fupubco.com/fusus MFB. Noor et al. /Future Sustainability February 2024| Volume 02 | Issue 01 | Pages 47-58 48 abrasion, immense environmental corrosion resistance, and exceptional catalytic characteristics that enable various chemical reactions [11–23]. Specific mechanical properties can be targeted utilizing the phases present in HEAs. These phases consist of amorphous (AM), intermetallic compounds (IM), single-phase solid solutions (SS), and hybrid SS and IM phases [6,24–26]. Predicting phase selection is essential for designing HEA, though the mechanism behind predicting phase selection is crucial in tailored HEA design, yet the mechanisms underlying phase formation are uncertain. Additionally, the properties of HEAs are significantly impacted by the phase structure, and despite improvements, designing HEA phases is still challenging and time-consuming [1, 5, 6]. Machine learning has become an important tool to help with material design [27–29]. Machine learning, inclusive of deep learning, requires extracting features from large datasets to recapitulate the relationships, which also offers the chance to predict the phase formation of HEAs focusing on existing research data by using a range of deep learning. Several studies reveal intriguing outcomes in phase formation prediction by compiling data on HEAs and developing deep learning algorithms [30, 31]. For predicting phase formation, Zhu et al. [5] have introduced a deep neural network (DNN) architecture using a residual network (ResNet), which achieves 81.9% overall accuracy. An ANN model is utilized in Islam et al.'s study for phase prediction, with 99% accuracy on training data, while the practical prediction accuracy was below 80%. [1]. Several algorithms were employed, including logistic regression, decision tree, support vector machine (SVM) classifier, random forest, gradient boosting classifier, and ANN in Y.V. Krishna et al.’s research work [32], ANN has demonstrated the best accuracy of more than 80% for the test data among these algorithms. New alloys were synthesized and characterized to validate the predictions that ANN is the most accurate prediction method in the studied alloy system. K-nearest neighbors (KNN), SVM, and ANN are the three machine learning algorithms used in the study by Huang et al. [33], and ANN exhibits superior testing accuracy than other models for predicting phases in new HEAs. In Uttam et al.'s [31] study, the use of a neural network (NN) model is introduced for the first time to predict the hardness of a refractory high entropy alloy (RHEAs), and the prediction is verified through experimental synthesis and microstructural analysis. This model successfully applies to various alloys to predict hardness, which is consistent with available experimental results. In another recent study [24], the predictive accuracy of an ANN model in determining phase selection across three distinct alloy types is evaluated. It emphasizes the extensive impact of atomic size differences (δ) on the phases within AM, SS, and IM alloys. The research effectively forecasts the phases in two novel alloys by leveraging this learning model in conjunction with a combination of three or four key parameters with confirmation through X-ray diffraction. This approach provides a potentially promising tool for advancing the composition design and phase selection of novel alloys. While the field is extending, challenges continue in predicting HEA phases, such as advancing deep learning algorithms and dealing with a lack of experimental data. Given the vast unexplored compositional design space, developing efficient machine learning algorithms based on existing data becomes crucial for precise HEA phase prediction. This work presents an ANN model that tackles the challenge of training with balanced data for each phase in HEAs, contrasting the frequent problem of unbalanced datasets in past research. The objective is to determine the hyperparameters that maximize predictive accuracy and generality when predicting phase selection in new HEAs, utilizing a balanced dataset. In this study for HEA phase formation prediction, an ANN model architecture is optimized for the current balanced dataset and fine-tuned hyperparameters such as batch size, learning rate, epochs, and dropout rate. The model's performance is evaluated on the final test set by measuring phase-wise accuracy, originating a confusion matrix, and determining the Micro-F1 score, and the results are then compared with prior studies. The study also analyses the significance of feature parameters in phase prediction outcomes, clarifying the relative importance of physical parameters influencing phase selection. 2. Computational Methods The HEA dataset underwent preprocessing and preparation using conventional data science techniques prior to being utilized to train the model. Three distinct datasets from various earlier studies [34–36] were selected and employed to construct the ANN-based model. A dataset of 240 HEAs was obtained after the elimination of redundant samples and sections with incomplete or duplicated data. An instance of randomly selected five rows of the dataset is presented in the Pandas DataFrame format in Table 1. In this dataset, there are an equal number of 80 data points for each of the AM, SS, and IM phases. The dataset consists of 240 instances and 6 features, including 1 categorical feature designating different phases (AM, SS, or IM) and 5 numeric features representing valence electron concentration (VEC), the difference in electronegativity difference (Δχ), atomic size difference (δ), mixing enthalpy (ΔHmix), and mixing entropy (ΔSmix). Table 1. A glimpse of Pandas displaying random 5 instances of the data employed in this study. The units for ΔHmix and ΔSmix are kJmol−1 and JK−1mol−1, respectively The labels were encoded into integers and assigned the values 0 for AM, 1 for SS, and 2 for IM phases, respectively, to denote the alloy phases in the ANN model. The formulas provided below are used to compute the numeric values for the five features [37–40]. 𝑉𝐸𝐶 = ∑ 𝑐𝑖𝑉𝐸𝐶𝑖 𝑛 𝑖=1 (1) ∆𝜒 = √∑ 𝑐𝑖 (𝜒𝑖 − 𝜒)2𝑛 𝑖=1 (2) MFB. Noor et al. /Future Sustainability February 2024| Volume 02 | Issue 01 | Pages 47-58 49 𝛿 = 100 𝑋 √∑ 𝑐𝑖 (1 − 𝑟𝑖 𝑟⁄ )2𝑛 𝑖=1 (3) Δ𝐻𝑚𝑖𝑥 = ∑ 4𝐻𝑖𝑗 𝑛 𝑖=1,𝑖<𝑗 𝑐𝑖𝑐𝑗 (4) Δ𝑆𝑚𝑖𝑥 = −𝑅 ∑ 𝑐𝑖 𝑙𝑛𝑐𝑖 𝑛 𝑖=1 (5) Here, ci (where 0 < ci < 1) denotes the atomic concentrations of the i-th element, while n represents the total number of components within a HEA. VECi and ri signifies the atomic concentration, VEC, and atomic radius of each species of the i- th element and R denotes the gas constant. Using Miedema's model, the enthalpy of atomic pairs, is calculated [41]. 𝜒 and 𝑟 refer for the weighted Pauling electronegativity and atomic radius, respectively, written as follows. 𝜒 = ∑ 𝑐𝑖𝜒𝑖 𝑛 𝑖=1 (6) 𝑟 = ∑ 𝑐𝑖𝑟𝑖 𝑛 𝑖=1 (7) The data undergo preprocessing for feature values before training the architecture. Using the Pandas library [42], these values are normalized and scaled them between 0 and 1, as shown below: 𝑋𝑛𝑒𝑤 = 𝑋𝑖 −𝑋𝑚𝑖𝑛,𝑖 𝑋𝑚𝑎𝑥,𝑖− 𝑋𝑚𝑖𝑛,𝑖 (8) Where, Xnew represents to the normalized feature, Xi refers to the actual feature information, Xmin,i and Xmax,i stand for the minimum and maximum values respectively. Through this normalization procedure, dimensionless numeric features are generated, which ensures effective uniform numeric scaling and consistent treatment of all features. A layer of neurons performs computational task with the ANN model. the output of each neuron within the hidden layer is denoted by aj, as expressed in the following equation. 𝑎𝑗 = ∑ 𝑥𝑖 𝑛 𝑖=1 𝑊𝑖𝑗 + 𝑏𝑗 (9) Where 𝑏𝑗 designates the bias coefficients and 𝑊𝑖𝑗 corresponds to the weights of each input parameter 𝑥𝑖 . Google's TensorFlow [43] is a well-recognized library in this field and based on that, the machine learning neural network architecture is used. Figure 1 shows the architecture used in this study which encompasses backpropagation functions and several hidden layers. Eq. (9) is used to calculate the value of 𝑎𝑗 for each neuron which is related with connection- specific weights. The activation function takes it as an input value. Five features of parameters are encompassed as input and the three neurons denoting different phases are included in the output layer. The leaky Rectified linear unit (LReLU) activation function has been applied within the hidden layers. The Rectified Linear Unit (ReLU) [44], illustrated in Figure 2, is a common and popular activation function in Neural Networks (NNs). By compelling precise tuning of the learning rate, it can extend past predefined bounds during the NN training process because of its easiness and subsequent reduction in training computation time. Due to this issue, the activation function remains inactive for the neurons within the negative region during the training process. By assigning a small constant value, like 0.2, to the negative region, Leaky ReLU (LReLU) [45], solves this, as presented in Figure 2. Three nodes of the model’s output layer represent the alloy phases which receive input from the final hidden layer and then these nodes employ activation functions to predict the phase. The broadly used activation function, SoftMax, illustrated in Figure 3, was utilized in the output layer for this classification. The probability of the input belonging to different classes is illustrated by this normalized exponential function illustrates. Generally, the SoftMax function [46] is expressed as: 𝜎(𝑦𝑖 ′) = 𝑒 𝑦𝑖 ′ ∑ 𝑒𝑦𝑖 ′ 𝑛 𝑖=1 (10) Here, 𝜎(𝑦𝑖 ′) denotes the subsequent probability and the prediction vector is referred as 𝑦𝑖 ′ . Consequently, the model's output was contrasted with the target labels to assess network's error. Here, cross-entropy [47] is utilized as cost (loss) function, which resembles to the negative logarithm of probability, and the following equation represents the function. 𝐻𝑦(𝑦 ′) = − ∑ 𝑦 log(𝜎(𝑦𝑖 ′))𝑛 𝑖=1 Here, 𝑦 ′ refer to the prediction and y stand for one of the three target vectors. The neural network’s final output is converted into a probability, and then, using cross-entropy, it is utilized to calculate the loss. The deviation between the actual distribution and the model's expected output distribution is computed by Cross-entropy. Afterward, the gradient descent algorithm is used, utilizing a learning rate of 0.013, to convey back this error through the network. The weights and bias are initiated randomly in the beginning of the training process. The loss function is minimized by adjusting them at each epoch. The accuracy of the network is quantified by the number of successful determinations of the target. The hyperparameter configuration for the artificial neural network (ANN) model encompassed a range of values and architectures, contributing to the systematic tuning process. For the number of hidden layers, the model was experimented with settings ranging from 3 to 5 layers, exploring the impact of network depth. Regularization techniques, such as L1 and L2, were introduced within the ranges of 0.01 to 0.025, allowing for the assessment of their influence on model generalization. Similarly, the learning rate, an essential factor in optimization, was varied between 0.001 and 0.013 to identify the optimal balance between convergence and avoiding local minima. Dropout rates, a regularization method to mitigate overfitting, were adjusted within the range of 0.1 to 0.4. Different batch sizes, ranging from 8 to 120, were examined to evaluate their effect on model training efficiency and convergence. These diverse configurations and their corresponding results constituted a comprehensive exploration of the ANN's hyperparameters to achieve the best predictive performance and phase wise accuracy. After training the model with the training dataset, a distinct, unseen test set, which was preserved during training, was used to evaluate the model. The feedback from validation provides guidance to adjust the parameter. The best model is chosen after demonstration of the optimum validation result. The hyperparameters are described in Table 2 and the test set is used to evaluate this model. MFB. Noor et al. /Future Sustainability February 2024| Volume 02 | Issue 01 | Pages 47-58 50 Figure 2. Rectified linear unit (ReLU) and leaky ReLU (LReLU) 3. Results and discussion 3.1 Data analysis Comprehending the dataset of 240 records is an essential preliminary stage before applying a machine learning algorithm. Then, we generate a scatter matrix plot using the Seaborn package and then compute the correlation matrix of the features using Pandas library. These two matrices aid in comprehending feature relationships within the curated HEA dataset and offer both qualitative and quantitative interconnection estimates. Our prediction pertains to the phases, with a specific emphasis on five quantitative features of the HEA compositions. To visualize the data, we employ a 5 × 5 scatter matrix plot, as depicted in Figure 4. The diagonal subfigures illustrate histograms of phase distributions, considering individual utilization of each of the five features. All histograms within subfigures overlap, suggesting no isolated feature for complete alloy phase classification. Correlations among the five features influence phase selection in HEAs, which is evident in off-diagonal subfigures of Figure 4. Figure 3. SoftMax function Figure 1. Illustration of the artificial neural network (ANN) architecture designed for predicting phase formations in High-Entropy Alloys (HEAs). For clarity, only five neurons (illustrated as circles) within the hidden layers are depicted. Empty squares symbolize input features and output values. The AM, SS, and IM phases are encoded as vectors 0, 1, and 2, respectively. MFB. Noor et al. /Future Sustainability February 2024| Volume 02 | Issue 01 | Pages 47-58 51 Table 2. Hyperparameters of ANN Hyperparameter Value Number of hidden layers 5 Number of hidden neurons 150 neurons Regularization L1: 0.025, L2: 0.01 Activation function LeakyReLU (alpha= 0.1) Dropout rate 0.4 Batch size 65 Learning rate 0.013 (Adam optimizer) Epochs 100 Loss function Categorical Crossentropy We also calculate the Pearson correlation coefficient for features x and y to provide a quantitative description of their correlations [48]. 𝑟𝑥𝑦 = 1 𝑛−1 ∑ (𝑥𝑖−𝑥)(𝑦𝑖− 𝑦)𝑛 𝑖=1 𝑆𝑥𝑆𝑦 (12) Here, x and y represent the mean values of two features, while 𝑆𝑥 and 𝑆𝑦 are their respective standard deviations. Correlation values can vary between -1 and 1, indicating negative or positive relationships. The computed correlation matrix elements are presented in Figure 5. Centering on the correlation between two distinct features, the matrix elements range from -0.61 to 0.72. Out of the ten distinct correlation matrix elements, seven exhibit negativity, while the remaining are positive. This outcome also exhibits resemblance to a prior study [1]. In the correlation matrix and scatter plot, of electronegativity difference (Δχ) and atomic size difference (δ), a positive correlation is observed, meaning that Δχ tends to increase with higher values of δ. Additionally, both Δχ and δ show negative correlations with valence electron concentration (VEC) and mixing enthalpy (ΔHmix). In general, the correlation matrix elements exhibit moderate magnitudes, allowing all five features to be employed collectively as input for our neural network architecture. Figure 4. The scatter plots presented in the off-diagonal sections reveal the correlations among the values of the five distinct features. Within the diagonal panels, the histograms illustrate the distributions of the three phases based on the five features. Each phase is represented using varying shapes and colors: a yellow circle signifies amorphous (AM), a blue diamond represents solid solution (SS), and a red square denotes intermetallic (IM). MFB. Noor et al. /Future Sustainability February 2024| Volume 02 | Issue 01 | Pages 47-58 52 Figure 5. Correlation matrix heatmap of the five features 3.2 Artificial Neural Network (ANN) Results The development of all Python network models is accomplished using the Keras framework with a TensorFlow backend for ANNs. Subsequently, the hyperparameter values are adjusted and the optimal model and hyperparameter settings for the ANN model are determined through a 3-fold cross-validation process as depicted in Figure 6. The best parameter resulted in an average cross-validation accuracy of 86.46%, as detailed in Table 2. Afterwards, the ANN architecture is employed to train on 80% of the developed balanced dataset and then applied to test the remaining 20% of the dataset. This process is visualized in Figure 7(a), (b) and (c) illustrating the progression of training loss and validation loss across number of epochs for three separate folds. Figure 6. Training and testing process of the ANN model Notably, both the training loss and validation loss curves exhibit a similar trajectory, demonstrating that the optimization algorithm consistently updates the weights of hidden layer neurons to minimize the loss and enhance the learning process at each epoch without overfitting the data. Across the three-fold training set, the loss converges to 1 after 40 epochs, maintaining a consistent trend thereafter. Furthermore, it's important to highlight that when the model is evaluated on the final set of data, there is a noticeable decrease in the loss value. This reduction brings the loss down to 0.5 after approximately 30 epochs of the training process. This trend is visually represented in Figure 7(d), where the curve illustrating the loss for the final test set which closely resembles the trajectory observed during the training process. This indicates that the model's performance on the test data doesn't show any signs of overfitting. This consistent reduction in loss and the convergence of the curves emphasize the model's ability to generalize well and perform effectively on new, unseen data for each of the HEA phases. In Figure 8(a), a visual representation is provided for the three- fold cross-validation process that was employed to assess the model’s performance, showcasing the accuracy achieved for each individual fold along with average accuracy while training the data. Notably, the calculated average validation accuracy across all three folds was determined to be 86.46%. This approach of three-fold cross-validation ensures proper evaluation of the model's effectiveness across different subsets of the data. Additionally, the prediction of phase-wise accuracy on the training datasets is depicted in Figure 8(b). Remarkably, the final validation set attains a prediction rate nearing 83.33%, affirming the strong performance of the developed ANN model and its favorable generalization capabilities. MFB. Noor et al. /Future Sustainability February 2024| Volume 02 | Issue 01 | Pages 47-58 53 Figure 9 illustrates the accuracy comparison between the training and validation processes for the final test set, revealing a positive correlation between them and overall accuracy improvement with epochs. Observing the training outcomes depicted in Figure 9, it's evident that the training set accuracy incrementally rises as iterations progress. After approximately 30 epochs, the accuracy stabilizes, suggesting effective convergence of the model. The effectiveness of the ANN model in predicting each of the phases is displayed in Figure 10. In contrast to some other studies that typically report overall accuracy, it is equally crucial to highlight phase- wise accuracy to illustrate the model's competence and its ability to predict various phases effectively. The results underscore the model's proficiency in accurately predicting distinct phases, and notably, the phase-wise accuracy reaches impressive levels, with AM achieving 86.67%, SS reaching 81.25%, and IM attaining 82.35%. These results exemplify the ANN model's effectiveness in predictive performance, particularly when it is trained on a balanced dataset for each phase. Using Micro-F1 to evaluate the prediction outcomes, the test set is employed to validate the effectiveness of the ANN model. The necessary equations for calculating the Micro-F1 score are provided below [5,49]:Using Micro-F1 to evaluate the prediction outcomes, the test set is employed to validate the effectiveness of the ANN model. The necessary equations for calculating the Micro-F1 score are provided below [5,49]: 𝑅𝑒𝑐𝑎𝑙𝑙𝑚𝑖𝑐𝑟𝑜 = ∑ 𝑇𝑃𝑖 𝑛 𝑖=1 ∑ 𝑇𝑃𝑖 𝑛 𝑖=1 + ∑ 𝐹𝑁𝑖 𝑛 𝑖=1 (13) 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛𝑚𝑖𝑐𝑟𝑜 = ∑ 𝑇𝑃𝑖 𝑛 𝑖=1 ∑ 𝑇𝑃𝑖 𝑛 𝑖=1 + ∑ 𝐹𝑃𝑖 𝑛 𝑖=1 (14) 𝐹1𝑚𝑖𝑐𝑟𝑜 = 2 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛𝑚𝑖𝑐𝑟𝑜∗𝑅𝑒𝑐𝑎𝑙𝑙𝑚𝑖𝑐𝑟𝑜 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛𝑚𝑖𝑐𝑟𝑜+𝑅𝑒𝑐𝑎𝑙𝑙𝑚𝑖𝑐𝑟𝑜 (15) Here, true positive (𝑇𝑃𝑖) means positive cases correctly identified as positive cases of the i element, false positive (𝐹𝑃𝑖) means negative cases is incorrectly identified positive cases of the i element, true positive (𝐹𝑁𝑖) means positive cases is incorrectly identified negative cases of the i element. 𝑅𝑒𝑐𝑎𝑙𝑙𝑚𝑖𝑐𝑟𝑜 measures the accuracy of correctly predicting actual positive samples within the sample space. 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛𝑚𝑖𝑐𝑟𝑜 quantifies the accuracy of forecasting positive predictions. F1micro is the aggregated average that considers both 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛𝑚𝑖𝑐𝑟𝑜 and 𝑅𝑒𝑐𝑎𝑙𝑙𝑚𝑖𝑐𝑟𝑜. The Micro F1 Score on final test set is 0.83. To assess the predictive performance of the ANN model for each of the HEA classes within the dataset, a confusion matrix was generated using a testing dataset comprising 48 samples, as illustrated in Figure 11. 0 1 2 3 4 5 6 7 0 50 100 L o ss Epochs Training Loss Validation Loss 0 1 2 3 4 5 6 7 0 50 100 L o ss Epochs Training Loss Validation Loss 0 1 2 3 4 5 6 7 0 50 100 L o ss Epochs Training Loss Validation Loss 0 1 2 3 4 0 20 40 60 L o ss Epochs Training Loss Validation Loss Figure 7. Comparing the training and validation loss of the ANN model for a) Fold 1, b) Fold 2, c) Fold 3, and d) Final test set. MFB. Noor et al. /Future Sustainability February 2024| Volume 02 | Issue 01 | Pages 47-58 54 Figure 9. Comparing the accuracy of the ANN model between the training set and final validation set Figure 10. Comparing phase-wise accuracy and average accuracy found for the final test set The confusion matrix indicates notably high precision and recall values across all three classes of HEA, affirming the model's robustness and generalization. The variance between the cross-validation accuracy and the confusion matrix accuracy arises because the former represents the average of all validation accuracies, while the latter directly reports accuracy for the testing dataset that has been kept separate during the model development process. The developed ANN model aims to predict the phase of previously unseen data, and its performance was benchmarked against other alternative methods, as illustrated in Figure 12. Among the various machine learning algorithms assessed for accuracy, our developed ANN model demonstrates the highest accuracy of 83.33%. Notably, while Islam et al. [1] and Krishna et al. [32] also utilized an ANN model, their datasets exhibit unequal proportions of data across different HEA phases. In contrast, our ANN model maintains consistency by employing the same number of instances for AM, IM, and SS phases, and this uniformity contributes to the model's robust performance, allowing it to achieve the noteworthy accuracy of 83.33%, surpassing the accuracy of other methods [5,33,50], including those that employed differing dataset compositions. The developed ANN model aims to predict the phase of previously unseen data, and its performance was benchmarked against other alternative methods, as illustrated in Figure 12. Among the various machine learning algorithms assessed for accuracy, our developed ANN model demonstrates the highest accuracy of 83.33%. Figure 11. Confusion matrices of ANN model used in amorphous, solid solution, and intermetallic phase prediction on Final Test Set. (b) (a) Figure 8. Comparing (a) fold-wise accuracy and (b) average phase-wise accuracy found for the cross-validation data employing the ANN model. MFB. Noor et al. /Future Sustainability February 2024| Volume 02 | Issue 01 | Pages 47-58 55 Figure 12. Evaluating the Precision of Machine Learning Algorithms Notably, while Islam et al. [1] and Krishna et al. [32] also utilized an ANN model, their datasets exhibit unequal proportions of data across different HEA phases. In contrast, our ANN model maintains consistency by employing the same number of instances for AM, IM, and SS phases, and this uniformity contributes to the model's robust performance, allowing it to achieve the noteworthy accuracy of 83.33%, surpassing the accuracy of other methods [5,33,50], including those that employed differing dataset compositions. 3.3 Relative feature impact assessment The ANN architecture was used to assess the relative significance of the five input features used to train the model. To investigate this, a series of five experiments were carried out, with each experiment systematically omitting one feature while keeping the remaining four. This procedure entailed retraining the model and making predictions in order to thoroughly investigate the impact on the test set accuracy. The results of these experiments can be seen in Figure 13, which depicts the decline in accuracy across the five scenarios mentioned above. This trend highlights an important observation: removing any of the five features consistently resulted in a decrease in the accuracy of the model, highlighting the significant influence that each feature has on test accuracy [5,25]. When compared to other features, differences in atomic sizes and the concentration of valence electrons have a greater influence on the accuracy of the model. Notably, it has been determined that the key design parameters derived from the current ANN approach—the atomic size difference and the valence electron concentration—align closely with the preexisting parametric guidelines for HEA phase formation. Figure 13. Effect on the test set accuracy upon the removal of individual features. This convergence highlights an intriguing correlation, confirming the developed ANN method's reliability. Consistent with Hume-Rothery principles, the atomic size difference plays a crucial role in phase formation of HEA, especially in case of solid solution (SS) phase [5,38,51,52]. Furthermore, the Hume-Rothery principles show that the number of valence electrons per atom is critical in determining the stability of solid solutions in metal binary systems [5,52–54], and this stability in the mentioned systems hinges on electron density, specifically where peaks in the density of states occur, coinciding with the point where the Fermi sphere intersects the Brillouin zone boundary. As a result, the structure becomes stable at a specific electron concentration level. While atomic radius and MFB. Noor et al. /Future Sustainability February 2024| Volume 02 | Issue 01 | Pages 47-58 56 electronegativity differences are not always conclusive predictors of outcomes, they're both highly indicative parameters in the design of HEA compositions [51–54], underscoring the significance of considering electronic structure alongside other material properties when designing HEAs. 4. Conclusion In this study, a carefully developed ANN model was introduced to address the persistent challenge of imbalanced datasets when predicting phase selection in HEAs. Through a rigorous optimization process encompassing various hyperparameters, the ANN model was developed using a balanced dataset, resulting in excellent predictive performance. Using a three-fold cross-validation strategy, the model's effectiveness was carefully evaluated. The results showed an impressive average validation accuracy of 86.46% across all three folds and eventually led to a high prediction rate of nearly 83.33% on the final test set, highlighting the model's robustness and capacity for generalization. This study also emphasized the importance of phase-wise accuracy, with the ANN model achieving remarkable accuracy levels for the studied HEA phases (86.67% for AM, 81.25% for SS, and 82.35% for IM). A detailed confusion matrix analysis also confirmed the model's robustness across all classes, highlighting its precision and recall balance, while comparison against alternative methods demonstrated its superior accuracy, and the Micro-F1 score validated the model's effectiveness with a score of 0.83 on the final test set. Notably, this was accomplished by maintaining dataset balance for each phase, which distinguished this approach from previous studies that frequently used imbalanced datasets and didn't mention phase-wise accuracy which is very important to showcase the model's ability for generalization. Furthermore, the study investigated the relative importance of input features, identifying that atomic size difference and valence electron concentration played critical roles in test accuracy, in line with established guidelines for HEA phase formation and reinforcing the developed ANN method's reliability. It should also be noted that including more data for each phase can contribute to even higher model performance, providing an exciting potential for further predictive accuracy improvement. Overall, this study not only provides an effective solution to an existing issue in materials science, but it also provides critical insights into the impact of physical parameters on phase selection, making it invaluable for future alloy design and engineering efforts. Ethical issue The authors are aware of and comply with best practices in publication ethics, specifically with regard to authorship (avoidance of guest authorship), dual submission, manipulation of figures, competing interests, and compliance with policies on research ethics. The authors adhere to publication requirements that the submitted work is original and has not been published elsewhere. Data availability statement The manuscript includes existing data, and additional data that support the findings of this study are openly available in High-Entropy-Alloy-Phase-Prediction-Using-Balanced- Dataset at https://github.com/fahel-bin-noor/High-Entropy- Alloy-Phase-Prediction-Using-Balanced-Dataset. Conflict of interest The authors declare no potential conflict of interest. References [1] N. Islam, W. Huang, H.L. Zhuang, Machine learning for phase selection in multi-principal element alloys, Computational Materials Science. 150 (2018) 230– 235. https://doi.org/10.1016/j.commatsci.2018.04.003. [2] D.B. Miracle, O.N. Senkov, A critical review of high entropy alloys and related concepts, Acta Materialia. 122 (2017) 448–511. https://doi.org/10.1016/j.actamat.2016.08.081. [3] J.-W. Yeh, S.-K. Chen, S.-J. Lin, J.-Y. Gan, T.-S. Chin, T.-T. Shun, C.-H. Tsau, S.-Y. Chang, Nanostructured High- Entropy Alloys with Multiple Principal Elements: Novel Alloy Design Concepts and Outcomes, Advanced Engineering Materials. 6 (2004) 299–303. https://doi.org/10.1002/adem.200300567. [4] B. Cantor, I.T.H. Chang, P. Knight, A.J.B. Vincent, Microstructural development in equiatomic multicomponent alloys, Materials Science and Engineering: A. 375–377 (2004) 213–218. https://doi.org/10.1016/j.msea.2003.10.257. [5] W. Zhu, W. Huo, S. Wang, X. Wang, K. Ren, S. Tan, F. Fang, Z. Xie, J. Jiang, Phase formation prediction of high-entropy alloys: a deep learning study, Journal of Materials Research and Technology. 18 (2022) 800– 809. https://doi.org/10.1016/j.jmrt.2022.01.172. [6] Y. Zhang, T.T. Zuo, Z. Tang, M.C. Gao, K.A. Dahmen, P.K. Liaw, Z.P. Lu, Microstructures and properties of high- entropy alloys, Progress in Materials Science. 61 (2014) 1–93. https://doi.org/10.1016/j.pmatsci.2013.10.001. [7] B. Gludovatz, A. Hohenwarter, D. Catoor, E.H. Chang, E.P. George, R.O. Ritchie, A fracture-resistant high- entropy alloy for cryogenic applications, Science. 345 (2014) 1153–1158. https://doi.org/10.1126/science.1254581. [8] W. Huo, F. Fang, X. Liu, S. Tan, Z. Xie, J. Jiang, Remarkable strain-rate sensitivity of nanotwinned CoCrFeNi alloys, Applied Physics Letters. 114 (2019) 101904. https://doi.org/10.1063/1.5088921. [9] E.G. Reineke, O.T. Inal, Crystallization behavior of amorphous Ni50Nb50 on continuous heating, Materials Science and Engineering. 57 (1983) 223– 231. https://doi.org/10.1016/0025-5416(83)90212- 4. [10] X. Chang, M. Zeng, K. Liu, L. Fu, Phase Engineering of High-Entropy Alloys, Advanced Materials. 32 (2020) 1907226. https://doi.org/10.1002/adma.201907226. [11] W. Huo, X. Liu, S. Tan, F. Fang, Z. Xie, J. Shang, J. Jiang, Ultrahigh hardness and high electrical resistivity in nano-twinned, nanocrystalline high-entropy alloy films, Applied Surface Science. 439 (2018) 222–225. https://doi.org/10.1016/j.apsusc.2018.01.050. [12] Z. Li, K.G. Pradeep, Y. Deng, D. Raabe, C.C. Tasan, Metastable high-entropy dual-phase alloys overcome https://github.com/fahel-bin-noor/High-Entropy-Alloy-Phase-Prediction-Using-Balanced-Dataset https://github.com/fahel-bin-noor/High-Entropy-Alloy-Phase-Prediction-Using-Balanced-Dataset MFB. Noor et al. /Future Sustainability February 2024| Volume 02 | Issue 01 | Pages 47-58 57 the strength–ductility trade-off, Nature. 534 (2016) 227–230. https://doi.org/10.1038/nature17981. [13] W. Huo, H. Zhou, F. Fang, X. Hu, Z. Xie, J. Jiang, Strain- rate effect upon the tensile behavior of CoCrFeNi high-entropy alloys, Materials Science and Engineering: A. 689 (2017) 366–369. https://doi.org/10.1016/j.msea.2017.02.077. [14] W. Huo, H. Shi, X. Ren, J. Zhang, Microstructure and Wear Behavior of CoCrFeMnNbNi High-Entropy Alloy Coating by TIG Cladding, Advances in Materials Science and Engineering. 2015 (2015) e647351. https://doi.org/10.1155/2015/647351. [15] Y. Shi, L. Collins, R. Feng, C. Zhang, N. Balke, P.K. Liaw, B. Yang, Homogenization of AlxCoCrFeNi high- entropy alloys with improved corrosion resistance, Corrosion Science. 133 (2018) 120–131. https://doi.org/10.1016/j.corsci.2018.01.030. [16] S. Wang, W. Huo, F. Fang, Z. Xie, J.K. Shang, J. Jiang, High entropy alloy/C nanoparticles derived from polymetallic MOF as promising electrocatalysts for alkaline oxygen evolution reaction, Chemical Engineering Journal. 429 (2022) 132410. https://doi.org/10.1016/j.cej.2021.132410. [17] Md.F. bin Noor, B. Mallick, A. Habib, Heat storage system: A modern way to reuse and recycle energy to reduce thermal pollution, in: International Conference on Mechanical, Industrial and Energy Engineering, 2018. [18] A.S. Lakhnot, R.A. Panchal, J. Datta, V. Mahajani, K. Bhimani, R. Jain, D. Datta, N. Koratkar, Intercalation Hosts for Multivalent-Ion Batteries, Small Structures. 4 (2023) 2200290. https://doi.org/10.1002/sstr.202200290. [19] J. Datta, N. Koratkar, D. Datta, Open-tunneled oxides as intercalation host for multivalent ion (Ca and Al) batteries: A DFT study, (2023). https://doi.org/10.48550/arXiv.2303.12301. [20] Z.U. Mahmud, S. Karmakar, A. Haque, K.C. Ghosh, A study of fabrication and characterization of NaxMnO2 as a cathode material for sodium-ion battery, MRS Advances. (2023). https://doi.org/10.1557/s43580- 023-00611-4. [21] M.S. Uddin, R.A. Mayanovic, M. Benamara, On the synthesis and characterization of bimagnetic CoO/NiFe2O4 heterostructured nanoparticles, AIP Advances. 13 (2023) 025314. https://doi.org/10.1063/9.0000561. [22] M.T. Islam, M.S. Rabbi, M.S. Uddin, Noise reduction of helicopter rotor blades by using spoiler, AIP Conference Proceedings. 2121 (2019) 040014. https://doi.org/10.1063/1.5115885. [23] N. Yasmin, M.F.B. Noor, T. Besara, Structure and Magnetism of the New Cage-structured Compound HfMn2Zn20, (2023). https://doi.org/10.48550/arXiv.2306.01146. [24] L. Wang, P. Li, W. Zhang, F. Wan, J. Wu, L. Yong, X. Liu, Prediction of phase selection of amorphous alloys and high entropy alloys by artificial neural network, Computational Materials Science. 223 (2023) 112129. https://doi.org/10.1016/j.commatsci.2023.112129. [25] S.Y. Lee, S. Byeon, H.S. Kim, H. Jin, S. Lee, Deep learning-based phase prediction of high-entropy alloys: Optimization, generation, and explanation, Materials & Design. 197 (2021) 109260. https://doi.org/10.1016/j.matdes.2020.109260. [26] B. Mallick, R. Das, S. Banik, Md.F. bin Noor, A. Habib, Performance Enhancement of an Automobile Radiator by Using a Nozzle Arrangement, International Journal of Innovative Technology and Exploring Engineering. X (2019). [27] F. Aydin, R. Durgut, Estimation of wear performance of AZ91 alloy under dry sliding conditions using machine learning methods, Transactions of Nonferrous Metals Society of China. 31 (2021) 125– 137. https://doi.org/10.1016/S1003- 6326(20)65482-6. [28] J. Datta, D. Datta, V. Sharma, Transferable and Robust Machine Learning Model for Predicting Stability of Si Anodes for Multivalent Cation Batteries, J Mater Sci. 58 (2023) 11085–11099. https://doi.org/10.1007/s10853-023-08705-y. [29] K.G. Naik, B.S. Vishnugopi, J. Datta, D. Datta, P.P. Mukherjee, Electro-Chemo-Mechanical Challenges and Perspective in Lithium Metal Batteries, Applied Mechanics Reviews. 75 (2023). https://doi.org/10.1115/1.4057039. [30] R. Machaka, Machine learning-based prediction of phases in high-entropy alloys, Computational Materials Science. 188 (2021) 110244. https://doi.org/10.1016/j.commatsci.2020.110244. [31] U. Bhandari, Md.R. Rafi, C. Zhang, S. Yang, Yield strength prediction of high-entropy alloys using machine learning, Materials Today Communications. 26 (2021) 101871. https://doi.org/10.1016/j.mtcomm.2020.101871. [32] Y.V. Krishna, U.K. Jaiswal, R.M. R, Machine learning approach to predict new multiphase high entropy alloys, Scripta Materialia. 197 (2021) 113804. https://doi.org/10.1016/j.scriptamat.2021.113804. [33] W. Huang, P. Martin, H.L. Zhuang, Machine-learning phase prediction of high-entropy alloys, Acta Materialia. 169 (2019) 225–236. https://doi.org/10.1016/j.actamat.2019.03.012. [34] S. Guo, C.T. Liu, Phase stability in high entropy alloys: Formation of solid-solution phase or amorphous phase, Progress in Natural Science: Materials International. 21 (2011) 433–446. https://doi.org/10.1016/S1002-0071(12)60080-X. [35] C.E. Precker, A. Gregores Coto, S. Muíños Landín, Materials for Design Open Repository. High Entropy Alloys, (2021). https://doi.org/10.5281/zenodo.6403257. [36] R. Machaka, Dataset for High-Entropy Alloys Phases, 3 (2021). https://doi.org/10.17632/7fhwrgfh2s.3. [37] S. Fang, X. Xiao, L. Xia, W. Li, Y. Dong, Relationship between the widths of supercooled liquid regions and bond parameters of Mg-based bulk metallic glasses, Journal of Non Crystalline Solids. 321 (2003) 120– 125. https://doi.org/10.1016/S0022- 3093(03)00155-8. MFB. Noor et al. /Future Sustainability February 2024| Volume 02 | Issue 01 | Pages 47-58 58 [38] Y. Zhang, Y. Zhou, J. Lin, G. Chen, P. Liaw, Solid‐ Solution Phase Formation Rules for Multi‐component Alloys, Advanced Engineering Materials. 10 (2008) 534–538. https://doi.org/10.1002/adem.200700240. [39] C.T. Liu, Physical metallurgy and mechanical properties of ductile ordered alloys (Fe, Co, Ni)3 V, International Metals Reviews. 29 (1984) 168–194. https://doi.org/10.1179/imtr.1984.29.1.168. [40] J.H. Zhu, P.K. Liaw, C.T. Liu, Effect of electron concentration on the phase stability of NbCr2-based Laves phase alloys, Materials Science and Engineering: A. 239–240 (1997) 260–264. https://doi.org/10.1016/S0921-5093(97)00590-X. [41] Classification of Bulk Metallic Glasses by Atomic Size Difference, Heat of Mixing and Period of Constituent Elements and Its Application to Characterization of the Main Alloying Element, (n.d.). https://www.jstage.jst.go.jp/article/matertrans/46/ 12/46_12_2817/_article (accessed August 25, 2023). [42] W. McKinney, Python for Data Analysis: Data Wrangling with Pandas, NumPy, and IPython, second edition, O’Reilly Media, Inc., 2017. [43] M. Abadi, P. Barham, J. Chen, Z. Chen, A. Davis, J. Dean, M. Devin, S. Ghemawat, G. Irving, M. Isard, M. Kudlur, J. Levenberg, R. Monga, S. Moore, D.G. Murray, B. Steiner, P. Tucker, V. Vasudevan, P. Warden, M. Wicke, Y. Yu, X. Zheng, TensorFlow: a system for large-scale machine learning, in: Proceedings of the 12th USENIX Conference on Operating Systems Design and Implementation, USENIX Association, USA, 2016: pp. 265–283. [44] M.M. Lau, K. Hann Lim, Review of Adaptive Activation Function in Deep Neural Network, in: 2018 IEEE- EMBS Conference on Biomedical Engineering and Sciences (IECBES), 2018: pp. 686–690. https://doi.org/10.1109/IECBES.2018.8626714. [45] A.L. Maas, Rectifier Nonlinearities Improve Neural Network Acoustic Models, in: 2013. https://www.semanticscholar.org/paper/Rectifier- Nonlinearities-Improve-Neural-Network- Maas/367f2c63a6f6a10b3b64b8729d601e69337ee3 cc (accessed August 30, 2023). [46] E. Byvatov, U. Fechner, J. Sadowski, G. Schneider, Comparison of support vector machine and artificial neural network systems for drug/nondrug classification, J Chem Inf Comput Sci. 43 (2003) 1882–1889. https://doi.org/10.1021/ci0341161. [47] L. Devroye, L. Györfi, G. Lugosi, A probabilistic theory of pattern recognition, 3. print, Springer, New York, NY, 2008. [48] J.D. Kelleher, B.M. Namee, A. D’Arcy, Fundamentals of Machine Learning for Predictive Data Analytics, (n.d.). [49] P. Vateekul, M. Kubat, Fast Induction of Multiple Decision Trees in Text Categorization from Large Scale, Imbalanced, and Multi-label Data, in: IEEE Computer Society, 2009: pp. 320–325. https://doi.org/10.1109/ICDMW.2009.94. [50] A. Oñate, J.P. Sanhueza, D. Zegpi, V. Tuninetti, J. Ramirez, C. Medina, M. Melendrez, D. Rojas, Supervised machine learning-based multi-class phase prediction in high-entropy alloys using robust databases, Journal of Alloys and Compounds. 962 (2023) 171224. https://doi.org/10.1016/j.jallcom.2023.171224. [51] A. Takeuchi, A. Inoue, Classification of Bulk Metallic Glasses by Atomic Size Difference, Heat of Mixing and Period of Constituent Elements and Its Application to Characterization of the Main Alloying Element, Materials Transactions. 46 (2005) 2817–2829. https://doi.org/10.2320/matertrans.46.2817. [52] M.G. Poletti, L. Battezzati, Electronic and thermodynamic criteria for the occurrence of high entropy alloys in metallic systems, Acta Materialia. 75 (2014) 297–306. https://doi.org/10.1016/j.actamat.2014.04.033. [53] U. Mizutani, H. Sato, T.B. Massalski, The original concepts of the Hume-Rothery rule extended to alloys and compounds whose bonding is metallic, ionic, or covalent, or a changing mixture of these, Progress in Materials Science. 120 (2021) 100719. https://doi.org/10.1016/j.pmatsci.2020.100719. [54] S. Guo, C. Ng, J. Lu, C.T. Liu, Effect of valence electron concentration on stability of fcc or bcc phase in high entropy alloys, Journal of Applied Physics. 109 (2011) 103505. https://doi.org/10.1063/1.3587228. This article is an open-access article distributed under the terms and conditions of the Creative Commons Attribution (CC BY) license (https://creativecommons.org/licenses/by/4.0/). https://creativecommons.org/licenses/by/4.0/ https://creativecommons.org/licenses/by/4.0/