Copyright © CC-BY-NC 2020, CRIBFB |BJMSR Bangladesh Journal of Multidisciplinary Scientific Research; Vol. 2, No. 1; 2020 ISSN 2687-850X E-ISSN 2687-8518 Published by Centre for Research on Islamic Banking & Finance and Business, USA 48 40 A Review on Artificial Neural Networks and its’ Applicability Mustafa Nizamul Aziz Senior Lecturer East West University Dhaka, Bangladesh E-mail: mustafa.nizamul@gmail.com Abstract The field of artificial neural networks (ANN) started from humble beginnings in the 1950s but got attention in the 1980s. ANN tries to emulate the neural structure of the brain, which consists of several thousand cells, neuron, which is interconnected in a large network. This is done through artificial neurons, handling the input and output, and connecting to other neurons, creating a large network. The potential for artificial neural networks is considered to be huge, today there are several different uses for ANN, ranging from academic research in such fields as mathematics and medicine to business-based purposes and sports prediction. The purpose of this paper is to give words to artificial neural networks and to show its applicability. Documents analysis was used here as the data collection method. The paper figured out network structures, steps for constructing an ANN, architectures, and learning algorithms. Keywords: Artificial Neural Networks, Artificial Neural Network Architectures, Artificial Neurons. 1. Introduction As early as 1943 a model was created by Warren McCulloch and Walter Pitts called McCulloch-Pitts neuron which tried to mimic the structure of a biological neural network. This model was divided into two parts, one consisting of a summation of weighted input and the other consisting of an output function of the sum. The neural network model created consisted of several binary neurons interconnected in a large network (Yegnanarayana, 2009). Artificial Neural Networks became a popular research topic in the late 1950s and early 1960s. The first computational trainable neural networks were created by Frank Rosenblatt and others in the late 1950s. Called Perceptron, this neural network consisted of two computational nodes and a single layer of interconnections and was used solely to solve linear problems. During the 1970s the interest for Artificial Neural Networks faded, one reason was the book written by Minsky & Papert in 1969, in which a somewhat pessimistic view of the future of the field was presented (Turban et al., 2011). Since the 1980s interest for artificial neural networks increased with more funding to research in the field in several countries, especially in Japan and the USA after a joint conference being held in Kyoto, Japan on neural networks. This new- found optimism stemmed from discoveries in the field which overcame some of the obstacles earlier encountered in creating Artificial Neural Network and progress in the fields of neuroscience and cognitive science (Anderson & McNeill, 1992). Successful implementations of Artificial Neural Networks have in recent times invoked interest from several agents outside the academic sphere, such as industry and business (Turban et al., 2011) 2. Findings from The Literature Study 2.1 Neural networks A neural network is described as a system built of several processing elements operating in parallel whose function is determined by network structure, connection strengths, and the processing performed at computing elements or nodes (Nemadi, 2012). The neural structure of the brain consists of approximately 100 billion cells, called neurons, which all are interconnected to several thousand other neurons through a huge network. A biological neuron consists basically of four components; dendrites which are accepting the input, soma which is processing the input, the axon which turns the processed input into output and synapses which is connecting the neuron to other neurons, allowing the neurons to communicate with each other. The communication between the different neurons takes place in the axon and dendrites of each neuron. The sheer number of cells, and the fact that the neurons seemingly are the only cells not regenerating, thus enabling humans to remember and use the experience to actions, are what often is regarded as the strength of the brain (Anderson & McNeill, 1992). Copyright © CC-BY-NC 2020, CRIBFB |BJMSR www.cribfb.com/journal/index.php/BJMSR Bangladesh Journal of Multidisciplinary Scientific Research Vol. 2, No. 1; 2020 49 2.2 Artificial Neural Networks While computers can keep ledgers and perform complex mathematical calculations, they often fail when faced with tasks involving recognizing patterns and learning by experience, tasks the brain manages far superior (Anderson & McNeill, 1992). Artificial neural networks are computational methodologies or models inspired by the networks of biological neural structures of the brain. However, these networks use mostly the main idea of a biological neural network, and should not be regarded as a correct model of the actual brain’s neural structure, since this neural structure is considered vastly more complicated. Like their biological counterpart, artificial neural networks consist of several small elements called artificial neurons which are responsible for processing information. The neurons of an artificial neural network work parallel and together while using some of the abilities of biological neural networks, such as self-organizing and learning. Just as the dendrite of the biological neuron receives input from other neurons by electrochemical impulse, the artificial neuron receives input from other neurons by analog signals. 2.2.1 Network Structure In translating the basic structure of the biological neuron to an artificial equivalent the names of the neurons different parts are translated to names more associated with computer science. The dendrites and axons are known as the input and output, the soma known as a node and the synapse translates to connection weights (Turban et al., 2011). The basic processing element of the neural network, which forms the neural network’s structure by being set up in different formations, is the neuron. In a network structure the processing element, the neuron, receives input either consisting of raw data or data processed in another neuron, processing the input, and delivers the output, which could be a final result or serve as input to other processing elements. Generally, an artificial neural network consists of groups of neurons, clustered together to different layers. Typically, these layers are the input layer, the hidden layer, and the output layer. The hidden layer is the layer of neurons responsible for transforming the input data from the input layer to suitable data for the output layer. Depending on the complexity of the application the number of the hidden layer varies, usually in the commercial system from one to three hidden layers, with each potentially containing thousands of processing elements (Turban et al., 2011) In a similar process of the brain, the different processing elements of the artificial neural network perform their computations simultaneously, known as parallel processing, a process that differs a lot from the traditional serial programming. The main practice of network information processing in artificial neural networks consists of some important concepts such as input, output, and connection weight. Often each input relates to a specific attribute or variable, describing a specific condition. The output of artificial neural networks is what’s considered the solution to the problem. For instance, if the problem at hand is a decision-making system the solution often can consist of a simple yes or no answer. Connection weights describe the amount a specific input affects another processing element and, in the end, the output. By adjusting the values of the weights, the network can learn patterns of information and store these (Turban et al., 2011). Each input value is multiplied with its weight to calculate a total weighted sum in a summation function. This calculation serves later as an activation level for the input, which determinates whether a neuron should produce an output or not. 2.2.2 Constructing an Artificial Neural Network When constructing an artificial neural network nine steps are usually followed and repeated. These are: ▪ Collecting data. ▪ Separating the data into subgroups; training, validation, and testing sets. ▪ Deciding on suitable architecture and structure of the proposed network. ▪ Selecting a learning algorithm. ▪ Setting network parameters and their initial values. ▪ Setting the initial values of the weights, and start committing training. ▪ Training is committed, check ▪ Training stops, weights are adjusted and the training process is iterated ▪ A stable set of weights have been found; the network can be used on new non-training-based cases. Much of the process of constructing an artificial neural network consists of collecting data that are appropriate and have enough information for creating test sets for the system to train on. The choice of structure and architecture consists of Copyright © CC-BY-NC 2020, CRIBFB |BJMSR www.cribfb.com/journal/index.php/BJMSR Bangladesh Journal of Multidisciplinary Scientific Research Vol. 2, No. 1; 2020 50 establishing input nodes, output nodes, number of hidden layers, and hidden nodes. Choosing a learning algorithm is done to find sets of connection weights that are deemed to have the best predictive accuracy and best suited for the training data. Testing is done in the 8th step by using the testing data set to verify that the input produces suitable output (Turban et al., 2011). 2.2.3 Artificial Neural Network Architectures There are several different types of Neural Network architectures being used in a different context, mainly depending on the actual task at hand. Another important aspect of the architecture of the networks is the learning process. Supervised learning in artificial neural networks is a concept where a training set is used to teach the network of the problem and its domain, unlike unsupervised learning where the neural network is working with a more self-organizing approach by learning pattern through repeated exposure (Turban et al., 2011). One of the most popular artificial neural network architectures is the multilayer architecture, which consists of several layers of neurons. Often the feedforward approach, where the output of a node in one layer is not connected to the input of a node in a previous layer nor the same layer, but only to nodes in subsequent layers, is used on this type of architecture. Other popular architectures are Kohonen’s self-organizing feature maps and Hopfield networks. Kohonen’s self- organizing feature maps, also known as SOM, is one of the most popular neural networks for use in data mining. In the SOM architecture, the network uses a type of unsupervised learning to produce a low-dimensional representation of the input which often is consisting of high dimensional data sets. Hopfield Networks are known as recurrent neural networks, which mean that it consists basically of a single layer of a neutron in which every neuron is connected. This differs from the feedforward type of networks where neurons are not connected in the same layer. The neutrons in a Hopfield network are all binary units, which are either active or inactive. Initially, all the neutrons have random values but change through iteration by checking the connection weight between neurons. This goes on until the neurons reach a stable state, which is deemed to be the final state (Turban et al., 2011). 2.2.4 Learning Algorithms A learning algorithm is used to help the neural network specify how it learns the relationship between inputs, or between inputs and outputs. One of the most used learning algorithms is the back-error propagation, often called backpropagation. This algorithm is often used on neural networks with a feedforward approach. A supervised learning algorithm, the backpropagation algorithm is trained with correct patterns being provided, with the weights of inputs being adjusted to match the patterns. The algorithm follows a few steps; firstly, the weights are given random values, the input and the desired output are read, calculate the actual output, compare the actual output to the desired output, and finally changing the weights. These steps are then repeated until the desired output and the actual output is consistent to a predetermined degree (Turban et al., 2011). To help with the process of the learning some learning laws or rules have been stated. One of the best known is Hebb’s Rule. This says that if a neuron receives input from another neuron, which approximately the same value, the weight between the neurons should be increased. Hopfield Law is in many ways similar to Hebb’s Rule but states that if the desired output and the input have the same value, the weights should be increased by the learning rate of the network. If not the case, the weights should be decreased by the learning rate. One of the most used learning laws is Delta rule. This rule is based on continuously modifying the connection weights to reduce the difference, or delta, between the desired output and input (Anderson & McNeill, 1992). 3. Discussion and Conclusion Today there are several different areas of use for artificial neural networks, in both research and business settings. Neural networks are suitable for data mining problems with categorical and numerical data where the relationship between output and input is nonlinear, problems in which traditional statistical tools often return unreliable results. Usually, the areas of use for neural networks fall into one or more of five categories of tasks: classification, regression, clustering, association, and prediction. Classification is an action in which patterns are recognized that describes the group of that a certain item belongs to. This is done by setting a set of rules based on already classified items. With clustering, groups consisting of elements that are deemed to have attributes similar to each other are created. When using prediction, the value of an item is set depending on previous set values (Singh & Chauhan, 2005). Attempts have been made to use artificial neural networks in the financial sphere, especially in trying to predict the stock market, with some results are successful in developing trading strategies. Banks have benefited from artificial neural networks by using systems that determine if loan applications should be approved, predicting solvency of mortgage applications and credit card fraud detection. Neural networks have also been used in predicting bankruptcy by being trained with several examples of failed banks (Nemadi, 2012). Copyright © CC-BY-NC 2020, CRIBFB |BJMSR www.cribfb.com/journal/index.php/BJMSR Bangladesh Journal of Multidisciplinary Scientific Research Vol. 2, No. 1; 2020 51 Neural networks have in some cases been shown to be able to predict sports results. In a study, a neural network application was given data from the National Football League, to predict match results in American football. The result of the study showed that the system could predict the correct result in 75 % of the games, compared to experts who predicted the correct result in 63 % of the games (Kahn, 2003). The process of forecasting is another field where artificial neural network are considered to apply to. Traditionally statistical tools have been used when creating forecasting models. However, these tools often have limitations in estimating the underlying relationship that exists between an input, consisting of past values, and an output, consisting of future values. Statistical methods often make assumptions of data distribution, which could make them unreliable in cases where the input is not normally distributed. Furthermore, the non-linear nature of many artificial neural networks has in many instances being deemed to better represent the often-non-linear nature of the problem at hand (Zhang et al., 1998). Diagnosis and pattern recognition in the fields of health care and medicine are areas where artificial neural networks are successful. Several studies have shown how neural networks can improve diagnostics and lead to more rapid decision making, which could potentially help save lives (Turban et al., 2011). References Anderson, D., & McNeill, G. (1992). Artificial Neural Networks Technology, Rome Laboratory, New York. Kahn, J. (2003). Neural Network Prediction of NFL Football Games, University of Wisconsin-Madison. Retrieved from http://homepages.cae.wisc.edu/~ece539/project/f03/kahn.pdf Nemadi, H. (2012). Introduction to Data Mining Using Artificial Neural Networks. Retrieved from http://www.uncg.edu/ism/ism611/neuralnet.pdf Singh, Y., & Chauhan, A. S. (2005). Neural Networks in Data Mining. Journal of Theoretical & Applied Information Technology, 5(1). Turban, E., Sharda, R., & Delen, D. (2011). Decision Support and Business Intelligence Systems, Prentice Hall Press, NJ, United States. Yegnanarayana, B. (2009). Artificial Neural Networks, PHI Learning Pvt. Ltd. Zhang, G., Patuwo, B. E., & Hu, M. Y. (1998). Forecasting with artificial neural networks: The state of the art. International journal of forecasting, 14(1), 35-62. Copyrights Copyright for this article is retained by the author(s), with first publication rights granted to the journal. This is an open-access article distributed under the terms and conditions of the Creative Commons Attribution license (http://creativecommons.org/licenses/by/4.0/). http://homepages.cae.wisc.edu/~ece539/project/f03/kahn.pdf http://www.uncg.edu/ism/ism611/neuralnet.pdf