Applications of Neural Networks | Artificial Intelligence

Practically every non-mechanical task performed by animals requires the interaction of neural networks; perception, recognition, memory, conscious thought, dreams, sensorimotor control—the list goes on. The desire to simulate some of these tasks has motivated the development of artificial neural network. For each task, we identify performance of a neural network in terms of success performing the task.

Following are some of the applications of neural networks:

Application # 1. Classification of Object:

Classification, the assignment of each object to a specific ‘class’ (one of many predetermined groups), is of fundamental importance in a number of areas ranging from image and speech recognition to the social sciences. We are provided with a ‘training set’ consisting of sample patterns which are representative of all classes, along with class membership information for each pattern. Using the training set, we deduce rules for membership in each class and create a classifier, which can then be used to assign other patterns to their respective classes according to these rules.

Neural networks have been used to classify samples, that is map input patterns to different classes. For instance, each output node can stand for one class. An input pattern is determined to belong to class i if the ith output node computers a higher value than all other output nodes when that input pattern is fed into the network. In some networks, an additional constraint is that the magnitude of that output node must exceed a minimal threshold, say 0.5.

For two-class problems, feed forward networks with a single output node are adequate. If the node output has permissible values ranging from 0 to 1, for instance, a value close to 1 (say > 0.9) is considered to indicate one class while a value close to 0 (say < 0.1) indicates the other class.

Neural networks have been used successfully in a large number of practical classification tasks, such as the following:

i. Recognising printed or handwritten characters.

ii. Classifying loan applications into credit-worthy and non-credit-worthy groups.

iii. Analysing sonar and radar data to determine the nature of the source of a signal.

Application # 2. Clustering of Object:

Clustering requires grouping together objects which are similar to each other. In classification problems, the identification of class is known beforehand, as is the membership of some sample in these classes.

In clustering problems, on the other hand, all which is available is a set of samples and distance relationships which can be derived from the sample descriptions. For example, in the spectra of a hundred thousand stars; if there are different types of stars revealed by the spectra, what are their types and what are their characteristics?

We are all familiar with the terms such as red giant and white draw but the stars do not carry these labels on their hats-astronomers had to perform unsupervised clustering to identify these categories. Other examples include the identification of species, genera, orders etc., in the Linnaean taxonomy of organisms and the creation of natural kinds to categorise objects, like flowers, into clusters, using features such as colour and number of petals.

Most clustering mechanisms are based on some distance measure. Each object is represented by an ordered set (vector) of features. Similar objects are those which have nearly the same values for different features. The samples are grouped so as to minimise intra-cluster distances while maximising inter-cluster distances, subject to constraints on the number of clusters which can be formed.

One way to measure intra-cluster distance would be to find the average distance of different samples in a- cluster from the cluster center. Similarly, inter-cluster distance could be measured using the distance between the centers of different clusters. Fig. 11.27 depicts a problem in which prior clustering (into five clusters) is helpful in a classification pattern.

The number of clusters depends on the problem, but should be as small as possible. Fig. 11.28, shows three ways of clustering the same data, of which the first is preferable since it has neither too many nor too few clusters.

Some neural networks accomplish clustering by the following method:

Initially, each node reacts randomly to the presentation of input samples. Nodes with higher outputs to an input sample learn to react even more strongly to that sample and to other input samples geographically near that sample.

In this way, different nodes specialise, responding strongly to different clusters of input samples. This method is analogous to the statistical approach of k-nearest neighbour clustering, in which each sample is placed in the same cluster as the majority of its immediate neighbours.

Vector Quantisation:

Neural networks have been used for compressing voluminous input data into a small number of weight vectors associated with nodes in the networks. Each input sample is associated with the nearest weight vector (with the smallest euclidean distance). Vector quantisation is the process of dividing up space into several connected regions (called ‘Voronoi regions’), a task similar to clustering. Each region is represented using a single/vector (called a ‘codebook vector’). Every point in the input space belong to one of these regions, and is mapped to the corresponding (nearest) codebook vector.

The set of codebook vectors is a compressed form of the set of input data vectors, since many different input data vectors may be mapped to the same codebook vector. Fig. 11.29 gives an example of such a division of two-dimensional space into voronoi regions, called a Voronoi diagram (or tessellation). For two-dimensional input spaces, the boundaries of Voronoi regions obtained by sketching the perpendicular bisectors of the lines joining neighbouring codebook vectors.

Application # 3. Pattern Association:

In pattern association, another important task which can be performed by neural networks, the presentation of an input sample, should trigger the generation of a specific output pattern. In auto association or associative memory tasks (Fig. 11.30), the input sample is presumed to be a corrupted, noisy, or partial version of the desired output pattern.

In hetero-association (Fig. 11.31), the output pattern may be any arbitrary pattern which is to be associated with a set of input patterns. An example of an auto associative task is the generation of a complete (uncorrupted) image, such as a face, from a corrupted version. An example of hetero-association is the generation of a name when the image of a face is presented as input.

In the context of neural networks, auto/hetero-association refers to the task of setting up weights which represent the mappings between input and output patterns, whereas recall refers to the retrieval of the output pattern corresponding to a specific input pattern.

A typical auto-associative neural network consists of a single layer of nodes, each node corresponds to each input pattern dimension, and the recall process involves repeatedly modifying node outputs until the resulting output pattern ceases to change. For hetero-associative recall, a second layer of nodes is needed to generate the output pattern corresponding to an input pattern.

Application # 4. Function Approximation:

Many computational models can be described as functions mapping some numerical input vectors to numerical outputs. The outputs corresponding to some input vectors may be known from training data, but we may not know the mathematical function describing the actual process which generates the outputs from the input vectors. Function approximation is the task of learning or constructing a function which generates approximately the same outputs from input vectors as the process being modeled, based on available training data.

Fig. 11.32, illustrates that the same finite set of samples can be used to obtain many different functions, all of which perform reasonably well on the given set of points. Since infinitely many functions exist which coincide for a finite set of points, additional criteria are necessary to decide which of these functions are desirable. Continuity and smoothness of the function are almost always required. Following established scientific practice, an important criterion is that of simplicity of the model is that the neural network should have as few parameters as possible.

Application # 5. Forecasting of Problems:

There are many real-life problems in which future events must be predicted on the basis of past history. An example is that of predicting the behaviour of stock market indices. It is observed that prediction hinges on two types of knowledge: knowledge of underlying laws, a very powerful and accurate means of prediction, and the discovery of strong empirical regularities in observations of a given system. However, laws under laying the behaviour of a system are not easily discovered, and empirical regularities or periodicities are not always evident, and can often be masked by noise.

Though perfect prediction is hardly ever possible, neural networks can be used to obtain reasonably good predictions in a number of cases. For instance, neural networks have succeeded in learning the 11-year cycle in sunspot data without being told a priori about the existence of such a cycle.

In forecasting problems, it is important to consider both short-term (‘one-lag’) and long-term (‘multilag’) predictions. In one-lag prediction, we forecast the next value based only on actual past values. In multilag prediction, on the other hand some predicted values are also used to predict future values.

For instance, a five input network is first used to predict a value n₆ from observed input data i₁‘……… i₅, then the next network prediction n₇ is made using inputs i₂, i₅,…. n₆, followed by the network prediction n₈ using inputs i₃ i₄, i₅, i₆, i₇, But one-lag prediction at the eight instant is made using only the actual input data values i₃, i₄, i₅, i₆, i₇. Multilag prediction is required, for example, if we want to predict the value of a variable six months from today, not knowing the values for the next five months.

A better understanding of difficult problems is often obtained by studying many related variables together rather than by studying just one variable. A multivariate time series consists of sequences of values of several variables concurrently changing with time.

The variables being measured may be significantly correlated, for example, when similar attributes are being measured at different geographic locations. Values for each variable may then be predicted with greater accuracy if variations in the other variables are also taken into account. To be successful, forecasting must be based on all available correlations and empirical interdependencies among different temporal sequences. Feed forward as well as recurrent networks have been used for forecasting.

Application # 6. Control Applications for Variables:

Many manufacturing and industrial applications have complex implicit relationships among inputs and outputs. Control addresses the task of determining the values for input variables in order to achieve desired values for output variables.

This is also a function approximation problem, for which feed forward, recurrent, and some specialised neural networks have been used successfully, adaptive control techniques have been developed for systems subject to large variations in parameter values, environmental conditions, and signal inputs. Neural networks can be employed in adaptive control systems to provide fast response, without requiring human intervention.

Systems modeled in control problems may be static or dynamic; in the latter, the system may map inputs to outputs in a time-dependent manner, and the system’s input-output mapping may change with time. A simple example of a static control system is one which maps input voltages into mechanical displacements of a robotic arm.

Irrespective of the history of the system, an input voltage will always generate the same output displacement. By contrast, the inverted pendulum control task, where the behaviour of a pendulum depends on time-dependent input variables such as velocity, is a dynamic control task.

(a) Forward system identification, and

(b) Inverse system identification.

Neural networks have been used for two tasks associated with control; in both tasks, learning is supervised because the system’s behaviour dictates what the neural network is to accomplish.

These tasks, illustrated in figure 11.32, are as follows:

1. System (forward) identification is the task of approximating the behaviour of a system using a neural network or other learning method. If the system maps a set of input variables I to output variables O then forward identification is conducted by a feed forward neural network whose inputs correspond to I and outputs correspond to O, trained to minimise an error measure which captures the difference between the network outputs and actual control system output.

2. Inverse identification is the task of learning the inverse of the behaviour of a system, possibly using neural networks. For instance, given the amount of force applied to a robotic arm system, the system’s behaviour results in displacement by a certain amount. The inverse problem in this case consists of determining the force required to produce a desired amount of displacement.

This can be conducted using a feed forward network which uses components of O as its inputs and components of I as its outputs, using an error measure which captures the difference in actual system inputs (i) from the result (NS(i)) of applying the actual system on those inputs, followed by the neural network.

If the neural network has been successfully trained to perform inverse system identification, it can generate values for system inputs needed to obtain desired system outputs. If the system behaviour changes with time, the same network which is used to generate inputs for the system may also be continually trained online, i.e., its weights are adjusted depending on the error measure.

N (i)-N(S(N (i))),

The deviation between the network outputs N(i) and the result of applying the network to the system’s output when applied to the network’s outputs.

If the system’s input-output mapping is not completely known, or varies with time, a neural network may have to be continually trained to track changes in system behaviour, A feedback control system is then appropriate, in which the error between actual and desired system output is used to modify the input signal, in order to produce the desired behaviour.

Application # 7. Optimisation:

Many problems in business and scientific modeling can be represented as optimisation problems in which the goal is to optimise (maximise or minimise) some functions, subject to some constraints.

An example is the task of arranging components on a circuit board such that the total length of wires is minimised, with additional constraints which require certain components to be connected to certain others, as shown in Fig. 11.34.

Some such problems can also be solved using neural networks. The best solutions for complex optimisation problems are often obtained using networks with stochastic behaviour in which network behaviour has a significant random or probabilistic component.

Application # 8. Search:

Search problems and their solutions occupy a central place in Artificial Intelligence. Search problems are characterised by a set of states, a description of how transitions (or moves) may be made among different states, and methods of making moves which make it possible to reach a goal, possibly minimising the number of moves or computational expense.

Many problems can be viewed as search problems: for instance, each ‘state’ in a neural network is a possible weight matrix, each ‘move’ is the possible change in the weight matrix which may be made by the learning algorithm, and the ‘goal’ state is one for which the mean squared error is at a local minimum.

Neural networks can also be applied to certain other search problems often solved using symbolic reasoning systems. In attempting to solve a search problem using neural networks, the first (and possibly the hardest) task is to obtain a suitable representation for the search space in terms of nodes and weights in a network.

For instance, in a neural network to be used for game playing, the inputs to the network describe the current state of the board game, and the desired output pattern identifies the best possible move to be made. The weights in the network can be trained based on an evaluation of the quality of previous moves made by the network in response to various input patterns. This is similar to what was done by Sameul in checker players’ game.

Application # 9. Combinatorial Problems:

Parallel relaxation can also be used to solve many constraint satisfaction problems. Hopfield and Tank (1985) show how a Hopfield network can be programmed to come up with approximate solutions to the travelling salesman problem.

The system employs n² neural networks, where n is the number of cities to be toured. Fig. 11.35., shows how tours of the salesman are represented. Each row stands for one city. The tour proceeds horizontally across the columns. The starting city is marked by the active unit in column 1, the next city by column 2 etc., the travel shown in the figure proceeds through cities D, B, E, H, F, C, A and back to D.

As in Hopfield networks, this n x n array contains a number of weighted connections. The connection weights are initialised to reflect exactly the constraints of a particular problem instance (this is just an example).

First, every unit is connected with a negative weight to every other unit in its column, because only one city at a time can be visited.

Second, every unit inhibits every other unit in its row, because each city can only be visited once.

Third, units in adjacent columns inhibit each other in proportion to the distance between cities represented by their rows.

For example, if city D is far from city G, then the fourth unit in column 3 will strongly inhibit the seventh units in columns 2 and 4. There is some global excitation, so in the absence of strong inhibition, individual units will prefer to be active.

We may note that each unit represents some hypothesis about the position of a particular city in the travel. To find out that travel, we start by assigning our units random activation values (weights). Once all the weights are set, the units update themselves asynchronously according to some set rule.

The updating continues until a stable state is reached. Hopfield and Tonk have used these networks to give quick, approximate solutions, Wilson and Powel (1988) have given a critique of this method. Another problem, graph-colouring can be cast as constraint satisfaction problem, and solved with parallel relaxation network.