A Quick Introduction to Artificial Neural Networks

A natural neural network (crayfish).

This system pre-processes visual information from the crustacean eye. Structures from the lamina ganglionaris to the medulla terminalis are found in the eyestalk of the animal. The optic tract is within the "head" and connects directly to the brain. Although much of the information is unidirectional (from the eye toward the brain), each layer communicates with the others in the system. This system is analogous to that of an artificial neural network. The retinular cells would represent the input axons of an artificial network while the various red structures represent processing elements within other axons. The black arrows are analogous to the synapses of the artificial network. The cross talk among the various layers is similar to learning algorithms in artificial networks such a back propagation.

Fly eye showing layering

The Concept

Major components of an artificial neural network (Neurosolutions).

• Input Axon: Accepts the stimulus from function generators, files, DLLs, noise generators, a series of bitmaps, etc. Axons are processing elements.

• Probe: Various types of probes can be placed on network components. In this case a virtual oscilloscope is being used. Other probes include matrix viewers, scatter or line graphs, gray scale images, or output to an external program (such as MatLab).

• Synapse: Various synapse constructs connect one axon to another. Synapse weights change during training.

• Inspector: I this example the StaticControl element is being examined. Each tab provides information on the properties of the object. In this case, the number of exemplars (samples) per epoch (runs through the data sample).

• The lower two figures show the output from the function generator with and without noise. The inspectors for the function generator and noise generator are also shown.
   

An artificial learning neural network

1. Starting at the input axon, the function generator produces a signal (a sine wave at 1 volt, 100 Hz, no offset). In addition, noise has been injected into the system (input is shown in black in the lower oscillogram.
2. The output from the first axon leads through a synape to a bias axon that processes the information.
3. Another synapse pays attention to the desired output (a since wave at 0.5 colt, 100 Hz, an offset of 1 volt, and no noise; red tracing in the left oscillogram). This third synapse (L2) computes mean square error between the input to the axon and the desired signal. The blue tracing in the left oscillogram shows how close the processing is to the desired output (red trace). The mean square error is one measure of leaning success (right oscillogram).
4. The error is sent back through the network through back propagation. At each processing element (the second axon and synapse), the weights of the PE is adjusted. Momentum controls adjust how quickly the data are fed back through the net.
5. Note that the mean square error decreases over time (right oscillogram) as the input sine wave is filtered to the desired output. This particular network was able to learn part of the task but didn't reduce the noise component. Longer training times or a larger network (bottom diagram) would be required.

Yet Another Explanation...

When applied correctly, a neural or adaptive system can considerably outperform other methods. Instead of being built a priori from specification, neural systems use external data to automatically set their parameters. These networks are “aware” of their output through a performance loop that includes a cost function that allows the system to change the parameters through systematic procedures called learning or training rules. Therefore, the system output improves with time (the error decreases through learning).

A learning network. Red arrows indicate the flow of information through the system.

The above figure depicts a learning network with many of the same properties required by the proposed system. Signals from sensors are accepted by an input axon and are passed to a process axon via a series of synapses (neural network function and terminology is based on neurobiology). The process axon analyses and transforms the input before it passes data on to another process axon that performs its own transformations before sending the data to an error axon. The error axon computes the mean square error by comparing the difference between network output and the desired output. These errors are passed through the network via a back propagation plane. As the errors are sent through the network, the transfer functions of the axons are adjusted and the synapse weights are changed to better fit the desired output. Momentum controllers adjust the rate of training.