Computational Intelligence: An Introduction [Unknown Binding]

The author divides his book into five parts, the first one being an introduction and history, and the next four being on neural networks, evolutionary computation, swarm intelligence, and fuzzy systems. The third part is relatively new on the scene in computational intelligence, and so its inclusion in this book is welcomed.

Neural networks have been extensively used in the last four decades in many different fields, such as financial engineering, bioinformatics, physics, and network analysis. The author devotes six chapters to their elucidation, and he discusses the most important types of artificial neural networks and the different learning schemes employed by them. Detailed derivations and pseudocode are given for these, particularly for back-propagation, and the "LeapFrog" and "conjugate gradient" approaches to optimization. Most interesting is the inclusion of a "swarm optimization" technique. Although brief, the author includes pseudocode and references for further reading.

The author includes a discussion of product unit neural networks, which are sometimes not treated in other books on neural networks. In this type of neural network, the neurons compute the weighted product of the input signals, and the author includes a "distortion factor" that allows the dynamical shaping of the activiation function during the training phase. Only a cursory discussion of what it actually means for a neural network to learn is given, but references for further reading are included. The author also includes a nice discussion of performance issues in the use of neural networks. Readers who eventually use them in practice will find that neural networks can be computationally intensive. An interesting inclusion in this discussion is that of the VC-dimension, and how it relates to the total number of weights in a 1-hidden layer network, and thus of the number of training examples needed. The reader is asked to investigate the role of the Huber function in making a neural network more robust to outliers. The author cautions the reader that the use of neural networks should not be taken lightly and great care is need to insure optimal performance.

In part 3 the author treats genetic and evolutionary programming, evolutionary strategies, differential and cultural evolution, and co-evolution. The use of these techniques has dramatically increased in the last decade, especially in optimization problems. The author points out the role of finding the right chromosome representation in the design of an evolutionary algorithm, this serving also as a kind of classification of the different approaches: binary strings for genetic algorithms, tree representations for genetic programming, and real numbers for evolutionary programming. The author compares briefly the use of evolutionary computing versus using techniques from classical optimization theory, with the role played by the "no-free-lunch" theorem. The author holds that evolutionary algorithms are superior for discontinuous, nondifferentiable, multimodal, and noisy problems.

Of personal interest to me, an application of genetic algorithms to routing optimization is discussed in this part. The author also briefly mentions "niching" as a method to locate multiple minima and asks the reader to develop a genetic algorithm to train a feed-forward neural network as an exercise. For readers who have a knowledge of LISP the discussion on genetic programming will seem much more natural. Evolutionary programming, which differs from genetic algorithms and genetic programming with its omission of crossover and its emphasis on simulating adaptive behavior in evolution, and not genetic models is given ample treatment, along with some useful pseudocode. An elementary application to function optimization is given. The discussion on evolutionary strategies is a fine one since the ones in the literature may be too difficult mathematically for a reader at this level. The discussions on differential and cultural evolution, and coevolution are too brief however (only a few pages devoted to each).

Of great interest to me was the part on swarm intelligence, as my knowledge of this field is very meager. It has taken on importance recently in research circles and so I was very interested in seeing if could be used successfully in practical optimization problems. Having just begun to get my feet wet in swarm intelligence algorithms, this part was a good help in giving me more insight into the use of them. This book is one of the few that covers swarm intelligence, most of the literature still being found in online preprints and published journal articles. The author calls one approach "particle swarm optimization" (PSO), and he explains how the different social interaction schemes serve to classify the algorithm, going by the namnes of star, ring, and wheels topology. He also discusses various modifications to PSO that improve its convergence and to increase its diversity. Another approach is based on the social interaction of ants, and is called appropriately "ant colony optimization". This approach has been applied with some success to route assignment in packet networks, and has been shown to be competetive with vector-distance and link-state shortest path algorithms. It remains to be seen if such an approach will be implemented in real-world network topologies, dominated as they are currently with legacy hardware. The author also gives brief discussions on some of its other applications, such as robotics, and the quadratic assignment, job-scheduling, and graph coloring problems.

I did not read the part on fuzzy systems so I will omit its review.