Theory of Neural Information Processing Systems
Within the last decade, the field of artificial neural networks has become a well-established research discipline. Due to its interdisciplinary character, many books currently appearing on neural network theory are quite specialised and focus primarily on problems and methods applicable in the field. This book fills the gap between older textbooks dealing with the progress made in the 1980´s and the up-to-date specialised texts. Among others, these new areas comprise applications of Bayesian methods, Gaussian processes and support vector machines, information theory for neural networks, and Amari´s information geometry.
The whole text is divided into five parts. The introductory Part I explains the basic principles of neurocomputing and two fundamental neural network architectures: perceptrons and recurrent networks. Part II discusses more advanced neural network models: vector quantization, Gaussian processes and RBF-networks, Bayesian techniques, and support vector machines. Part III deals with information theory (measuring information, entropy, Shannon’s information theory and statistical inference) and its application to neural networks. Part IV is devoted to macroscopic analysis of learning dynamics and the final Part V concludes with equilibrium statistical mechanics of neural networks. Each of the five parts encompasses a brief overview of the discussed methods. Nine appendices are attached explaining the background of some essential theoretical concepts.
The book provides an excellent class-tested material for graduate courses in artificial neural networks. It is completely self-contained and also includes a thorough introduction to each discipline-specific topic and the chapters are accompanied with a number of exercises. Notes on historical background and suggestions for further reading are included at the end of each part, guiding the reader to the relevant literature. The text is written clearly, containing many illustrative figures, supporting graphs and sufficient references. Therefore, this book represents a good reference source of applicable ideas for a wide audience including students, researchers and application specialists.