Theory of Neural Information Processing Systems

This interdisciplinary graduate text gives a full, explicit, coherent and up-to-date account of the modern theory of neural information processing systems and is aimed at student with an undergraduate degree in any quantitative discipline (e.g. computer science, physics, engineering, biology, or mathematics). The book covers all the major theoretical developments from the 1940s tot he present day, using a uniform and rigorous style of presentation and of mathematical notation. The text starts with simple model neurons and moves gradually to the latest advances in neural processing. An ideal textbook for postgraduate courses in artificial neural networks, the material has been class-tested. It is fully self contained and includes introductions to the various discipline-specific mathematical tools as well as multiple exercises on each topic.

Theory of Neural Information Processing Systems provides an explicit, coherent, and up-to-date account of the modern theory of neural information processing systems. It has been carefully developed for graduate students from any quantitative discipline, including mathematics, computer science, physics, engineering or biology, and has been thoroughly class-tested by the authors over a period of some 8 years. Exercises are presented throughout the text and notes on historical background and further reading guide the student into the literature. All mathematical details are included and appendices provide further background material, including probability theory, linear algebra and stochastic processes, making this textbook accessible to a wide audience.

1 - General introduction
2 - Layered networks
3 - Recurrent networks with binary neurons
4 - Competitive unsupervised learning processes
5 - Bayesian techniques in supervised learning
6 - Gaussian processes
7 - Support vector machines for binary classification
8 - Measuring information
9 - Identification of entropy as an information measure
10 - Building blocks of Shannon's information theory
11 - Information theory and statistical inference
12 - Applications to neural networks
13 - Network operation: macroscopic dynamics
14 - Dynamics of online learning in binary perceptrons
15 - Dynamics of online gradient descent learning
16 - Basics of equilibrium statistical mechanics
17 - Network operation: equilibrium analysis
18 - Gardner theory of task realizability
A - Historical and bibliographical notes
B - Probability theory in a nutshell
C - Conditions for central limit theorem to apply
D - Some simple summation identities
E - Gaussian integrals and probability distributions
F - Matrix identities
G - The delta-distribution
H - Inequalities based on convexity
I - Metrics for parametrized probability distributions
J - Saddle-point integration