Markov Chains

This book covers the classical theory of Markov chains on general state-spaces as well as many recent developments. The theoretical results are illustrated by simple examples, many of which are taken from Markov Chain Monte Carlo methods. The book is self-contained, while all the results are carefully and concisely proven. Bibliographical notes are added at the end of each chapter to provide an overview of the literature.

Part I lays the foundations of the theory of Markov chain on general states-space. Part II covers the basic theory of irreducible Markov chains on general states-space, relying heavily on regeneration techniques. These two parts can serve as a text on general state-space applied Markov chain theory. Although the choice of topics is quite different from what is usually covered, where most of the emphasis is put on countable state space, a graduate student should be able to read almost all these developments without any mathematical background deeperthan that needed to study countable state space (very little measure theory is required).

Part III covers advanced topics on the theory of irreducible Markov chains. The emphasis is on geometric and subgeometric convergence rates and also on computable bounds. Some results appeared for a first time in a book and others are original. Part IV are selected topics on Markov chains, covering mostly hot recent developments.

Part I Foundations.- Markov Chains: Basic Definitions.- Examples of Markov Chains.- Stopping Times and the Strong Markov Property.- Martingales, Harmonic Functions and Polsson-Dirichlet  Problems.- Ergodic Theory for Markov Chains.- Part II Irreducible Chains: Basics.- Atomic Chains.- Markov Chains on a Discrete State Space.- Convergence of Atomic Markov Chains.- Small Sets, Irreducibility and Aperiodicity.- Transience, Recurrence and Harris Recurrence.- Splitting Construction and Invariant Measures.- Feller and T-kernels.- Part III Irreducible Chains: Advanced Topics.- Rates of Convergence for Atomic Markov Chains.- Geometric Recurrence and Regularity.- Geometric Rates of Convergence.- (f, r)-recurrence and Regularity.- Subgeometric Rates of Convergence.- Uniform and V-geometric Ergodicity by Operator Methods.- Coupling for Irreducible Kernels.- Part IV Selected Topics.- Convergence in the Wasserstein Distance.- Central Limit Theorems.- Spectral Theory.- Concentration Inequalities.- Appendices.- A Notations.- B Topology, Measure, and Probability.- C Weak Convergence.- D Total and V-total Variation Distances.- E Martingales.- F Mixing Coefficients.- G Solutions to Selected Exercises.

Randal Douc is a Professor in the CITI Department at Telecom SudParis. His research interests include parameter estimation in general Hidden Markov models and Markov Chain Monte Carlo (MCMC) and sequential Monte Carlo methods.