Tools for Statistical Inference Methods for the Exploration of Posterior Distributions and Likelihood Functions

This book provides a unified introduction to a variety of computational algorithms for Bayesian and likelihood inference. In this third edition, I have attempted to expand the treatment of many of the techniques discussed. I have added some new examples, as well as included recent results. Exercises have been added at the end of each chapter. Prerequisites for this book include an understanding of mathematical statistics at the level of Bickel and Doksum (1977), some understanding of the Bayesian approach as in Box and Tiao (1973), some exposure to statistical models as found in McCullagh and NeIder (1989), and for Section 6. 6 some experience with condi­ tional inference at the level of Cox and Snell (1989). I have chosen not to present proofs of convergence or rates of convergence for the Metropolis algorithm or the Gibbs sampler since these may require substantial background in Markov chain theory that is beyond the scope of this book. However, references to these proofs are given. There has been an explosion of papers in the area of Markov chain Monte Carlo in the past ten years. I have attempted to identify key references-though due to the volatility of the field some work may have been missed.

1. Introduction.- Exercises.- 2. Normal Approximations to Likelihoods and to Posteriors.- 2.1. Likelihood/Posterior Density.- 2.2. Specification of the Prior.- 2.3. Maximum Likelihood.- 2.4. Normal-Based Inference.- 2.5. The ?-Method (Propagation of Errors).- 2.6. Highest Posterior Density Regions.- Exercises.- 3. Nonnormal Approximations to Likelihoods and Posteriors.- 3.1. Numerical Integration.- 3.2. Posterior Moments and Marginalization Based on Laplace’s Method.- 3.3. Monte Carlo Methods.- Exercises.- 4. The EM Algorithm.- 4.1. Introduction.- 4.2. Theory.- 4.3. EM in the Exponential Family.- 4.4. Standard Errors in the Context of EM.- 4.5. Monte Carlo Implementation of the E-Step.- 4.6. Acceleration of EM (Louis’ Turbo EM).- 4.7. Facilitating the M-Step.- Exercises.- 5. The Data Augmentation Algorithm.- 5.1. Introduction and Motivation.- 5.2. Computing and Sampling from the Predictive Distribution.- 5.3. Calculating the Content and Boundary of the HPD Region.- 5.4. Remarks on the General Implementation of the Data Augmentation Algorithm.- 5.5. Overview of the Convergence Theory of Data Augmentation.- 5.6. Poor Man’s Data Augmentation Algorithms.- 5.7. Sampling/Importance Resampling (SIR).- 5.8. General Imputation Methods.- 5.9. Further Importance Sampling Ideas.- 5.10. Sampling in the Context of Multinomial Data.- Exercises.- 6. Markov Chain Monte Carlo: The Gibbs Sampler and the Metropolis Algorithm.- 6.1. Introduction to the Gibbs Sampler.- 6.2. Examples.- 6.3. Assessing Convergence of the Chain.- 6.4. The Griddy Gibbs Sampler.- 6.5. The Metropolis Algorithm.- 6.6. Conditional Inference via the Gibbs Sampler.- Exercises.- References.