Statistics for Making Decisions

Making decisions is a ubiquitous mental activity in our private and professional or public lives. It entails choosing one course of action from an available shortlist of options. Statistics for Making Decisions places decision making at the centre of statistical inference, proposing its theory as a new paradigm for statistical practice. The analysis in this paradigm is earnest about prior information and the consequences of the various kinds of errors that may be committed. Its conclusion is a course of action tailored to the perspective of the specific client or sponsor of the analysis. The author’s intention is a wholesale replacement of hypothesis testing, indicting it with the argument that it has no means of incorporating the consequences of errors which self-evidently matter to the client. The volume appeals to the analyst who deals with the simplest statistical problems of comparing two samples (which one has a greater mean or variance), or deciding whether a parameter is positive or negative. It combines highlighting the deficiencies of hypothesis testing with promoting a principled solution based on the idea of a currency for error, of which we want to spend as little as possible. This is implemented by selecting the option for which the expected loss is smallest (the Bayes rule). The price to pay is the need for a more detailed description of the options, and eliciting and quantifying the consequences (ramifications) of the errors. This is what our clients do informally and often inexpertly after receiving outputs of the analysis in an established format, such as the verdict of a hypothesis test or an estimate and its standard error. As a scientific discipline and profession, statistics has a potential to do this much better and deliver to the client a more complete and more relevant product. Nicholas T. Longford is a senior statistician at Imperial College, London, specialising in statistical methods for neonatal medicine. His interests include causal analysis of observational studies, decision theory, and the contest of modelling and design in data analysis. His longer-term appointments in the past include Educational Testing Service, Princeton, NJ, USA, de Montfort University, Leicester, England, and directorship of SNTL, a statistics research and consulting company. He is the author of over 100 journal articles and six other monographs on a variety of topics in applied statistics.

1 First steps What shall we do? Example The setting Losses and gains States, spaces and parameters Estimation Fixed and random Study design Exercises 2. Statistical paradigms Frequentist paradigm Bias and variance Distributions Sampling from finite populations Bayesian paradigm Computer-based replications Design and estimation Likelihood and fiducial distribution Example Variance estimation From estimate to decision Hypothesis testing Hypothesis test and decision Combining values and probabilities Additivity Further reading Exercises 3. Positive or negative? Constant loss Equilibrium and critical value The margin of error Quadratic loss Combining loss functions Equilibrium function Example Example Plausible values and impasse Elicitation Post-analysis elicitation Plausible rectangles Example Summary Further reading Exercises 4. Non-normally distributed estimators Student t distribution Fiducial distribution for the t ratio Example Example Verdicts for variances Linear loss for variances Verdicts for standard deviations Comparing two variances Example Statistics with binomial and Poisson distributions Poisson distribution Example Further reading Exercises Appendix 5. Small or large? Piecewise constant loss Asymmetric loss Piecewise linear loss Example Piecewise quadratic loss Example Example Ordinal categories Piecewise linear and quadratic losses Multitude of options Discrete options Continuum of options Further reading Exercises Appendix A Expected loss Ql in equation () B Continuation of Example C Continuation of Example 6. Study design Design and analysis How big a study? Planning for impasse Probability of impasse Example Further reading Exercises Appendix Sample size calculation for hypothesis testing 7. Medical screening Separating positives and negatives Example Cutpoints specific to subpopulations Distributions other than normal Normal and t distributions A nearly perfect but expensive test Example Further reading Exercises 8. Many decisions Ordinary and exceptional units Example Extreme selections Example Grey zone Actions in a sequence Further reading Exercises Appendix A Moment-matching estimator B The potential outcomes framework 9. Performance of institutions The setting and the task Evidence of poor performance Assessment as a classification Outliers As good as the best Empirical Bayes estimation Assessment based on rare events Further reading Exercises Appendix A Estimation of _ and _ B Adjustment and matching on background 10. Clinical trials Randomisation Analysis by hypothesis testing Electing a course of action — approve or reject Decision about superiority More complex loss functions Trials for non-inferiority Trials for bioequivalence Crossover design Composition of within-period estimators Further reading Exercises 11. Model uncertainty Ordinary regression Ordinary regression and model uncertainty Some related approaches Bounded bias Composition Composition of a complete set of candidate models Summary Further reading Exercises Appendix A Inverse of a partitioned matrix B Mixtures EM algorithm C Linear loss 12. Postscript References Index Solutions to exercises

Nicholas T. Longford is a Senior Statistician at Imperial College, London, specialising in statistical methods for neonatal medicine. His interests include causal analysis of observational studies, decision theory, and the contest of modelling and design in data analysis. His longer-term appointments in the past include Educational Testing Service, Princeton, NJ, U.S.A., de Montfort University, Leicester, England, and directorship of SNTL, a statistics research and consulting company. He is the author of over 100 journal articles and six other monographs on a variety of topics in applied statistics.