Test Fraud Statistical Detection and Methodology

There has been an increase in awareness (and perhaps occurrence) of individual and organized cheating on tests. Recent reports of widespread problems with state student accountability tests and teacher certification testing have raised questions about the very validity of assessment programs. While there are several books that specifically detail the issues of test security cheating on assessments, few outline the statistical procedures used for detecting various types of potential test fraud and the associated research findings. Without a significant research literature base, the new generation of researchers will have little opportunity or incentive to improve on existing methods. Enlisting a variety of experts and scholars in different fields of testing, this edited volume expands on the current literature base by including examples of detailed research findings arrived at by statistical methodology. It also provides a synthesis of the current state of the art with regard to the statistical detection of testing infidelity, particularly for large-scale assessments.
By presenting methods currently used by testing organizations and research on new methods, the volume offers an important forum for expanding the literature in this area.

1. Introduction Neal Kingston and Amy Clark 2. A Brief History of Research on Test Fraud Detection and Prevention Amy Clark and Neal Kingston 3. Cheating: Some Ways to Detect it Badly Howard Wainer Part 1: Similarities in Responses 4. Relationships of Examinee Pair Characteristics and Item Response Similarity Jeff Allen 5. A Parametric Approach to Detect a Disproportionate Number of Identical Item Responses on a Test Leonardo S. Sotaridona, Arianto Wibowo, and Irene Hendrawan 6. Detection of Non-Independent Test Taking by Similarity Analysis Dennis Maynes Part 2: Macro Level Cheating 7. Local Outlier Detection in Data Forensics: Data Mining Approach to Flag Unusual Schools Mayuko Simon 8. Macro Level Systems of Statistical Evidence Indicative of Cheating Michael Chajewski, YoungKoung Kim, Judit Antal, and Kevin Sweeney 9. A Bayesian Hierarchical Linear Modeling Approach for Detecting Cheating and Aberrance William Skorupski and Karla Egan Part 3: Answer Changing Behavior 10. Patterns of Erasure Behavior for a Large-Scale Assessment Andrew A. Mroch, Yang Lu, Chi-Yu Huang, and Deborah J. Harris 11. AYP consequences and Erasure Behavior Vincent Primoli 12. An Exploration of Answer Changing Behavior on a Computer-Based High-Stakes Achievement Test Gail C. Tiemann and Neal M. Kingston Part 4: Detection of Aberrant Responses 13. Identifying Non-Effortful Student Behavior on Adaptive Tests: Implications for Test Fraud Detection Steven L. Wise, Lingling Ma, and Robert A. Theaker 14. A Method for Measuring Performance Inconsistency by Using Score Differences Dennis Maynes Part 5: Multiple Methods 15. Data Forensics: A Compare-and-Contrast Analysis of Multiple Methods Christie Plackner and Vincent Primoli 16. Using Multiple Methods to Detect Aberrant Data Karla Egan and Jessalyn Smith 17. Test Security for Multistage Tests: A Quality Control Perspective Charles Lewis, Yi-Hsuan Lee and Alina A. von Davier

Neal Kingston is the Director of the Achievement and Assessment Institute, Co-Director of the Center for Educational Testing and Evaluation, and Professor of Educational Psychology at the University of Kansas. He has managed all aspects of the educational testing process for both general and alternate assessments, including as an Executive Director at Educational Testing Service, Associate Commissioner for Curriculum and Assessment at the Kentucky Department of Education, Senior Vice President at Measured Progress, Vice President and General Manager at CTB/McGraw-Hill, and Director of CETE at the University of Kansas. He has published and presented more than 100 articles, papers, and book chapters on assessment. Amy Clark is a Research Associate in psychometrics at the Center for Educational Testing and Evaluation at the University of Kansas. She began her career as a classroom teacher and received both her M.S and Ph.D. in Educational Psychology from the University of Kansas specializing in Research, Evaluation, Measurement, and Statistics. Her research interests include exploring potential threats to validity, accountability issues, and diagnostic assessment.