Off-campus UMass Amherst users: To download campus access dissertations, please use the following link to log into our proxy server with your UMass Amherst user name and password.
Non-UMass Amherst users: Please talk to your librarian about requesting this dissertation through interlibrary loan.
Dissertations that have an embargo placed on them will not be available to anyone until the embargo expires.
Author ORCID Identifier
Open Access Dissertation
Doctor of Philosophy (PhD)
Year Degree Awarded
Month Degree Awarded
Educational Assessment, Evaluation, and Research
The detection of aberrant responses in psychological and educational testing is crucial for maintaining the validity and reliability of assessment scores. This dissertation evaluates the performance of a new detection method, known as the bootstrap method, specifically targeting random guessing behaviors. A simulation design is used to compare the bootstrap method with traditional cumulative sum control chart (CUSUM) and change-point analysis (CPA) methods, considering various test lengths and severity rates.
The findings indicate that the bootstrap method did not exhibit superior performance in detecting aberrant responses compared to the traditional methods. It demonstrated lower detection power and higher type I error rates, particularly when examinees had similar probabilities of providing correct responses regardless of random guessing or thoughtful answering. While the traditional CUSUM and CPA methods remain effective, the bootstrap method requires further refinement to enhance its detection power.
To address these limitations, future research can explore the integration of additional statistics into the bootstrap method to improve the accuracy of ability estimation, especially when a higher number of aberrant responses are present. This research contributes to the field of educational measurement by highlighting the need for ongoing refinement and optimization of the bootstrap method to effectively identify various forms of aberrant behaviors in psychological and educational testing settings.
The findings underscore the importance of robust detection methods for maintaining the integrity and validity of assessments, emphasizing the significance of ongoing research and development in this area.
Wan, Siyu, "Aberrant Response Detection: Incorporating Cumulative Sum Control Chart and Change-Point Analysis" (2023). Doctoral Dissertations. 2915.
Available for download on Friday, March 01, 2024