Off-campus UMass Amherst users: To download campus access dissertations, please use the following link to log into our proxy server with your UMass Amherst user name and password.
Non-UMass Amherst users: Please talk to your librarian about requesting this dissertation through interlibrary loan.
Dissertations that have an embargo placed on them will not be available to anyone until the embargo expires.
Author ORCID Identifier
AccessType
Campus-Only Access for Five (5) Years
Document Type
dissertation
Degree Name
Doctor of Philosophy (PhD)
Degree Program
Education
Year Degree Awarded
2020
Month Degree Awarded
February
First Advisor
Stephen G. Sireci
Second Advisor
Ronald K. Hambleton
Third Advisor
Craig S. Wells
Fourth Advisor
Frederic Robin
Subject Categories
Educational Assessment, Evaluation, and Research
Abstract
When there is a time limit for examinees to complete a test, this unequitable test response times (RTs) can be a threat to validity and fairness in testing, if the test performance of those examinees is be affected by the time un-equity. This study proposed a new approach to use response times to create multistage adaptive tests (MSTs) as an attempt to resolve this problem. Specifically, the study investigated the impact of MST design choices on the performance of MST using response time (MST-RT) with respect to the number of stages, the starting stage for speed routing, and the types of speed thresholds. Simulation was used to evaluate the performance of MST-RT in four areas: a) measurement precision, b) overall gains with respect to the total amount of test RTs, c) variability of test RTs, d) proportions of examinees routed to slow modules for various difficulty levels.
The results showed that MST-RT achieved similar measurement precision compared to MST, with respect to module test information functions (TIFs) and the root mean square errors (RMSEs) of ability estimates. Moreover, MST-RT conditions showed larger overall gains in terms of the total amount of time that exceeded by examinees, indicating that using MST-RT had an advantage over MST to help examinees complete a test within the time limit. Moreover, MST-RT achieved better RT equitability by reducing the variability in examinee response time, especially for the examinee group whose ability was high, but speed was slow. Particularly, the effect of reducing RT variability was greater when there were more stages, allowing for more frequent speed routings. Finally, using different speed-routing thresholds for different difficulty levels split examinees more evenly between slow and fast modules when the correlation between ability and speed was large.
Future studies include investigating the performance of MST-RT when the testing purpose is certification of examinees. Also, investigating how other MST design features and characteristics of items and examinees when constructing MST using response times (RTs) will help obtain a more complete picture regarding the performance of MST-RT.
DOI
https://doi.org/10.7275/15761762
Recommended Citation
Park, Yooyoung, "CONSTRUCTING MULTISTAGE ADAPTIVE TESTS USING RESPONSE TIMES" (2020). Doctoral Dissertations. 1860.
https://doi.org/10.7275/15761762
https://scholarworks.umass.edu/dissertations_2/1860