Loading...
Citations
Abstract
When there is a time limit for examinees to complete a test, this unequitable test response times (RTs) can be a threat to validity and fairness in testing, if the test performance of those examinees is be affected by the time un-equity. This study proposed a new approach to use response times to create multistage adaptive tests (MSTs) as an attempt to resolve this problem. Specifically, the study investigated the impact of MST design choices on the performance of MST using response time (MST-RT) with respect to the number of stages, the starting stage for speed routing, and the types of speed thresholds. Simulation was used to evaluate the performance of MST-RT in four areas: a) measurement precision, b) overall gains with respect to the total amount of test RTs, c) variability of test RTs, d) proportions of examinees routed to slow modules for various difficulty levels. The results showed that MST-RT achieved similar measurement precision compared to MST, with respect to module test information functions (TIFs) and the root mean square errors (RMSEs) of ability estimates. Moreover, MST-RT conditions showed larger overall gains in terms of the total amount of time that exceeded by examinees, indicating that using MST-RT had an advantage over MST to help examinees complete a test within the time limit. Moreover, MST-RT achieved better RT equitability by reducing the variability in examinee response time, especially for the examinee group whose ability was high, but speed was slow. Particularly, the effect of reducing RT variability was greater when there were more stages, allowing for more frequent speed routings. Finally, using different speed-routing thresholds for different difficulty levels split examinees more evenly between slow and fast modules when the correlation between ability and speed was large. Future studies include investigating the performance of MST-RT when the testing purpose is certification of examinees. Also, investigating how other MST design features and characteristics of items and examinees when constructing MST using response times (RTs) will help obtain a more complete picture regarding the performance of MST-RT.
Type
Dissertation (5 Years Campus Access Only)
Date
2020-02
Publisher
Degree
License
License
Files
Draft_Dissertation_YP_submit2.pdf
Adobe PDF, 4.88 MB
- Embargoed until 2029-06-01