Off-campus UMass Amherst users: To download campus access dissertations, please use the following link to log into our proxy server with your UMass Amherst user name and password.

Non-UMass Amherst users: Please talk to your librarian about requesting this dissertation through interlibrary loan.

Dissertations that have an embargo placed on them will not be available to anyone until the embargo expires.

Author ORCID Identifier

https://orcid.org/0000-0002-7949-4271

AccessType

Campus-Only Access for Five (5) Years

Document Type

dissertation

Degree Name

Doctor of Philosophy (PhD)

Degree Program

Education

Year Degree Awarded

2020

Month Degree Awarded

February

First Advisor

Stephen G. Sireci

Second Advisor

Ronald K. Hambleton

Third Advisor

Craig S. Wells

Fourth Advisor

Frederic Robin

Subject Categories

Educational Assessment, Evaluation, and Research

Abstract

When there is a time limit for examinees to complete a test, this unequitable test response times (RTs) can be a threat to validity and fairness in testing, if the test performance of those examinees is be affected by the time un-equity. This study proposed a new approach to use response times to create multistage adaptive tests (MSTs) as an attempt to resolve this problem. Specifically, the study investigated the impact of MST design choices on the performance of MST using response time (MST-RT) with respect to the number of stages, the starting stage for speed routing, and the types of speed thresholds. Simulation was used to evaluate the performance of MST-RT in four areas: a) measurement precision, b) overall gains with respect to the total amount of test RTs, c) variability of test RTs, d) proportions of examinees routed to slow modules for various difficulty levels. The results showed that MST-RT achieved similar measurement precision compared to MST, with respect to module test information functions (TIFs) and the root mean square errors (RMSEs) of ability estimates. Moreover, MST-RT conditions showed larger overall gains in terms of the total amount of time that exceeded by examinees, indicating that using MST-RT had an advantage over MST to help examinees complete a test within the time limit. Moreover, MST-RT achieved better RT equitability by reducing the variability in examinee response time, especially for the examinee group whose ability was high, but speed was slow. Particularly, the effect of reducing RT variability was greater when there were more stages, allowing for more frequent speed routings. Finally, using different speed-routing thresholds for different difficulty levels split examinees more evenly between slow and fast modules when the correlation between ability and speed was large. Future studies include investigating the performance of MST-RT when the testing purpose is certification of examinees. Also, investigating how other MST design features and characteristics of items and examinees when constructing MST using response times (RTs) will help obtain a more complete picture regarding the performance of MST-RT.

DOI

https://doi.org/10.7275/15761762

Download
Off-campus Download

Share

COinS