Off-campus UMass Amherst users: To download campus access dissertations, please use the following link to log into our proxy server with your UMass Amherst user name and password.
Non-UMass Amherst users: Please talk to your librarian about requesting this dissertation through interlibrary loan.
Dissertations that have an embargo placed on them will not be available to anyone until the embargo expires.
Author ORCID Identifier
AccessType
Open Access Dissertation
Document Type
dissertation
Degree Name
Doctor of Philosophy (PhD)
Degree Program
Education
Year Degree Awarded
2019
Month Degree Awarded
May
First Advisor
Craig S. Wells
Second Advisor
April L. Zenisky
Third Advisor
Stephen G. Sireci
Fourth Advisor
Timothy C. Davey
Subject Categories
Educational Assessment, Evaluation, and Research
Abstract
In multistage-adaptive testing (MST), there are many interrelated design variables that impact the nature and quality of ability estimation. Previous research has identified general principles for the effective design of MSTs in terms of measurement performance. However, those principles are unlikely to apply uniformly to every testing context.
The purpose of this dissertation is to propose a process of finding an MST design that has optimal measurement properties, given a specific set of test circumstances. To achieve this goal, an efficient strategy was introduced at each of three phases to discover the optimal design of the MST; constructing MSTs, systematically searching a design space of the MST, and evaluating the MST performance. For the first phase, a top-down approach was applied in this study. For the second phase, a way to systematically search the parameterized design space of an MST was used. For the third phase, a new analytical evaluation method for MST was proposed.
In the dissertation, Study 1 proposed a new analytical evaluation method for MST. Using this new approach, measurement precision of ability estimation and classification accuracy could be derived analytically. The simulation results indicated that the new analytical method produced more exact measurement properties of an MST than the Monte Carlo simulation method. Therefore, the new analytical method would be the most efficient and competitive tool to asses measurement performance of an MST among other evaluation methods.
Study 2 proposed a process to find a design of an MST that shows optimal measurement properties applying the three efficient strategies, given a specific set of testing context. The process consists of four important features: (1) setting a testing circumstance and MST design space, (2) systematically searching the MST design space using the top-down approach, (3) analytically evaluating measurement performance of an MST, and (4) computing objective functions. The suggested process was applied to a real item pool from a large-scale assessment. The results of the application study provided evidence that the process could be generalized to more complex and realistic test circumstances to create optimal designs of MST.
DOI
https://doi.org/10.7275/14218580
Recommended Citation
LIM, HWANGGYU, "A TOP-DOWN APPROACH FOR OPTIMALLY DESIGNING MULTISTAGE-ADAPTIVE TESTS" (2019). Doctoral Dissertations. 1644.
https://doi.org/10.7275/14218580
https://scholarworks.umass.edu/dissertations_2/1644