Off-campus UMass Amherst users: To download dissertations, please use the following link to log into our proxy server with your UMass Amherst user name and password.
Non-UMass Amherst users, please click the view more button below to purchase a copy of this dissertation from Proquest.
(Some titles may also be available free of charge in our Open Access Dissertation Collection, so please check there first.)
A procedure for developing a common metric in item response theory when parameter posterior distributions are known
Abstract
Because item response theory (IRT) models are arbitrarily identified, independently estimated parameters must be transformed to a common metric before they can be compared. To accomplish this, the transformation constants must be estimated and because these estimates are imperfect, there is a propagation of error effect when transforming parameter estimates. However, this error propagation is typically ignored and estimates of the transformation constants are treated as true when transforming parameter estimates to a common metric. To address this shortcoming, a procedure is proposed and evaluated that accounts for the uncertainty in the transformation constants when adjusting for differences in metric. This procedure utilizes random draws from model parameter posterior distributions, which are available when IRT models are estimated using Markov chain Monte Carlo (MCMC) methods. Given two test forms with model parameter vectors Λ Y and ΛX, the proposed procedure works by sampling the posterior of ΛY and Λ X, estimating the transformation constants using these two samples, and transforming sample X to the scale of sample Y. This process is repeated N times, where N is the desired number of transformed posterior draws. A simulation study is conducted to evaluate the feasibility and success of the proposed strategy compared to the traditional strategy of treated scaling constants estimates as error-free. Results were evaluated by comparing the observed coverage probabilities of the transformed posteriors to their expectation. The proposed strategy yielded equal or superior coverage probabilities compared to the traditional strategy for 140 of the 144 comparisons made in this study (97%). Conditions included four methods of estimated the scaling constants and three anchor lengths.
Subject Area
Educational tests & measurements|Quantitative psychology|Educational evaluation
Recommended Citation
Baldwin, Peter, "A procedure for developing a common metric in item response theory when parameter posterior distributions are known" (2008). Doctoral Dissertations Available from Proquest. AAI3315470.
https://scholarworks.umass.edu/dissertations/AAI3315470