Off-campus UMass Amherst users: To download dissertations, please use the following link to log into our proxy server with your UMass Amherst user name and password.
Non-UMass Amherst users, please click the view more button below to purchase a copy of this dissertation from Proquest.
(Some titles may also be available free of charge in our Open Access Dissertation Collection, so please check there first.)
The impact of local dependencies on various IRT outcomes
Abstract
This research explores the effect of violations of the IRT local independence assumption. The assumption states that, conditional on ability, the responses of test takers to the items on a test are statistically independent. While this assumption is critical for the application of IRT to test data, it is such a strict requirement that it is unlikely to be met completely by any test. This research examines the extent to which the local independence assumption is violated in specific testing situations, and uses this information to determine the effect various levels of dependence have on IRT-based outcomes. Three tests, the LSAT, P-ACT+, and GMAT, were studied using the Q$\sb3$ statistic to evaluate the degree to which the local independence assumption is violated in practice. Each test examined violated the assumption to some degree. As expected, there was more dependence within test sections F than between test sections, and sections with item sets displayed more dependence than those without item sets. Within test sections, more dependence was displayed within item sets than between item sets. Based on these results, four dependence levels (zero, low, medium and high) were defined, and data were simulated to recover these dependencies. The simulated data were then compared to the true data to analyze the effect of these dependencies on calibration results and score distributions. The results indicated that high levels of dependency cause low scores to be underestimated and high scores to be overestimated. The expected effects of this result were observed for the item parameters, ability parameters and item and test characteristic curves. In terms of the score distribution, a normally distributed population of scores is spread out at the tails and flattened in the center as a result of a greater number of low and high scores. For the most part, the effects observed were not problematic for low to medium levels of dependence. These results have implications for many IRT applications, such as test assembly, equating, differential item functioning, and computer adaptive testing.
Subject Area
Educational evaluation|Educational psychology|Psychological tests
Recommended Citation
Fennessy, Lynda M, "The impact of local dependencies on various IRT outcomes" (1995). Doctoral Dissertations Available from Proquest. AAI9524701.
https://scholarworks.umass.edu/dissertations/AAI9524701