Off-campus UMass Amherst users: To download dissertations, please use the following link to log into our proxy server with your UMass Amherst user name and password.
Non-UMass Amherst users, please click the view more button below to purchase a copy of this dissertation from Proquest.
(Some titles may also be available free of charge in our Open Access Dissertation Collection, so please check there first.)
An investigation of the effects of conditioning on two ability estimates in DIF analyses when the data are two-dimensional
Abstract
Differential Item functioning is present when examinees of the same ability, but belonging to different groups, have differing probabilities of success on an item. Traditionally, DIF detection procedures have been implemented conditioning on total test score. However, if there are group differences on the abilities underlying test performance, and total score is used as the matching criterion, multidimensional item impact may be incorrectly identified as DIF. This study sought to confirm earlier research which demonstrated that multidimensional item impact may be identified as DIF, and then to determine whether conditioning on multiple ability estimates would improve item classification accuracy. Data were generated to simulate responses for 1000 reference group members and 1000 focal group members to two-dimensional tests. The focal group mean on the second ability was one standard deviation less than the reference group mean. The dimensional structure of the tests, the discrimination of the items, and the correlation between the two abilities were varied. Logistic regression and Mantel-Haenszel DIF analyses were conducted using total score as the matching criterion. As anticipated, substantial numbers of items were identified as DIF. Items were then selected into subtests based on item measurement direction. The logistic regression procedure was re-implemented, with subtest scores substituted for total score. In the majority of the conditions simulated, this change in criterion resulted in substantial reductions in Type I errors. The magnitude of the reductions were related to the dimensional structure of the test, and the discrimination of the items. Finally, DIF analyses of two real data sets were conducted, using the same procedures. For one of the two tests, substituting subtest scores for total score resulted in a reduction in number of items identified as DIF. These results suggest that multidimensionality in a data set may have a significant impact on the results of DIF analyses. If total score is used as the matching criterion very high Type I error rates may be expected under some conditions. By conditioning on subtest scores in lieu of total score in logistic regression analyses it may be possible to substantially reduce the number of Type I errors, at least in some circumstances.
Subject Area
Educational tests & measurements|Quantitative psychology|Educational evaluation
Recommended Citation
Mazor, Kathleen Michele, "An investigation of the effects of conditioning on two ability estimates in DIF analyses when the data are two-dimensional" (1993). Doctoral Dissertations Available from Proquest. AAI9408310.
https://scholarworks.umass.edu/dissertations/AAI9408310