Off-campus UMass Amherst users: To download dissertations, please use the following link to log into our proxy server with your UMass Amherst user name and password.
Non-UMass Amherst users, please click the view more button below to purchase a copy of this dissertation from Proquest.
(Some titles may also be available free of charge in our Open Access Dissertation Collection, so please check there first.)
USING RESIDUAL ANALYSES TO ASSESS ITEM RESPONSE MODEL-TEST DATA FIT (MEASUREMENT TESTING)
Abstract
Statistical tests are commonly used for studying item response model-test data fit. But, many of these tests have well-known problems associated with them. The biggest concern is the confounding of sample size in the interpretation of fit results. In the study, the fit of three item response models was investigated using a different approach: exploratory residual procedures. These residual techniques rely on the use of judgment for interpreting the size and direction of discrepancies between observed and expected examinee performances. The objectives of the study were to investigate if exploratory procedures involving residuals are valuable for judging instances of model-data fit, and to examine the fit of the one-parameter, two-parameter, and three-parameter logistic models to National Assessment of Educational Progress (NAEP) and Maryland Functional Reading Test (MFRT) data. The objectives were investigated by determining if judgments about model-data fit are altered if different variations of residuals are used in the analysis, and by examining fit at the item, ability, and overall test level using plots and simple summary statistics. Reasons for model misfit were sought by analyzing associations between the residuals and important item variables. The results showed that the statistics based on average raw and standardized residuals provided useful fit information, but that when compared, the statistics based on standardized residuals presented a more accurate picture of model-data fit and therefore, provided the best overall fit information. Other results revealed that with the NAEP and MFRT type of items, failure to consider variations in item discriminating power resulted in the one-parameter model providing substantially poorer fits to the data sets. Also, guessing on difficult NAEP multiple-choice items affected the degree of model-data fit. The main recommendation from the study is that because the residual analyses provide substantial amounts of empirical evidence about fit, practitioners should consider these procedures as one of the several types of strategies to employ when dealing with the goodness of fit question.
Subject Area
Educational evaluation
Recommended Citation
MURRAY, LINDA NORINE, "USING RESIDUAL ANALYSES TO ASSESS ITEM RESPONSE MODEL-TEST DATA FIT (MEASUREMENT TESTING)" (1985). Doctoral Dissertations Available from Proquest. AAI8509581.
https://scholarworks.umass.edu/dissertations/AAI8509581