Publication:
Item Parameter Drift as an Indication of Differential Opportunity to Learn: An Exploration of item Flagging Methods & Accurate Classification of Examinees

Simple item page
dc.contributor.advisorLisa A. Keller
dc.contributor.advisorCraig S. Wells
dc.contributor.advisorGeorge R. Milne
dc.contributor.authorSukin, Tia M.
dc.contributor.departmentUniversity of Massachusetts Amherst
dc.date2023-09-22T22:20:28.000
dc.date.accessioned2024-04-26T19:47:55Z
dc.date.available2024-04-26T19:47:55Z
dc.date.issued2010-09-01
dc.description.abstractThe presence of outlying anchor items is an issue faced by many testing agencies. The decision to retain or remove an item is a difficult one, especially when the content representation of the anchor set becomes questionable by item removal decisions. Additionally, the reason for the aberrancy is not always clear, and if the performance of the item has changed due to improvements in instruction, then removing the anchor item may not be appropriate and might produce misleading conclusions about the proficiency of the examinees. This study is conducted in two parts consisting of both a simulation and empirical data analysis. In these studies, the effect on examinee classification was investigated when the decision was made to remove or retain aberrant anchor items. Three methods of detection were explored; (1) delta plot, (2) IRT b-parameter plots, and (3) the RPU method. In the simulation study, degree of aberrancy was manipulated as well as the ability distribution of examinees and five aberrant item schemes were employed. In the empirical data analysis, archived statewide science achievement data that was suspected to possess differential opportunity to learn between administrations was re-analyzed using the various item parameter drift detection methods. The results for both the simulation and empirical data study provide support for eliminating the use of flagged items for linking assessments when a matrix-sampling design is used and a large number of items are used within that anchor. While neither the delta nor the IRT b-parameter plot methods produced results that would overwhelmingly support their use, it is recommended that both methods be employed in practice until further research is conducted for alternative methods, such as the RPU method since classification accuracy increases when such methods are employed and items are removed and most often, growth is not misrepresented by doing so.
dc.description.degreeDoctor of Education (EdD)
dc.description.departmentEducation (also CAGS)
dc.identifier.doihttps://doi.org/10.7275/1672130
dc.identifier.urihttps://hdl.handle.net/20.500.14394/38737
dc.relation.urlhttps://scholarworks.umass.edu/cgi/viewcontent.cgi?article=1309&context=open_access_dissertations&unstamped=1
dc.source.statuspublished
dc.subjectclassification accuracy
dc.subjectequating
dc.subjectitem parameter drift
dc.subjectmatrix-sampling design
dc.subjectopportunity to learn
dc.subjectEducation
dc.titleItem Parameter Drift as an Indication of Differential Opportunity to Learn: An Exploration of item Flagging Methods & Accurate Classification of Examinees
dc.typedissertation
dc.typearticle
dc.typedissertation
digcom.contributor.authorisAuthorOfPublication|email:tiacorliss@hotmail.com|institution:University of Massachusetts Amherst|Sukin, Tia M.
digcom.identifieropen_access_dissertations/301
digcom.identifier.contextkey1672130
digcom.identifier.submissionpathopen_access_dissertations/301
dspace.entity.typePublication
Files
Original bundle
Now showing 1 - 1 of 1
No Thumbnail Available
Name:
Sukin_umass_0118D_10543.pdf
Size:
4.69 MB
Format:
Adobe Portable Document Format
Name: Sukin_umass_0118D_10543.pdf
Size: 4.69 MB
Kind: Adobe PDF
Collections
Dissertations