Fairness Concerns of Discrete Option Multiple Choice Items

Authors

Carol Eckerly
Russell Smith
John Sowles

DOI

https://doi.org/10.7275/chaw-y360

Abstract

The Discrete Option Multiple Choice (DOMC) item format was introduced by Foster and Miller (2009) with the intent of improving the security of test content. However, by changing the amount and order of the content presented, the test taking experience varies by test taker, thereby introducing potential fairness issues. In this paper we investigated fairness concerns by evaluating the impact on test takers of the differing testing experiences when items are administered in the DOMC format. Specifically, we described the impact of the presentation order of the key on item difficulty and discrimination as well as the cumulative impact at the test level. We recommend not including DOMC items in exams until the methodology of scoring test takers on these items is revised to address specific fairness concerns identified in this paper. Accessed 1,375 times on https://pareonline.net from November 02, 2018 to December 31, 2019. For downloads from January 1, 2020 forward, please click on the PlumX Metrics link to the right.

Comments

https://doi.org/10.7275/chaw-y360

Creative Commons License

This work is licensed under a Creative Commons Attribution-NonCommercial-No Derivative Works 4.0 International License.

Recommended Citation

Eckerly, Carol; Smith, Russell; and Sowles, John (2019) "Fairness Concerns of Discrete Option Multiple Choice Items," Practical Assessment, Research, and Evaluation: Vol. 23, Article 16.
DOI: https://doi.org/10.7275/chaw-y360
Available at: https://scholarworks.umass.edu/pare/vol23/iss1/16