A Mixed-methods Approach for Assessing Student Learning Gains in English Listening Comprehension

Authors

Dr. Yingchen Wang, School of Internatinal Studies, Shandong Youth University of Political Science, PRC, ChinaFollow
Dr. Xiaoxin Wei, Cambium Assessment Inc., Washington D.C., USAFollow
Yamin Liu, School of Internatinal Studies, Shandong Youth University of Political Science, PRC, ChinaFollow
Tiffany Chiu, Department of Education, Health, and Behavior Studies, University of North DakotaFollow

DOI

https://doi.org/10.7275/zj5z-3s83

Abstract

The growth of literature in student learning gain in recent decades has posed challenges to address the issue within the classroom. To further shed light on this scientific body of knowledge, the current study implemented a sequential explanatory mixed-methods design on a sample of 76 students. First, test data were analyzed using the interval approach to calculate gain scores for the group. The learning gain was small. Second, individual learning gains were analyzed using the pseudo-anchors identified based on estimated Rasch item parameters. T-tests were performed on the stacked pseudo-anchors. While most students exhibited insignificant improvement, 10 students exhibited significant improvements in their anchors. Third, a Wright map was obtained to assess the student’s self-reported gains. Specifically, learning gain was the highest when the students used VOA special English materials. Fourth, learning data were investigated using the multi-facet Rasch model. In the qualitative phase, follow-up interviews were administered and student learning logs were investigated. Thematic analyses and learning pattern examinations revealed the different motivation levels and learning strategies among the students. The connection of quantitative and qualitative analyses provided a more conceptual understanding of student learning gain. The mixed-method approach can be implemented and generalized to other settings, such as the classroom.

Recommended Citation

Wang, Dr. Yingchen; Wei, Dr. Xiaoxin; Liu, Yamin; and Chiu, Tiffany (2022) "A Mixed-methods Approach for Assessing Student Learning Gains in English Listening Comprehension," Practical Assessment, Research, and Evaluation: Vol. 27, Article 12.
DOI: https://doi.org/10.7275/zj5z-3s83
Available at: https://scholarworks.umass.edu/pare/vol27/iss1/12