Off-campus UMass Amherst users: To download campus access dissertations, please use the following link to log into our proxy server with your UMass Amherst user name and password.
Non-UMass Amherst users: Please talk to your librarian about requesting this dissertation through interlibrary loan.
Dissertations that have an embargo placed on them will not be available to anyone until the embargo expires.
Author ORCID Identifier
https://orcid.org/0000-0002-5687-9208
AccessType
Open Access Dissertation
Document Type
dissertation
Degree Name
Doctor of Philosophy (PhD)
Degree Program
Education
Year Degree Awarded
2023
Month Degree Awarded
February
First Advisor
Stephen Sireci
Second Advisor
Jennifer Randall
Third Advisor
April Zenisky
Fourth Advisor
Michael Walker
Subject Categories
Educational Assessment, Evaluation, and Research
Abstract
In this Dissertation, I will present the theoretical basis and two proof-of-concept studies of something I will call an “Experiential” approach to Test Design and Validation. This approach finds its origins in Game design, where the effort goes around the experience elicited by a device (a game). The idea of the experiential approach is to apply the mindset of game design into test design.
First, I present a literature review of conceptual frameworks referring to “Experience” and a novel conceptual framework for the context of Educational Testing. Then, I do a literature review of scientific epistemologies and feminist epistemologies in particular, that will set the stage for an Experiential approach to Validation.
After this, I present the general idea of an Experiential approach to Test Validation. This approach includes four questions: in what ways is this validity argument partial and situated? (Partiality question), is it ethical to use this test? (Appropriateness question), is it the test adequate for its purposes? (Adequacy question), why do I prefer this test versus other ethical and adequate tests? (Preferability question). I refer to this as the P.A.A.P. minimum in Test Validation.
I continue the Dissertation with two empirical studies that will serve as proofs-of-concept for each, the design and validation part of the experiential approach. Across both studies, I will respond to two conceptual hypotheses about the Experiential approach: what is the value above and beyond other approaches? And, is it practical/feasible to implement the Experiential approach?
First, I present the work done with Professor Lisa Keller about a diagnostic test we built together for her regression class at UMass Amherst. Second, I present an analysis done on information coming from Twitter (around 1,200 tweets) about the experiences of test users and other stakeholders in Chile relative to a test used there for college admission during the COVID-19 pandemic.
I finish this Dissertation with a Discussion about the Experiential approach. Overall, the experiential approach proposes something valuable above and beyond existing methodologies and is feasible to implement.
DOI
https://doi.org/10.7275/33072110
Recommended Citation
Araneda Galarce, Sergio Andres, "AN EXPERIENTIAL APPROACH TO TEST DESIGN AND VALIDATION" (2023). Doctoral Dissertations. 2725.
https://doi.org/10.7275/33072110
https://scholarworks.umass.edu/dissertations_2/2725
Creative Commons License
This work is licensed under a Creative Commons Attribution-Noncommercial 4.0 License