The use of educational tests for making high stakes decisions has had societal consequences for decades. Parents, teachers, and administrators have been willing to pay off, lie, cheat, and steal so that their children, students, and they themselves would not fall prey to the negative consequences of subpar performance on educational assessments. Respected psychometric scholars have supported Samuel Messick’s claim over the years, but their advocacy has caught minimal traction. I founded an initiative in 2019 – The Wrong Answer Project – that shows promise as a vehicle for collecting validity evidence based on the social consequences of testing and raising the voices of perpetually marginalized groups. My hope is to ignite grassroots solutions for achieving more equity centered designs in educational testing. This study is an investigation of the usefulness of my initiative by exploring the pilot data from the 2019 implementation using an evaluability assessment approach. The findings from the assessment supports strategy that evolves my project to a more manageable product in service of social justice for all who have been oppressed by high stakes tests.