How does the subjective experience of categorization interact with the perception of gender? When observers perform a binary categorization task on stimuli manipulated along a continuous dimension, stimuli that can unambiguously fit into either category elicit more positive appraisals than those that cannot. A potential mechanism for this phenomenon is that the relative processing fluency of the categorization adds a positive valence to the associated stimuli (Winkielman et al., 2003). This extends to social categories: when observers sort human cross-gender facial blends according to conventional binary gender (”male” vs ”female”), androgynous faces are subsequently rated as as less attractive (Owen et al., 2016). We expand this paradigm by adding label conditions designed to increase the fluency of the androgynous faces rather than the gender-conforming ones. While the alternate categorical labels reliably changed RT and attractiveness ratings given to the stimuli, overall response patterns did not straightforwardly fit a task-based fluency account alone. Through modeling, we explore multiple ways to best characterize the observed trends.