Celebrity endorsements have emerged as a prominent marketing strategy extensively employed by destination marketers to attract tourists through celebrities’ influence. The niche segment of celebrity fans represents a distinct, emotion-driven tourism market, characterized by deep involvement and affection for their favorite celebrities. However, existing studies have primarily focused on the broader impact of celebrity endorsements on general tourists, specially concerning their’ attitudes toward the destination and travel intentions. Yet, there remains a gap in understanding the influence of fans’ affection for a celebrity on their visit intentions to the endorsed destination. This study explores the effect of celebrity endorsements on their fans’ travel behavior, focusing on how fans’ affection for celebrities (“love me”) drives them to visit destinations endorsed by these celebrities (“love the destination”). Utilizing affect transfer theory, the research sequentially employs a questionnaire survey and an experimental design to analyze this phenomenon. In Study 1, the questionnaire was administered to fans of Jay Chou from Taiwan (a province of China) and Stefanie Sun from Singapore, both renowned singers. The findings demonstrate that the degree of celebrity involvement and celebrity worship significantly shape fans’ travel intentions, mediated by factors like cognitive and emotional image of the destination, as well as the fans’ place attachment to the place. Study 2 implements an experimental design to further test the effect of celebrity–destination congruence on the relationship between celebrity involvement and celebrity worship, destination image, and place attachment. Chinese film celebrity Zhu Yilong, born in Wuhan (the provincial capital city of Hubei, China), was selected to endorse three cities: Wuhan, Tianjin, and Urumqi. Conditional process analysis reveals that the relationship between celebrity involvement (or celebrity worship) and travel intention, mediated by place attachment, is further moderated by the celebrity-destination congruency. This study delves into the mechanism underlying the “love me, love my destination” phenomenon among fans in celebrity-endorsed tourism by synthesizing the match-up hypothesis with the affect transfer theory, providing a more comprehensive understanding of affect transfer in celebrity-endorsed tourism. These insights highlight the emotional drivers of fan tourism and offer implications for destination marketers to leverage fan affection by creating emotionally engaging experiences and aligning celebrity endorsements with destination image.