The accessory olfactory bulb (AOB) processes pheromonal signals which in turn drive social behaviors. Here we identify a tract of aromatase-expressing (arom+) fibers in the dorsal lateral olfactory tract (dLOT) which terminate in the granule cell layer (GCL) of the AOB. We utilized a retrograde tracer in aromatase reporter animals to delineate the source of these fibers. We show that these input fibers emerge almost exclusively from a contiguous population of arom+ neurons that spans the bed nucleus of the accessory olfactory tract (BAOT) and posterioventral subnucleus of the medial amygdala (MeApv). This population of neurons expresses the estrogen receptor alpha and contains more aromatase neurons in male mice than female mice. Thus, this population of feedback neurons can detect neuroendocrine changes and modulate the output of AOB projection neurons in a way that is sexually dimorphic and could influence every downstream target of the AOB.