The British producers Tony Macaulay and Barry Mason wrote this song with Sylvan Mason, who was Barry's wife at the time. (Sylvan is often uncredited, but her divorce agreement provides hard evidence that she co-wrote this song and the Tom Jones hit "Delilah
.") The song is about a free spirit named Rosemary who leaves the singer besotted.
Many have claimed to be the actual Rosemary the song is about, but Sylvan Mason says that like Delilah, no such person exists. She told Songfacts: "Tony [Macaulay] came over with a melody and rough idea for a song which title originally was 'It's My Heart You'll Be Breaking Apart' but he said he wanted to put a girl's name in the title because that's what sold records in those days. The girl's name Rosemary fitted with the title so we started the song from scratch merely using the name Rosemary."