R.E.M. lead singer Michael Stipe talked to Rolling Stone, November 12, 2009, after releasing their album Live at the Olympia, which included this track. Said Stipe: "It's about a couple of swingers, a straight couple who employ younger men and women and seduce them. I did not succumb. But I was aware of the danger and electricity they created. The reason I said 'Not quite a family story' was three of my goddaughters, all under 14, were in the audience. This was not appropriate for them to hear from their godfather onstage."
This song, and others from the album, is part of a short film called Left of Reckoning, directed by James Herbert. The four-minute clip has the band wandering around folk artist R.A. Miller's Whirligig Farm in Rabbittown, Georgia.
Kacey Musgraves originally offered "Follow Your Arrow" to her friend Katy Perry. However, Perry thought Musgraves should record the song herself as it seemed, "like something that you would totally say."
Geffen Records made history on June 27, 1994 when Aerosmith's "Head First" became the first major label song made available for exclusive digital download. Download speeds at the time were so slow it took around 75 minutes to download the track.