Cullum explained this song: "I've been introducing it onstage as a disco song about the death of the music industry. It's a love song for the music geek in me. The ultimate romantic act back-in-the-day used to be making a mixtape for a girl. I decided the ultimate modern romantic act would be to get your tape player down from the attic and go out and find a Maxell C90 and record a mixtape. It's even more romantic now because of the technological rewind you have to do."
Kacey Musgraves originally offered "Follow Your Arrow" to her friend Katy Perry. However, Perry thought Musgraves should record the song herself as it seemed, "like something that you would totally say."