Frontman John Grant told Uncut magazine January 2014 about the background to this song: "'My Love' was me putting one of my favorite pieces of Russian animation in a song. We sampled it, and even got permission from the wife of the guy who did it (Vadim Kurchevsky). It was called 'My Green Crocodile.' It's about a crocodile who falls in love with a beautiful cow."
"Piano Man" was inspired by Billy Joel's time playing at a piano bar in Los Angeles. The "real estate novelist" was a guy who always talked about writing a book, but spent all his spare time in the bar.
Kacey Musgraves originally offered "Follow Your Arrow" to her friend Katy Perry. However, Perry thought Musgraves should record the song herself as it seemed, "like something that you would totally say."