This was released as the second single from Time Will Die and Love Will Bury It, following the album's lead release "Rituals." Vocalist Eva Spence explained to Kerrang:
"We wanted the next single to be something that carried on from the intensity of Rituals, while starting to show other elements to the album. It's rare that we put a repeating chorus in a song - but it's there in amongst some awkward time signatures and the jazz influenced intro that makes it sound like us."
"Cleveland Rocks" was written by an Englishman. Ian Hunter wrote the song after touring America in the late '70s and finding that Cleveland was by far the most receptive city to his brand of Glam Rock.