This is one of several songs on Lullaby And… The Ceaseless Roar that addresses Robert Plant's decision to break up with American singer Patty Griffin, and return to his native England. He told Billboard magazine this, "is about the regret of turning around."
The song features Plant's good friend, the Welsh-language folk singer Julie Murphy, performing the verse of a 14th century poem Marwnad yr Ehedydd in her native language. The pair previously worked together in 2001 when they sung a duet "Life Begin Again" on Afro Celt Sound System's album Volume 3: Further in Time.
"Cleveland Rocks" was written by an Englishman. Ian Hunter wrote the song after touring America in the late '70s and finding that Cleveland was by far the most receptive city to his brand of Glam Rock.