This song originally appeared on the compilation album Golden as an outtake from Failure's 1992 Comfort record. It was later re-recorded with a new bridge for The Heart Is a Monster.
The Heart Is a Monster is also a lyric on the retooled version of this song. Multi-instumentalist Greg Edwards explained to About.com: "I had that title when we were kicking around the idea of naming the album. And then Kellii and Ken both really liked it. Soon after we decided that would be the name of the record we were working on that bridge for 'Petting the Carpet'. I was writing lyrics and decided spur of the moment to work that title into the song."
There was a rumor in the Soviet Union that The Beatles had secretly visited the U.S.S.R. and given a private concert for the children of top Communist party members. They believed the song "Back In The U.S.S.R." was written because of the concert. Actually, some fans still believe so.