English Post-Punk group Savages open their debut album with this track, which establishes the idea of 'silencing yourself' in order to gain some focus in the cacophony of modern life.
The song opens with an excerpt from the John Cassavetes' 1977 movie Opening Night, which is about a Broadway actress coming to terms with getting older. The clip was used at the suggestion of co-producer Johnny Hostile.
The artistic statement "You are distracted. You are available" is printed on the cover of Silence Yourself. The same manifesto is read aloud in this song's video by singer Jehnny Beth.
The lyrics for "Mary, Did You Know?" were written by Christian singer and comedian Mark Lowry, after his pastor asked him to write a Christmas musical for their church. Southern gospel musician Buddy Greene later added music to his words.