Lead singer Ed Macfarlane told Drowned In Sound the story of this song: "Before this song we only wrote twelve minute post-rock instrumentals. We attempted to try and write a short song and discovered we could actually do a great job of it. The other members of the band forced me to sing and before we knew it we had ourselves a pop song. The song was two years old before it made it to record."
The lyrics for "Mary, Did You Know?" were written by Christian singer and comedian Mark Lowry, after his pastor asked him to write a Christmas musical for their church. Southern gospel musician Buddy Greene later added music to his words.