Phoenix singer Thomas Mars explained to the Washington DC free newspaper Express that Wolfgang Amadeus Phoenix producer Philippe Zdar was responsible for this song. Said Mars "He left with this demo that we threw away and when he came back he already had memories with it. He had the best holiday. And he said, 'There's no way this song isn't going be on the record; if it's not on the record I'll leave right now.' We tried to incorporate it and it really didn't fit [at first]. We try to make [the tunes] breathe and this song is really the most electronic... It's basically like giving life to these [electronic] instruments."
The lyrics for "Mary, Did You Know?" were written by Christian singer and comedian Mark Lowry, after his pastor asked him to write a Christmas musical for their church. Southern gospel musician Buddy Greene later added music to his words.