Singer and guitarist Dan Auerbach told The Sun March 27, 2008: "We had many versions of this song in demo form but when we went to record it we decided we wanted to change it up again. Brian (producer Danger Mouse) gave us a tempo and we rolled with it, adapting the song to the new tempo, and it all just seemed to fall nicely into place."
The lyrics for "Mary, Did You Know?" were written by Christian singer and comedian Mark Lowry, after his pastor asked him to write a Christmas musical for their church. Southern gospel musician Buddy Greene later added music to his words.