Frontman Charlie Fink harks back on this autobiographical song to the innocence, simplicity and camaraderie of N&TH's early days playing in west London folk-den Bosun's locker bar. He used to play there alongside his former girlfriend, songstress Laura Marling, and his old schoolmate, Mumford & Sons banjo player Winston Marshall. "I'll give it all back just to do it again," he sings. "Turn back time, be with my friends."
The song's music video was directed by Fink and stars children playing younger versions of the Noah & the Whale band members. Fink told NME: "The song is the story of a young band's first performance, and so the video is a similar thing."
The lyrics for "Mary, Did You Know?" were written by Christian singer and comedian Mark Lowry, after his pastor asked him to write a Christmas musical for their church. Southern gospel musician Buddy Greene later added music to his words.