Vampire Weekend debuted this song on July 12, 2012 at their show at the House of Blues in Cleveland, when it was tentatively titled "New Song No. 2." Four and a half months later, it was performed on the Halloween episode of Jimmy Kimmel Live!, now with its official title, "Unbelievers" with the band made up as skeletons. Lead singer Ezra Koenig told NME: "Even though that song has some heavy themes about trying to figure out your place in the world and what you believe in and how your beliefs affect your relationships to other people, we don't mind goofing off a little bit."
Koenig explained the song's meaning to NME: "There's something very powerful about religious imagery," he said, "and I've always been very interested in various religions, whether it was the one I was raised in or ones that I experienced myself. I don't like to be dismissive in any way of religion. In some ways I feel very connected to it, some of the imagery in that song, talking about the fire, is something that I find powerful.
Something that I started thinking about on the last record (was) how much I can't relate to any ways of thinking that divide the world into two distinct parts," he continued. "Whether that's atheism versus religion or Republican versus Democrat, there's all these false dichotomies in the world that can be very confusing when you're trying to make up your mind about how you feel about a situation. It's almost like a sports mentality, about being on one team or the other. That doesn't mean that I don't have my own deeply helf beliefs about things that I think are right or moral or true but - and it's only gotten worse as I've gotten older - I've always had this extreme dislike of these false choices that you're presented with, so in any situation I want to try to be open-minded and look for what you can learn from something. Even in that song, the idea of being an unbeliever can mean almost anything because everybody is an unbeliever to somebody else. It doesn't matter how fervently you believe in your faith, there's always going to be another faith that calls you an unbeliever. In any sort of multicultural society, it's something that people have to grapple with and figure out."
Ezra Koenig told NME: "This came during a period where we hadn't written anything we were excited about, so it was a perfect moment. The lyrics and American-ness are new for us."
Though Koenig and multi-instrumentalist Rostam Batmanglij found the writing for Modern Vampires hard work this song came relatively effortlessly. "Rostam and I were over at his apartment," the singer recalled to MTV News, "he started playing on his piano, I had my laptop, started writing lyrics, and within a couple of hours we had all the main parts. So that was easy."