Frontman Charlie Fink's lyrics for Last Night on Earth were heavily influenced by poetry. Fink explained in a press release how this song was influenced by Frank O'Hara's poem Having a Coke With You, which he read for the first time in 2010: "I wanted ['Just Before We Met '] to have a feel of that, or a tempo. Just the notion of it. Having a Coke With You is I think a very romantic poem, and that's the only outwardly romantic song on the album, and so I wanted it to have the same approach."
Bruce Springsteen wrote "Blinded By The Light," which was a #1 hit for Manfred Mann's Earth Band. The "Madman Drummers" line is a reference to Springsteen's first E-Street drummer, Vinnie "Mad dog" Lopez.