Vocalist Hamilton Leithauser told the story of this song in an interview with Express Night Out: "That song was probably the most worked on song on the record, which is weird because it ended up just being this raw, very simple thing. It's just the same part over and over and then we have the big end. We did a country version. We did one that sounded kind of like Sesame Street music. We did one that was rock and really just trying to get everybody involved. And then in the end ... I had the idea for [a] groove change. We recorded that, and then we said, 'Let's just not talk about it until we get to Texas.' So then we went to Texas and it was a one shot deal: Either it works or it doesn't. So we played the recording. Then immediately everybody started playing, hit record and just recorded all of the song. And it was like a gift from the Lord."
"Pink Cadillac" was a B-side for Bruce Springsteen in 1984, but after Aretha Franklin sang about pink Cadillacs on "Freeway Of Love" the following year, Natalie Cole covered the song and had a hit with it in 1988.