Breeders frontwoman Kim Deal told Uncut the story of the song:
"I heard 'All Nerve' in my head while I was sleeping, and I woke up with that song in my head. It's a very dramatic song - melodrama I guess. When I had it in my head it was like a Bronski Beat-esque sort of thing - I could hear a guy singing it, it wasn't me. I heard the instrumentation, and it was very reverbed-out guitars, and thin drums, and sung by one of those cruel, yet hurt, gay singers from the 80s - like Bronski Beat, they're very cruel but fragile at the same time."
Kim Deal worked up a demo to show Breeders drummer James McPherson. She recalled, "He sits at the drum kit and starts playing with his huge foot on the kick drum all of a sudden... I'm like, 'OK, well, this song is gonna sound different now.'"