All the music on A Woman a Man Walked By was written by Harvey's long-time West Country associate John Parish. He also played most of the instruments, whilst Harvey penned the lyrics.
This song is filled with defiant yelps over crazed guitars. Harvey told The Sun March 26, 2009: "I think the music was so obtuse and peculiar and aggressive and kind of ugly, particularly that song. Whatever I do with John's music, it has to strengthen what's already there. I had to match or better the power of the atmosphere."