On A Woman a Man Walked By, Harvey shares the limelight with her long-time associate John Parish. The duo previously shared billing on the 1996 album Dance Hall at Louise Point. However the two have worked together since the late 1980s when they were both members of Automatic Diamini.
Harvey told Mojo magazine February 2008 about the album: "I think it follows on from where Dance Hall at Louise Point left off, but it magnifies everything by 100. So all the extremes, the left and the right, the quiet and the loud become more extreme either way."