Julianne Regan in an interview on BBC Radio: "This song happened by accident when we were recording the first album. It was one of those things when we had the day off and we were sat in this very idyllic setting beneath a willow tree besides a stream by this beautiful residential recording studio and it just came out so naturally. It was a miracle of a little song and its very dear to us because of that, because it was very pure, a really happy accident. Everybody went to the pub and we put it down and by the time they got back Martha's Harbour was committed to tape. Martha's Harbour is a fictitious backdrop for this happening."
The song came to more people's attention after a well known incident when the band performed a mimed version on the BBC Television show Top Of The Pops. There was a technical error which meant the band couldn't hear the backing track in their ear pieces and they were shown standing waiting for the song to begin while the studio audience heard the recorded version of the song.
The bedrock of David Guetta's Nicki Minaj-featuring single "Hey Mama" is a sample of "Rosie," a 1940s prison recording from folk archivist Alan Lomax that songwriter Esther Dean first showed the French DJ on YouTube.