Jez Williams (guitarist) told Drowned in Sound: "The lyrics are inspired by a haunted house really. The song itself though is us trying to write a classic garage rock song." He added: "I guess the main inspiration behind that song was rawness. We actually wanted it to sound like it was recorded on a tape recorder, so that's what we did! We managed to get hold of this authentic 1970s tape recorder and the song was recorded entirely on that piece of equipment." Williams concluded: "To me its also reminiscent of heavier elements of Northern Soul. R Dean Taylor's 'There's A Ghost In My House' is one of my favourite songs of all time and to me it's kind of our take on that."

Andy Williams (drums) revealed to Drowned In Sound that this song "almost didn't make the final selection on Kingdom of Rust. It was that close to getting binned." Jez Williams added: "We literally stripped it bare, started again, then re-built it. It was one of those we had to wrestle to the ground, literally beat the living daylights out of it!"

Another inspiration for this song were the Nuggets and Pebbles compilation albums of 1960s garage and psychedelic rock that were put together by Jac Holzman, founder of Elektra Records, and Lenny Kaye, later lead guitarist for Patti Smith.

The Doves told the NME April 11, 2009: "We mixed this through a 1970s East German tape recorder so it sounded crunched. It's full of attitude, it's about your past haunting you."