The song features gospel vocals by the legendary Waters Sisters, who once backed Michael Jackson on his hit "Wanna Be Startin' Somethin'
." Fink discussed their contribution in an interview with Gigwise
: "It was genuinely amazing. I basically said to Jason, who we made the record with, that I wanted some gospel vocals, and he said that if I wanted that type of sound they were who I needed to see. He had already recorded with them on a Jenny Lewis track. They were one of the great pleasures of the album. One of my favourite parts was when we were all around the piano singing 'Old Joy' for the first time together, working out harmonies. There's just something magical when you get a harmony right. It was a real treat to get to play with them."