Yazoo, who were known in America as Yaz, was the duo of instrumentalist/programmer Vince Clarke and vocalist Alison Moyet. Clarke was an original member of Depeche Mode, and wrote this song for them. He was about to leave the band in late 1981, and he wanted them to have a good single starting off 1982. Depeche Mode didn't want it; instead they released "See You," a song Martin Gore wrote when he was only 15. This song ended up on Yazoo's debut album.
This was the first song Clarke and Moyet recorded together. In our 2010 interview with Vince Clarke
, he explained: "One of the big reasons I asked Alison to sing 'Only You' was because I felt that it was a ballad that needed someone with a soulful voice. And it wasn't like I thought to myself, 'Okay, you have a chance to marry electronics with song.' It just turned out that way, and it happened Alison had a great voice, and it worked really well."