Written by David Mann and Bob Hilliard, this song was first introduced as the title track of Frank Sinatra's melancholy album In the Wee Small Hours in an arrangement by Nelson Riddle.
The album of torch songs about loneliness and heartache, drawn from Sinatra's personal struggles with his divorce from his first wife, Nancy, and his troubled marriage to actress Ava Gardner, is considered to be one of the first concept albums. Charles Pignone, Senior Vice President of Frank Sinatra Enterprises, explained the crooner's thought process behind his series of concept albums in a Songfacts interview: "He had envisioned that instead of people just making singles, to do a whole album thematically so it would hold people's attention. It's all changed now because everything is digital or on an iPod, and people shuffle it. You have to remember what a revolutionary concept - pardon the pun - it was because I think Frank felt that when people would drop that needle down, there's 15 or 18 or 20 minutes on one side, and it's pretty jarring if you're all over the place with a ballad, then a tempo song. He really came up with the idea of doing these concept albums where they thematically, musically and orchestrally would fit together, and a lot of work went into that."
The album peaked at #2 on the US albums chart for two weeks.
This song came at the last minute when Hilliard convinced Mann to stick around his New Jersey home for some late-night songwriting.
This song was covered by several artists, including Johnny Mathis, Ella Fitzgerald, Barbra Streisand, Carly Simon, Liza Minnelli, Rosemary Clooney, and Diana Krall.
Law & Order: Criminal Intent borrowed the album title for a season 5 episode in 2005.
Carly Simon's version was used on the popular soundtrack to Sleepless in Seattle (1993), starring Tom Hanks and Meg Ryan.