This heartfelt ballad was written by Jack Wolf, Joel Herron, and Frank Sinatra. The crooner first recorded it on March 27, 1951 with the Ray Charles singers in an arrangement by Axel Stordahl.

That's right, Sinatra has a writing credit on this one, a rarity for a singer before the late-'50s when entertainers would begin taking over their own songwriting duties rather than rely on professional songwriters. At the time, Sinatra had left his first wife, Nancy, to marry actress Ava Gardner, a destructive union that caused much heartache on both sides. As he was awaiting his impending divorce from Nancy, before he would marry Ava that November, he brought his emotional distress to the studio when he recorded this song.

"Frank changed part of the lyric, and made it say what he felt when he was doing it. We said, 'He's gotta be on this song!' and we invited him in as co-writer," Joel Herron remembered in Frank Sinatra: An American Legend by Nancy Sinatra (Frank's daughter).

Author/music critic Will Friedwald claimed in a Columbia retrospective that Frank was "so overcome with grief, that he bolted from the studio in tears."

Charles Pignone, Senior Vice President of Frank Sinatra Enterprises, doubts the veracity of this part of the story. He said in a Songfacts interview: "I'm not sure how true it is because looking at the session sheets on that, he would do another song after "I'm A Fool To Want You" at the session. Sometimes these stories get handed down. Without talking to a person that was actually there or talking to Frank about it, it's hard to say. But I do think that that was a very emotional song for him, and if he was given credit for the lyrics on that, he must have contributed something that the songwriters felt was viable and contributed to the overall quality of the song."

This became a jazz and pop standard recorded by many artists, including Peggy Lee, Tony Bennett, Chet Baker, Sammy Davis Jr., Elvis Costello, Billie Holiday, Linda Ronstadt and Carly Simon.

Sinatra recorded another version of this song in 1957 at the Capitol Tower in Hollywood, this time with arranger Gordon Jenkins. It was released on Where Are You?, his first album with Capitol Records.

In 2015 Bob Dylan released Shadows in the Night, an album consisting of covers of traditional pop standards made famous by Frank Sinatra. He recorded this tune for the record's opening track. Dylan told AARP magazine: "A song like 'I'm a Fool to Want You': I know that song. I can sing that song. I've felt every word in that song. I mean, I know that song. It's like I wrote it. It's easier for me to sing that song than to sing, 'Won't you come see me, Queen Jane.' At one time that wouldn't have been so. But now it is. Because 'Queen Jane' might be a little bit outdated. It can't be outrun. But this song is not outdated. It has to do with human emotion, which is a constant thing. There's nothing contrived in these songs. There's not one false word in any of them. They're eternal, lyrically and musically."