Sinatra sang this with a scowl in his voice that was out of character, but exactly what the song needed. A story circulated that producer Jimmy Bowen told Sinatra, after the recording session, to get out of his car and back in the studio to re-record the vocal, which made Frank very angry and resulted in his edgy vocal. The song's co-writer, Dean Kay, told Songfacts the real story.
"I'm the writer of 'That's Life' and was sitting five feet away from Frank Sinatra and producer Jimmy Bowen when they listened to the playback of the first take," he said. "It was then and there that Bowen asked Sinatra to take a second pass at the song. It's a common myth that Bowen followed Sinatra to his car and made him come back into the studio for another take. It is true that Sinatra, famous for doing almost everything he did in one take, was not happy to do it again. And, it is true that his displeasure is manifested in the extra bite in his performance, which is exactly what Bowen was looking for. The 'My, My' ending was directed at Bowen in a 'how do you like that, Charlie' sort of way. The first take ended with, 'Oh, yeah.'
'My, My' is the catch phrase that has been associated with the song - and Sinatra - from that night forward. Frank Sinatra recorded my song when I was 26. I have been, and will always be, grateful for his magnificent recording that changed my life forever. (more at deankay.com