In the "You're So Vain
" vein, this song was rumored to be about James Taylor, who was good friends with King and played on the Tapestry
. In early 1971, Taylor and King toured together with King the opening act. Many people tended to think that this song was about a short romance between the two. King never confirmed these rumors, and Taylor later dated and married Carly Simon.
Toni Stern, however, did have a relationship with Taylor and wrote the lyrics after they broke up and he moved on to Joni Mitchell, but she won't confirm it's about the troubadour. "I won't say who 'It's Too Late' is about," she told Sheila Weller, author of Girls Like Us
. "I don't kiss and tell."