Angaleena Presley was in the middle of a divorce, living in the ghetto, sleeping next to her three-year-old son when she was invited to join the Pistol Annies. She recalled: "One night, at like 2 a.m., I got a call from my friend, Ashley Monroe, and Miranda Lambert. They had been listening to my record – and probably drinking a lot – and asked me if I wanted to start a band with them. I almost hung up 'cause I thought it was a prank call, but I went ahead and told them 'yes,' and I'm glad I did."
This song was inspired by Angaleena leaving her former husband and the effect on her infant son. She told Billboard magazine: "Every word of it is true. My husband was awful, and it was such an awful situation. But to leave that situation, I had to break my baby's heart, and that broke my heart even worse than my husband broke my heart. I can see why people stay in bad marriages, because I can't stand to see him goin' through that."
Mike Campbell from Tom Petty & the Heartbreakers played the slide guitar on "Sixth Avenue Heartache." There is a connection here - Wallflowers lead singer Jakob's dad, Bob Dylan, played with Tom Petty in The Traveling Wilburys.