The idea for the song came to Beth when she was looking at the poetry shelves in a San Francisco bookshop. She recalled to Pitchfork
"I found this book called Crime Against Nature
by a poet called Minnie Bruce Pratt. It's about her specific story about leaving her family and abandoning her children for a woman. It's really touching. The idea of doing something big for love and the guilt that you have to deal with afterwards was really interesting for me. How much love comes with its opposite: fear, jealousy, anxiety, abandon.
And the story touched me because what she realized is that she was a poet. She had a very conventional life - a husband and two kids - and was bored to death. But then she overcame that and realized there was something else in life that could let her be the person she was supposed to become. And I think that's what you [make art] for, so you can pass that on."