"Bright Yellow Gun" was written and sung by Throwing Muses frontwoman Kristin Hersh. Many of her songs are expressions of an alternate personality that emerged after suffering head trauma when she was 16. Hersh later gave this person a name, Rat Girl, and realized it was part of her dissociative disorder, which was earlier diagnosed as bipolar disorder.
Unlike the alter egos songwriters often use
to express themselves in a different voice, Hersh had no control of Rat Girl, and had no idea when the songs would appear. Often, fragments of songs would come to her in the middle of the night and she would piece them together in the morning. This song alludes to this cycle: I have nothing to offer but confusion
And the circus in my head
In the middle of the bed
In the middle of the night
In a Songfacts interview with Hersh
, she explained: "Songs are dreamy, hyper-real takes on life that I don't argue with. 'Bright Yellow Gun' can say whatever it wants to... like one of my kids. I have to let them invent themselves."