This song was inspired by a photo of a friend's cat named Goose which Aimee Mann received while on tour in Ireland. The singer explained to Billboard magazine: "My friends post a lot of pictures of their cat, Goose, on Instagram, and she's got white fur, and she really looks like a snow cone ball. So I start writing this song, and it's really more about being homesick and lonely than it is about the cute little kitty. It's very dumb that I kept the name, but once I wrote the song, I was stuck with it."
Mann used the actual Goose in the video, where we see her in some mortal jeopardy. The songstress recalled: "Her owners moved out here, so I put Goose in the video. Goose is a good cat. She's very photogenic. She wasn't really cooperative. I was amazed what they did with editing where they got shots later that made it look like she was cooperating."
Phil Oakey recorded his vocals for "Don't You Want Me" in the studio bathroom. The recording was disrupted by guitarist Jo Callis reaching through an open window from outside to repeatedly flush one of the toilets.