This was the first song written by Spoon after taking four years off to let each member explore other projects. Frontman Britt Daniel recalled to Q magazine: "I brought us back together. I called them to find out whether there would write some music, which I would sing on top of. They came out with an amazing song, 'Outlier' which I just loved from the beginning."
Geffen Records made history on June 27, 1994 when Aerosmith's "Head First" became the first major label song made available for exclusive digital download. Download speeds at the time were so slow it took around 75 minutes to download the track.