This song is about a guy who doesn't really know what he wants. On one hand, he longs to be with his girlfriend: "Don't be gone when I get home/I need you there." Pessimism soon comes into play, however, and he starts to feel like their relationship isn't working after all: "It's like you're falling in love/While I just fall apart."
Matt Pryor told 17 Dots that The Get Up Kids like to close their gigs with this song: "We close with it a lot. It's a driving song, it's a party song - everyone goes nuts. It was kinda like, 'Do you want to go out with a ballad or do you want to go out with a party?' And it was 'Let's go out with a party.' So that's why. It's a fun song. There ended up being, like, 45 people on stage by the time the song was over."
In 1999, this song was released on vinyl as part of Sub Pop Records' Singles Club. 1300 pressings were made on black vinyl, while a further batch of 100 were printed on clear vinyl.
"The Night Chicago Died" was written and recorded by the British group Paper Lace. They talk about Al Capone in the song, but got a lot of details wrong - understandable since they wrote it based on gangster movies.