The title of this bass-heavy song references a small neighborhood in New York's most distant borough, Staten Island. It features rapped vocals from Ezra Koening. SBTRKT told The Guardian how he convinced the Vampire Weekend frontman to rap over the track. "We talked about it beforehand, and it made sense because Ezra had these great lyrics and I was intrigued by the fact he had this hip-hop past," he explained. "It's a quintessential New York song, it feels like it has ownership of a place and time, and that was what my album was about for me."
Kacey Musgraves originally offered "Follow Your Arrow" to her friend Katy Perry. However, Perry thought Musgraves should record the song herself as it seemed, "like something that you would totally say."