The fourth and final single from Tori Amos' fifth album, this track is named for the concertina, a squeezebox-type instrument with expanding and contracting bellows with keys or buttons on the ends. Amos uses it to describe how she feels when social anxiety hits. She told Alternative Press in 1999:
"Do you ever feel like you walk in a room, and you don't know why, but you're just so uncomfortable you're crawling out of your skin, even though nobody's touched you, physically? That's in 'Concertina,' when you feel like you haven't excavated enough of your different personalities that when one pops up, you're not sure where it came from, and you try to hack it out of yourself. It shocks you that you could have this kind of fault, or that other people could bring it out in you."
Amos told Australia's Time-Off how she let the song, which features contrasting acoustic piano and electronic drums, dictate how it wanted to sound: "You always have to be listening to the song itself and to the soul of the song. Because sometimes there were different directions I could have taken the songs into and it's not where the song itself wanted to go. It's funny, during 'Concertina,' the band all looked at me and said, 'Oh, just do it like you played it this morning on the piano.' But I cut it to a loopy click track and said, 'Get in there and pick up your instruments and we'll find it.' I wanted those electronic drums that Matt [Chamberlain] was playing with because particle by particle, she slowly changes, and I wanted the sense of the acoustic piano with the electronic drums. That also re-occurs in 'Lust.' So there was this dichotomy going on and I'm really drawn to that."
Thanks to its second disc, To Venus And Back is notable for producing the first live album of Amos' career. Along with a disc of original studio tracks (including "Concertina"), the double album also features a disc of live material from her Plugged '98 tour.