The Tarpeian Rock was a steep cliff about 80 feet tall off the southern summit of the Capitoline Hill, overlooking the Roman Forum in Ancient Rome. It was named after Tarpeia, the faithless daughter of the governor of the citadel, who who betrayed the city of Rome to the Sabines in exchange for what she thought would be a reward of jewelry. Instead she was flung from the cliff by the Sabines. It became a site used during the Roman Republic as an execution site for traitors, who were hurled from the rock to their deaths.
This song was inspired by the site. Vocalist Joe Casey explained to NME: "I was reading a book about Roman history, and how they would chuck people off the rock. And I saw a picture of the place nowadays and think it's funny that it's just like a parking lot now, and it used to be where they executed all of these people all of those years ago. And an easy way to write a song is to just rant. I can do that pretty easily with a list of gripes."
"The song started serious I think, and I was being serious with my gripes," he continued. "But then with singing it, it comes off more funny I think. So I started putting in more day-to-day things that disillusion you. Like a display for ants in my bath tub - every summer I get ants which is a gripe. So I'd try to throw in like real things that bother me."