Beck's father, orchestrator David Campbell, helped with the brass and string arrangements on this song and the rest of the album. Beck told NPR how they worked together. "I usually get some kind of keyboard and I'll play the basic voicings and we'll map it out," he said. "With strings, a lot of times I'll play or sing parts over the song, which we'll transcribe."
He added: "For 'Paper Tiger' there's a whole solo that the orchestra does; that was actually something I sang, and then we just transcribed it."
A paper tiger is a person or thing that seems strong or threatening, but is, in fact, feeble or ineffective. The expression arose from a Chinese saying applied by Chairman Mao in the 1950s to the USA. Beck sings of the paper tiger protagonist in this song as being, "torn apart by idle hands."
Kacey Musgraves originally offered "Follow Your Arrow" to her friend Katy Perry. However, Perry thought Musgraves should record the song herself as it seemed, "like something that you would totally say."