The song title originates from a repeated remark made by James Johnston. Vocalist Simon Neil told This Is Fake DIY: "Pause It And Turn It Up' was when we were listening to some music and James was blitzed, and kept going 'Aw man, quit and pause it and just turn it up!' He meant to say 'rewind it' but just the way it came out was quite funny."
Though the song is listed as the final song on Infinity Land, approximately 18 minutes of silence follows before Simon Neil reads a short poem titled "Tradition Feed."
"(What's So Funny 'Bout) Peace, Love, and Understanding" was written by Nick Lowe in 1974. The original version with his group Brinsley Schwarz was kind of somber, but Elvis Costello made it a classic with his 1978 uptempo take.
The bedrock of David Guetta's Nicki Minaj-featuring single "Hey Mama" is a sample of "Rosie," a 1940s prison recording from folk archivist Alan Lomax that songwriter Esther Dean first showed the French DJ on YouTube.