This song is one of Peter Buck's favorite R.E.M. songs to listen to and play, he noted in the liner notes for the compilation Part Lies, Part Heart, Part Truth, Part Garbage. "It set a tough beginning for Lifes Rich Pageant, and I remember it was one of the first things we finished for the record. I recall bringing it in and although it was kind of a complicated song, it paves the way for the fact that Pageant is going a little more focused and loud. Lyrically it points to what we were doing on the record and it pushed us in that direction."
Frontman Michael Stipe added: "I always liked opening a song with 'a birdie and a hand.' Two fantastically honest gestures, the f--k off!, and the rampant applause."
"Begin the Begin" is also the title of a season two episode of Grey's Anatomy.