Many of the lyrics were borrowed from the 1955 short story "The River" by Flannery O'Connor. In the story, a young boy is neglected by his hard-partying parents and is taken by his fundamentalist Christian babysitter to the river to be baptized by a preacher. The boy, expecting his troubles to be over after the cleansing ritual, returns home to find that nothing has changed with his parents, leading to disastrous results.
The refrain, "Throw your pain in the river, leave your pain in the river to be washed away slow," was taken from the preacher's words during the baptism:
"All the rivers come from that one River and go back to it like it was the ocean sea and if you believe, you can lay your pain in that River and get rid of it because that's the River that was made to carry sin. It's a River full of pain itself, pain itself, moving toward the Kingdom of Christ, to be washed away, slow, you people, slow as this here old red water river round my feet."
Likewise, Harvey's image of two silent birds circling was taken from the boy's observation during the ceremony as his "eyes followed drowsily the slow circles of two silent birds revolving high in the air."