Downtown Church consists almost entirely of religious and gospel songs. However this cover of a Big Mama Thornton R&B track cataloguing the excesses of an unfaithful, hard-drinking man is the one exception. This was the song producer Buddy Miller played through the PA regularly so as to center Griffin's backing group. She explained with a chuckle why she decided to cover this very secular song on an album contained sacred numbers: "The band was so greasy by the end of that week. I wanted to do it, just to have it. I didn't really think we would get it on the record, and Buddy said, 'Oh, I'll figure out a way.' And he did. I think he also had in mind tipping the hat to skepticism. Not a bad thing to include."
Thornton is best known for cutting Lieber & Stoller's "Hound Dog" before Elvis - it seems she liked animal metaphors.
The song was featured in an episode of True Blood called "I Smell A Rat," which aired in 2010.
Movie director Michel Gondry played live drums on the Late Registration track, "Diamonds From Sierra Leone." The Eternal Sunshine of The Spotless Mind filmmaker happened to be in the studio on a day when producer Jon Brion was setting up a drum kit