The refrain of this song is adapted from a nursery rhyme called "Ladybug! Ladybug!": "Ladybug! Ladybug! Fly away home Your house is on fire And your children all gone All except one, And that's Ann, For she has crept under The frying pan."
The chorus of "Jockey Full of Bourbon" strongly connects to the album's core themes of wandering and being lost with the line "Your house is on fire."
"Jockey" is American slang for a cab or bus driver - in this case one that is inebriated.
In the first verse, "Dutch Pink" is slang for blood.
Tom Waits starred in a 1986 movie called Down By Law in which he plays a disc jockey who ends up in jail with Roberto Begnini of Life Is Beautiful fame. This song opens the movie.