The "sweet by and by" is heaven, a "land that is fairer than day."
The song was first released as sheet music, but has gone on to become a gospel standard and one of the most recognizable Christian hymns of the modern age.
S. Fillmore Bennett wrote the lyrics and Joseph P. Webster wrote the music.
Bennett started the song after Webster walked into his business in Elkhorn, Wisconsin. The two were friends and Bennett was accustomed to Webster's persistent melancholy, so much so that he could tell at once that Webster was feeling down. When Bennett asked what was wrong, Webster replied, "It's no matter, it will be all right by and by." The idea for the song immediately hit Bennett "like a flash of sunlight."
Bennett scribbled down the lyrics and Webster starting fleshing out the song on his violin. Within half an hour one of American history's most enduring songs was complete.
New Orleans has a tradition called a "jazz funeral," in which musicians play in a funeral musician for deceased peers. "In The Sweet By And By" is one of the standard songs played in the jazz funerals.