Simon wrote this song with Joseph Shabalala, lead singer of the South African vocal group Ladysmith Black Mambazo (the group's name comes from the town of Ladysmith, South Africa, situated near the Indian Ocean coast in South Africa). Simon had seen the group on a BBC documentary called Rhythm of Resistance: The Music of South Africa, and wanted to record with them. When Simon traveled to South Africa in 1985, he met with Shabalala, who gave him some of the group's albums, which Simon listened to every night. In the documentary Under African Skies, Simon explained: "I was bewitched by Ladysmith Black Mambazo because they were so beautiful. The music was enchanting - it was all a cappella, and it was so beautiful that I was intimidated. They were so good at what they did and it was so contained that I didn't know at the time how I could possibly fit into their world, and if they wanted me to fit into their world."
It was decided that Simon would write a song to work on with the group, and they would record it somewhere outside of South Africa, since Simon didn't feel comfortable going back for political reasons. When he returned to America, Simon wrote "Homeless" and put a demo of the song on a tape, which he sent to the group, letting them know they could change it any way they wanted. The demo cassette was Simon on piano singing only the line, "We are homeless, homeless, moonlight sleeping on the midnight lake." Shabalala continued the story in Zulu to complete the songwriting process.
Regarding the meaning of the song, Joseph Shabalala said, "We're far away from home and we're sleeping. Our fists are our pillows."
The bridge of "somebody say..." is based on an already-existing Ladysmith song, one of Simon's favorites.
The introduction is based on a traditional Zulu wedding song, with re-written lyrics to continue the idea of homelessness.
Simon and Ladysmith Black Mambazo recorded this song in two days at Abbey Road studios in London, England. Simon spent almost two years working on the 11 tracks for Graceland, but Ladysmith was used to doing that many songs in a single day. That first day of recording did not go well, and the group couldn't get the sound Simon wanted. They were disappointed, but determined; that night the group went to the hotel, had dinner and practiced the song until midnight. The next day they played Simon what they had practiced, and Simon know right where to come in with the word "Homeless." That day, they got it in two takes. The experience brought Simon and the group together on the same musical wavelength, and they had many more successful collaborations: Ladysmith also sang on "Diamonds on the Soles of Her Shoes" and backed Simon on the Graceland world tour. Simon produced the group's 1987 album Shaka Zulu, which was their first release in America.