The song contains multiple literary references. Here are a few:
Rue Morgue Avenue hearkens to Edgar Allen Poe.
Tom Thumb was a character from English folklore who would, in 1621, star in the first fairy tale ever published in English.
Housing Project Hill comes from the Jack Kerouac novel Desolation Angels. Dylan would later say that he came to find the lifestyle espoused in Kerouac's books to be pointless, but his first impression was very different. "I read On the Road in maybe 1959," Dylan has stated. "It changed my life like it changed everyone else's."
Among the archives in the Bob Dylan collection housed in Tulsa, Oklahoma is an early lyric sheet to this song, which reveals that at one point it was titled "Just Like Juarez."