This song was originally written and recorded by Suicide under the title of "Keep Your Dreams," for their 1977 debut EP. The electronic protopunk duo remade the dark, electronic tune two years later under the title of "Keep Your Dreams," and released as a non album single. It found a fan in Springsteen, who transformed it into a warmer and more optimistic version, which he performed a solo keyboard version to close the concerts on his 2005 Devils & Dust tour.
The track later resurfaced as the soundtrack to a video released in October 2013 as a thank you to Springsteen's fans for supporting the 2012 and 2013 Wrecking Ball tour and then was included on his 2014 High Hopes album.
Suicide have been recording intermittently active since 1970. A flyer the band circulated in November of that year was the first ever to use the phrase "punk music" to advertise a concert. Alan Vega's vocals were backed by a primitive drum machine and Martin Rev's fuzz organ and their nihilistic attitude and lack of guitars often aroused bewildered reactions amongst their audiences. Despite their lack of popularity amongst the general public, Suicide's albums of the late '70s and early '80s are regarded as highly influential by music critics, having helped shape the direction of indie rock, industrial and dance music.
Alan Vega commented of Springsteen's cover in an 2005 interview with Backstreets Magazine. "A lot of bands have done my stuff, Suicide stuff, and they basically try and copy and do it the way that you do it," he said. "He just - thank god! - finally somebody did their version of it. They interpreted my song, he did it his way, and such a great way, that I'm going to have to sing it that way, or not sing it at all any more!"
High Hopes producer Ron Aniello admitted to Rolling Stone that he didn't know this was such a well-known song. He recalled: "(Bruce) played me the thing and said it was an experience more than anything at the end of his Devils and Dust shows. It took a lot to get that experience on a record for him. We had to do that song probably 10 times – different versions of it – until he was satisfied it was the right version."
Aniello told Rolling Stone about the recording process for High Hopes at Springsteen's home studio. "Each day he'd give me a list of things and I'd write them down," the producer said. "When he starts talking, I start writing. Then I try to distil what his advice is about the song to make it work for him. We play the song again and get it to a place."
"Take the song 'Dream Baby Dream," he added. "I had some versions of the song and he was like, 'That sounds good, but it's not quite there,' or 'It's good. You're really close.' Then the next day he'd be like, 'Ah, it's terrible. Throw it out.' I was like, 'Come on, love this song. I gotta deliver this song for you.' We went over and and over it. You never know what he's thinking. It's not like when he was 21 and he'd go 16 hours in the studio without eating. He doesn't have to do that now. He has a home studio and it's quite comfortable for him."