The song title is both the name of a speech given by Dwight D. Eisenhower in 1953 and the motto of the International Atomic Energy Agency.
The song opens with the lines "No more going to the dark side with your flying saucer eyes. No more falling down a wormhole that I have to pull you out." According to The Observer newspaper, they refer to "something of an admonition" from Yorke's wife Rachel to him to lighten up.
In 2009 Yorke gathered a band of musician friends together to see if The Eraser could be played live. They toured briefly, initially billed warily as "??????", before being named "Atoms for Peace" after this song.