"Pitch the Baby" is about Cocteau Twins singer Elizabeth Fraser's pregnancy and childbirth. The song, as with most of Heaven Or Las Vegas, marked a departure in Fraser's approach to writing lyrics. Previously, she'd used nonsense words and words from languages she couldn't speak or understand. With "Pitch The Baby," she'd begun to sing about more concrete subjects.
The birth of Lucy Belle Guthrie was significant not only for the obvious reasons, but also because both Fraser and father/fellow Cocteau Twin Robin Guthrie hoped the child would anchor Guthrie to the real world and help him get over his drug and alcohol addictions.
Kacey Musgraves originally offered "Follow Your Arrow" to her friend Katy Perry. However, Perry thought Musgraves should record the song herself as it seemed, "like something that you would totally say."