Horowitz is born in London. His father, David was raised by high-school teachers who were members of the Communist Party USA and supporters of Joseph Stalin. From 1956 to 1975 his father was an outspoken adherent of the New Left before rejecting leftism completely and becoming a conservative commentator.
Ben’s mother is Horowitz’s first wife, Elissa Krauthamer. They married on June 14, 1959. Ben has four siblings: Jonathan Daniel, Anne Pilat and Sarah Rose Horowitz, who dies in 2008.