Human Compatible (Stuart Russell)

Summary: Stuart Russell lays out the AI Alignment problem succinctly in a way that is interpretible to a wider audience than most previous publications. Russell presents the problem as being inherent in what he calls the "standard model": AI systems that operate via fixed objective functions and are certain of those objections. Russell argues that the only solution is to throw away the standard model for general AI (though it will still be helpful for chess playing programs and the like) and start with a new model. He introduces three principals: AI should act to satisfy human preferences, it should be uncertain about those preferences at first, and the ultimate source of information about human preferences is human behavior. Russell also discusses psychology, game theory, and philosophy, to discuss multi-agent environments and what "human preferences" means in the first place.

Opinion: I thought this was a great introduction to Russell's particular brand of AI alignment thinking. In contrast to Superintelligence, which was often cited as the go-to book on the subject, I much prefer Human Compatible for presenting to people unfamiliar with AI alignment. The main reason for this is because I think it is broader: it presents more threat models, discusses more complications, and broaches the important idea that the standard model can feasibly be thrown out.

Sapiens (Yuval Noah Harari)

Summary: Harari lays out a broad history of humanity, focusing on the cognitive revolution, the agricultural revolution, the "unification of humankind" and the scientific revolution. Harari focuses on major developments: the formation of empires, agriculture, science, and social structure. Throughout the book, he emphasizes the idea of myths: corporations, "religions" (a category which for him includes ideologies like liberalism), nations. Myths, he claims, are absolutely necessary for the organization of humanity, and have allowed for humanity to grow to where it is today. Harari consciously asks where history should be going. Should we aim for "happiness" and if so, what is it and how have historical processes shaped it?

Opinion: I really liked Harari's focus on well-being, something which I have not seen a lot of in historical narratives. He goes beyond the obvious and suggests that the agricultural revolution likely made people worse off, at least for thousands of years. Rather than attempting to attribute one cause to suffering (e.g., freedom was taken away, so people suffered), Harari tries to develop an understanding of happiness. I also thought it was valuable for a historian to bring up animal suffering, something which often goes undiscussed.