Life 3.0: Being Human in the Age of Artificial Intelligence

In the penultimate chapter, Tegmark discusses the nature of goals, their origin and development, and the types of goals that we should have for AI. He describes goal determination as the “thorniest” issue in AI research (249). His guiding question this time: “Should we give AI goals, and is so, whose goals?” (249). Like Chapter 6, this chapter heavily relies upon concepts from physics.

Tegmark believes that goal-oriented behavior is embedded in the laws of physics and is a fundamental part of our reality. He notes that one of the two ways we can describe the actions of nature is via a process of optimization. Connectedly, he discusses the heat death of the universe through entropy, a tendency of the world to become “maximally messy” as described through the second law of thermodynamics (251). He describes how gravity helped form the universe into recognizable places, like galaxies. When discussing the “dissipation-driven adaptation” of any group of particles, he comes to the following conclusion: “nature appears to have a built-in goal of producing self-organizing systems that are increasingly complex and lifelike, and this goal is hardwired into the very laws of physics.

He then makes the jump from physics to biology and explains the way that goal-like behavior “evolves” in the cases of living organisms. In the case of life, the goal has shifted from dissipation to replication, or procreation. He makes note of Darwinian evolution as an explanation for this. The goal of dissipation does not go away but is funneled through a subgoal, which is the replication of life and contributes to the rapidity of entropy.

Tegmark discusses “the rebellion against goals” (255). Our genes might have the goal of replication, but our brains, which are vastly more intelligent, can subvert this goal and replace it, if so inclined. Tegmark takes the view that humans are led by their feelings and that these feelings don’t lead to “a well-defined goal at all,” at least on the species level (256).

The discussion advances to engineering and the creation of AIs. Tegmark posits a central distinction between goal-oriented design and goal-oriented behavior. The things we’ve engineered thus far, like roads, bridges, and calculators, reflect goal-oriented design. Unlike evolved lifeforms, which all have the goal of replications (except human beings, psychologically speaking), designed entities can reflect, and potentially enact, all kinds of goals: “Stoves try to heat food while refrigerators try to cool food” (258). The problem when developing ever more powerful forms of machine intelligence is making sure that these machines and AI programs share our goals (259). For Tegmark, there are three “subproblems” with the goal-oriented design of AIs; they not only need to understand the goal but also adopt and retain that goal (260). Having a high-resolution model of the world is an important step toward proper understanding of the goal because the goal, as explicitly stated, is often unsatisfactory as a representation of the truth. This is part of the “value-loading problem,” which Tegmark notes is even more difficult than the “moral education of children” (262).

Tegmark articulates the structure of goal-oriented modelling in Figure 7.2, in which he develops the relationships between the ultimate goal, the retention of various goals, self-preservation, and the continuous project of building better models of the world. All of this complexity is tied to the adoption and retention of goals that are aligned with what is in the best interests of humanity. He notes how the relationship between the ultimate goal and the various subgoals, like self-preservation or world-modelling, could be deeply in tension: “With increasing intelligence may come not merely a quantitative improvement in the ability to attain the same old goals, but a qualitatively different understanding of the nature of reality that reveals the old goals to be misguided, meaningless, or even undefined” (267). The AI, through dialectical engagement with the world, could continuously update its model of reality and fundamentally change its guiding philosophy in accordance with this update. This makes the future goal-oriented behavior of such a creature impossible to predict.

Tegmark believes that technical problems about the development of properly aligned goals in AI is crucial. Even more crucial, though, is the ethical problem of establishing broad consensus for what the ultimate human goals ought to be and what we want out of AI. He discusses a few ethical principles on which he believes there is deep cross-cultural agreement: utilitarianism, diversity, autonomy, and legacy (271). He also notes the “Golden Rule,” or the ethical imperative to treat other intelligent and sentient creatures as you would hope to be treated. He thinks we can all agree to promote survival, flourishing, freedom, diversity, and the long-term future. Though he recognizes that agreement on these principles may be overly general, he writes, “let’s not let perfect be the enemy of the good: there are many examples of uncontroversial ‘kindergarten ethics’ that can and should be built into tomorrow’s technology” (274). Just because we cannot determine the most accurate solution as to what a person or AI ought to do in every case, we should still advocate for and instill general principles of right action.

Tegmark describes the possibility of a “goodness function” that could be programmed into AI. This function assigns ethical values to possible outcome and tasks the AI with achieving the greatest outcomes. For many, this form of consequence-based reasoning might be extremely frightening. (Note the criticisms of effective altruism and “longtermism” discussed in the background section above.) Some of the quantitatively best outcomes, in Tegmark’s view, include the highest potential amount of living matter, the greatest “computational capacity of our Universe” (278-79), and the sheer amount of conscious life.

To determine what we should do and how we should govern and direct the future of technology, we need to become clear on what we want and the most ethical course of action to achieve it. For Tegmark, this means that we need to strive, with renewed diligence, toward answers to timeless ethical/philosophical questions: “To program a friendly AI, we need to capture the meaning of life” (279).