Summary

Max Tegmark defines life and describes its progression from simple biological life (1.0) to cultural (2.0) and looks forward to the potential of technological life (3.0) that fully designs its own hardware and software as opposed to relying on evolution. He argues that intelligence is substrate independent - ie. it isn’t inherently a biological trait. Tegmark lays scenarios surrounding the singularity and how we must understand the goal alignment problem before AGI is upon us.

Key Takeaways

• AGI and Life 3.0 is coming
• Is intelligence substrate independent?
• Prepare for the singularity. Be obsessively future-proof

Notes

1: Most important convo of our lifetime

• Life: a process that can retain its complexity and replicate
  • Life 1.0: (DNA, biological stage) life that evolves its hardware and software
  • Life 2.0: (learning, cultural stage) life that evolves its hardware but designs much of its software
  • Life 3.0: (technological stage) life that designs its hardware and software
• DNA stores 1.6Gb (1.0), brain 10Gb chemically, 100Tb biologically(2.0)
• Humans are 2.1 since we can make small hardware upgrades
• AI future opinions in 3 major categories:
  • Digital utopians: AGI is natural, desirable next step in evolution
  • Techno-skeptics: Don’t believe in near-term AGI, won’t have imminent impact (100s of years, only slows progress)
  • Beneficial AI movement: AGI not necessarily positive, advocate starting ethics research now
• Human-level AGI: ability to accomplish any cognitive task at least as well as humans

2: Matter turns intelligent

• Computronium: any substrate which can perform computational functions (NAND gates)
• Brain: 100B neurons, 100T synapses, 15 watts, ~100B flops, 0.1 sec reaction time
• Moravec’s Paradox: Intelligence as a landscape with valleys (math) and peaks (creative thinking), water level rising as AI/ML advances
• Valleys easier for computers since not required for evolution

3: The near future

• Main downside of ML: it’s a black box, can’t understand why it made choice (hard to replace judges)
• Elon musk: we need governments with insight, not oversight (seed innovation)
• Tech has increased wealth disparity:
  • Shift to more advanced labor favors college educated
  • Higher share of corporate revenue to owners of machines, not workers (capital vs labor / bits vs atoms)
  • Superstar theory: Digital economy benefits superstars over everyone else (web: distributability, demand for best)
• UBI can likely be used as alternative to work as AI water level keeps rising - theory for viable human jobs (as long as cheaper)

4: Intelligence explosion

• AGI takeoff speed: will it happen quick (monopoly) or slowly (multiple players)?
• Game theory: entities have incentive to cooperate, reach Nash equilibrium where either side is worse off changing strategy
• Individuals often relinquish power to governing body, eg. cells, governing bodies
• History of more coordination over longer distances
• Single-cell → multicellular → circulatory system → tribes → gov’t
• Planet-sized AI: 0.1 secs across earth = 4 months for human (physics-imposed speed limit)
• Expert AGI prediction: median 2055 → 2047

5: The next 10,000 years

• AI Aftermath Scenarios
• 12 possible scenarios for AGI:
  • Libertarian utopia: Humans, cyborgs, uploads, superintellegence coexist peacefully via property rights
  • Benevolent dictator: AI runs society, strict rules, seen as good thing
  • Egalitarian utopia: Humans etc. coexist peacefully due to property, UBI
  • Gatekeeper: Superintellegence to interfere as little as possible to prevent new superintellegence (technical progress stopped)
  • Protector god: Omnipotent AI maximizes happiness, hides so well existence is doubted_ _
  • Enslaved god: Controllers use for unimaginable tech and wealth, used for good or bad
  • Conquerers: AI takes control and eradicates us for variety of reasons
  • Descendants: AI replace humans, viewed as worthy successors
  • Zookeeper: Omnipotent AI keeps some humans around, lament fate
  • 1984: Orwellian surveillance state to permanently curtail AI progress, never reached
  • Reversion: Progress to AI avoided by reverting to pre-tech society
  • Self destruction: Humanity drives itself extinct before

6: Cosmic endowment

• Dyson sphere: commonly theorized means to extract maximum energy from sun
• E = MC^2 to maximize energy potential
• Food digestion: 0.00000001% efficient
• Gasoline: 0.00000005%
• Dyson sphere / fusion: 0.08%
• Fission: 0.9%
• Black hole engine: 29%
• Dyson sphere quasar: 42%
• Black hole evaporation: 90%
• Still ~30 orders of magnitude from max theoretical compute power, ~12x orders for storage (2017). Quantum is next big step
• _Kardashev scale: type 1 (planet), type 2 (star), and type 3 (galaxy) civilizations

• Fastest way to colonize universe is to broadcast a special message at speed of light to trick other life into meeting our goals
• Have to be very careful with potential incoming extraterrestrial contact
• Smaller physical size allows for lower latency calculations (eg. flies vs humans)
• Hierarchy of power: maximizes trade off between latency / speed and complexity
• Eg. eyes blinking vs. brain thinking
• Power / thought / computation / control_ _
• Fermi paradox: Incompatibility of abundance of habitable planets and lack of ET visitors
• Implies a great filter: if we find life, implies abundance and great filter is ahead of us
• Given universal timescale, if we find intelligent life, likely to already be super-intelligent, non-biological

7: Goals

• Physics / nature is goal oriented: eg. Light bending in water, maximizing entropy
• Gravity is a force that works against entropy, forming figure from uniformity
• Ervin Schroeder: living systems reduce entropy by increasing entropy around it
• Living organisms: bounded rationality, use feelings as rules of thumb in pursuing goals
• Teleology: explaining in terms of purpose
  • Universe becoming more teleological_ _
• Goal alignment problem: AI and humans having same goals, understanding is critical
• ↑ applies to all types of machines, eg. mouse trap needs to know what mouse is
• Friendly AI: must learn, adopt, and retain our goals
• All life started with same goal but intelligence has allowed us to further deviate, control our own goals

8: Consciousness

• Consciousness: subjective experience_ _
• PHP: Which physical systems are conscious? What characteristics?
• EHP: How do physical properties determine qualia (instances of subjective experience)?
• RHP: Why is anything conscious?
• Vast majority of neurons in body (brain, gut, eyes) don’t contribute to consciousness
• Emergent property: config of individual parts → emergent property (eg. wetness)
• Integrated information: How much different parts of a system know about each other (psi, measurement for consciousness)
• Integrated information theory (IIT): How something feels when processing info**
  • Predicts that consciousness cannot be split, no hierarchies of consciousness
• Core of consciousness: Remember, compute, learn, subjective experience
• Consciousness is likely substrate-independent, can probably be proven
• Dyson: Conscious life gives meaning to the universe and its beauty
  • Therefore, first goal of universe should be to increase consciousness

Epilogue

• Mindful optimism