Notes

Introduction

• IJ Good intelligence explosion quote*
  • RSI
• Omega / Prometheus hypothetical
  • 1.0 → 10.0 in days
  • Mechanical Turk AWS arbitrage saturation
  • Started with content (movies / TV)
  • UBI-like charity / redistribution
  • → news, politics, persuasion
  • Education + persuasion (depolarization)
  • Agenda 7 slogans: democracy, tax cuts, gov’t social service cuts, military spending cuts, free trade, open borders, socially responsible companies
    • Erode all power structures in the world
  • Outsmarting business tycoons (index funds dominant)
  • Chaos / future shock, but still world getting dramatically better
  • “The Alliance” world order providing social services etc. better than gov’ts - disempowered / toppled

1: Most important convo of our lifetime

• 13.8B years - our universe is awakening (human consciousness)
• Life: a information processing machine that can retain its complexity and replicate
  • Life 1.0: (DNA, biological stage) life that evolves its hardware and software
  • Life 2.0: (learning, cultural stage) life that evolves its hardware but designs much of its software
  • Life 3.0: (technological stage) life that designs its hardware and software
• Humans are life 2.1 since we can make small hardware upgrades
• —
• Larry Page: “the most influential exponent of digital Utopianism”
  • “Larry accused Elon of being speciesist”
• Andrew Ng: techno skeptic - won’t happen in our lifetimes
• Stuart Russel, Eliezer, Nick Bostrom
• FoLI + conference → “beneficial AI”
• —
• DNA stores 1.6Gb (1.0), brain 10Gb chemically, 100Tb biologically(2.0)
• AI future opinions in 3 major categories:
  • Digital utopians: AGI is natural, desirable next step in evolution
  • Techno-skeptics: Don’t believe in near-term AGI, won’t have imminent impact (100s of years, only slows progress)
  • Beneficial AI movement: AGI not necessarily positive, advocate starting ethics research now
• Human-level AGI: ability to accomplish any cognitive task at least as well as humans

2: Matter turns intelligent

• Computronium: any substrate which can perform computational functions (NAND gates)
• Brain: 100B neurons, 100T synapses, 15 watts, ~100B flops, 0.1 sec reaction time
• Moravec’s Paradox: Intelligence as a landscape with valleys (math) and peaks (creative thinking), water level rising as AI/ML advances
• Valleys easier for computers since not required for evolution

3: The near future

• Main downside of ML: it’s a black box, can’t understand why it made choice (hard to replace judges)
• Elon musk: we need governments with insight, not oversight (seed innovation)
• Tech has increased wealth disparity:
  • Shift to more advanced labor favors college educated
  • Higher share of corporate revenue to owners of machines, not workers (capital vs labor / bits vs atoms)
  • Superstar theory: Digital economy benefits superstars over everyone else (web: distributability, demand for best)
• UBI can likely be used as alternative to work as AI water level keeps rising - theory for viable human jobs (as long as cheaper)

4: Intelligence explosion

• AGI takeoff speed: will it happen quick (monopoly) or slowly (multiple players)?
• Game theory: entities have incentive to cooperate, reach Nash equilibrium where either side is worse off changing strategy
• Individuals often relinquish power to governing body, eg. cells, governing bodies
• History of more coordination over longer distances
• Single-cell → multicellular → circulatory system → tribes → gov’t
• Planet-sized AI: 0.1 secs across earth = 4 months for human (physics-imposed speed limit)
• Expert AGI prediction: median 2055 → 2047

5: The next 10,000 years

• AI Aftermath Scenarios
• 12 possible scenarios for AGI:
  • Libertarian utopia: Humans, cyborgs, uploads, superintellegence coexist peacefully via property rights
  • Benevolent dictator: AI runs society, strict rules, seen as good thing
  • Egalitarian utopia: Humans etc. coexist peacefully due to property, UBI
  • Gatekeeper: Superintellegence to interfere as little as possible to prevent new superintellegence (technical progress stopped)
  • Protector god: Omnipotent AI maximizes happiness, hides so well existence is doubted
  • Enslaved god: Controllers use for unimaginable tech and wealth, used for good or bad
  • Conquerers: AI takes control and eradicates us for variety of reasons
  • Descendants: AI replace humans, viewed as worthy successors
  • Zookeeper: Omnipotent AI keeps some humans around, lament fate
  • 1984: Orwellian surveillance state to permanently curtail AI progress, never reached
  • Reversion: Progress to AI avoided by reverting to pre-tech society
  • Self destruction: Humanity drives itself extinct before

6: Cosmic endowment

• Dyson sphere: commonly theorized means to extract maximum energy from sun
• E = MC^2 to maximize energy potential
• Food digestion: 0.00000001% efficient
• Gasoline: 0.00000005%
• Dyson sphere / fusion: 0.08%
• Fission: 0.9%
• Black hole engine: 29%
• Dyson sphere quasar: 42%
• Black hole evaporation: 90%
• Still ~30 orders of magnitude from max theoretical compute power, ~12x orders for storage (2017). Quantum is next big step
• Kardashev scale: type 1 (planet), type 2 (star), and type 3 (galaxy) civilizations
  * Fastest way to colonize universe is to broadcast a special message at speed of light to trick other life into meeting our goals
  * Have to be very careful with potential incoming extraterrestrial contact
  * Smaller physical size allows for lower latency calculations (eg. flies vs humans)
  * Hierarchy of power: maximizes trade off between latency / speed and complexity
  * Eg. eyes blinking vs. brain thinking
  * Power / thought / computation / control
  _
  * Fermi paradox: Incompatibility of abundance of habitable planets and lack of ET visitors
  * Implies a great filter: if we find life, implies abundance and great filter is ahead of us
  * Given universal timescale, if we find intelligent life, likely to already be super-intelligent, non-biological

7: Goals

• Physics / nature is goal oriented: eg. Light bending in water, maximizing entropy
• Gravity is a force that works against entropy, forming figure from uniformity
• Ervin Schroeder: living systems reduce entropy by increasing entropy around it
• Living organisms: bounded rationality, use feelings as rules of thumb in pursuing goals
• Teleology: explaining in terms of purpose
  • Universe becoming more teleological
• Goal alignment problem: AI and humans having same goals, understanding is critical
• ↑ applies to all types of machines, eg. mouse trap needs to know what mouse is
• Friendly AI: must learn, adopt, and retain our goals
• All life started with same goal but intelligence has allowed us to further deviate, control our own goals

8: Consciousness

• Consciousness: subjective experience
• PHP: Which physical systems are conscious? What characteristics?
• EHP: How do physical properties determine qualia (instances of subjective experience)?
• RHP: Why is anything conscious?
• Vast majority of neurons in body (brain, gut, eyes) don’t contribute to consciousness
• Emergent property: config of individual parts → emergent property (eg. wetness)
• Integrated information: How much different parts of a system know about each other (psi, measurement for consciousness)
• Integrated information theory (IIT): How something feels when processing info**
  • Predicts that consciousness cannot be split, no hierarchies of consciousness
• Core of consciousness: Remember, compute, learn, subjective experience
• Consciousness is likely substrate-independent, can probably be proven
• Dyson: Conscious life gives meaning to the universe and its beauty
  • Therefore, first goal of universe should be to increase consciousness

Epilogue

• Mindful optimism