Summary

Ray Kurzweil explores the exponential growth of intelligence through AI and neuroscience. Compatibilism is the compatibility of determinism and free will. Three major AI risks: misuse, outer misalignment, and inner misalignment. Our top priority should be to improve human governance and social institutions to keep pace with technological advancements and maintain ethical standards.

“One of the great challenges of adapting to technological changes is that they tend to bring diffuse benefits to a large population but concentrated harms to a small group”

“Ultimately, the most important approach we can take to keep AI safe is to protect and improve on our human governance and social institutions”

“The best way to avoid destructive conflict in the future is to continue the advance of our ethical ideals”

Key Takeaways

• Exponential growth of intelligence evidenced by AI/ML + neuro
• Consciousness involves awareness and subjective experiences

• AI → universal basic income (UBI) by 2030s
  • De-skilling → up-skilling → non-skilling
  • “Our main struggle will be for purpose and meaning. In effect, we’re moving up Maslow’s hierarchy of needs”
• Consumer surplus: monetary replacement rate for G+S (eg. Facebook)
• 3 AI risks:
  1. Misuse
  2. Outer misalignment: mismatch with programmer’s actual intentions
  3. Inner misalignment: mismatch between learned goals and behavior
• Asimolar AI principles
• Priorities: improve human governance, social institutions, advance ethical ideals

Notes

1: Where are we in the 6 stages?

• Information is basis of consciousness
• 6 epochs:
  • Physics
  • Life
  • Brains
  • Digital computation
  • BCIs
  • Computronium

2: Reinventing intelligence

• History of AI
• Mechanistic interpretability
• ML architecture & training
• Unconscious competence:
• Fixed action patterns: hardwired in species
• Neocortex: rise of and architecture
• 3 key features of human neocortex:
  • Can propagate neuron firing pattern broadly across its structure
  • Given firing pattern can be attributed to many related concepts
  • Millions of firing patterns can occur simultaneously and interact
• Law of accelerating returns:
• Transformers, zero-shot learning
• Big data + training data
• Intelligence explosion
• Neurons fire 0.29-1 times / sec
• Brain simulation: 10^13 - 10^25 FLOPS

3: Who am I?

• Consciousness: two related concepts:
  • Aware of surroundings, internal thoughts in relation to external world
  • Qualia: the ability to have subjective experiences inside a mind
• Hard problem of consciousness:
• Panprotopsychism:
  • Middle ground of dualism, materialism
• Cellular automata: class 4 rules
• Emergence:
• “If reality is based upon principles like class 4 rules at its deepest level then we could explain seeming randomness at the quantum scale in deterministic terms but there would be no summarizing algorithm that could look ahead and predict the exact state of the universe at some time in the future”
• Compatibilism: determinism and free will ↑
• Society of mind: brain has many independent units
• The ship of Theseus: continuity of information patterns
• Anthropic principle: must be living to contemplate fine-tuned nature of universe
• Replicants
• Moravec’s paradox
• Whole-brain emulation: 11 levels, need to determine which level is sufficient

4: Life is getting exponentially better

• LOAR for everything: only applies for systems with positive feedback loops
• Evolutionarily attraction to bad news
• Optimism is a self-fulfilling prophecy
• How the world is getting better*
• Medicine → information technology
• UN’s millennium development goals
• Rapid increase in choice to meet preferences not reflected in GDP
• Real income / hour
• Decline in hours worked
• Expanding circle of empathy
• Less zero-sum thinking (eg. internet goods)
• Key reinforcing loop: technologies that make it easier to share ideas
• Digital technology is deflationary
• Renewable energy
• Decentralized manufacturing
• 3D printed buildings
• Longevity escape velocity by 2030s

5: The future of jobs

• Waymo’s simulated driving data
• Transportation directly employs ~10% of US workers
• Containerization → globalization, deindustrialization
• More investment into education*
• Why is this time different? culmination of successive waves of change:
  • 1. de-skilling: less skill needed for labor
  • 2. up-skilling: technologies that require more skill
  • 3. non-skilling: AI > humans
• Slowing productivity gains since 2004: great economic mystery
• Consumer surplus: $$ replacement rate
  • Eg. FB $8B revenue vs $280B surplus
• Dropping labor force participation rate
• ↑ 2 confounding factors:
  • Teens not working, more education
  • Boomers aging out
• ↑ controlling for 25-54, basically flat
• Fundamental issues:
• 1. employment is a means to an end
• 2. purpose and meaning: contributing
  • Creativity: more opportunity, artistic possibilities than ever, eg. TV shows
• US: 18.7% spending on social welfare 2019
• Prediction: UBI by 2030s
• Disproportionate advantages, eg. compute costs vs healthcare cost → young, elderly
• “Our main struggle will be for purpose and meaning. In effect, we’re moving up Maslow’s hierarchy of needs”
• “One of the great challenges of adapting to technological changes is that they tend to bring diffuse benefits to a large population but concentrated harms to a small group”

6: Next 30 yrs in health + wellbeing

• Biotech + AI + big data
• Personalized medicine
• Medical simulation
• “Humans can only be second class robots”
• Nanotechnology, atomic, graphene
• Self-replicating machines
• Value moving from physical → info
  • Eg. e-books
• Won’t eliminate scarcity in short term due to cultural values of “original copies”
• Different types of aging for before 110, after
  • +2% death/yr → +3.5%
• Nano-biological augmentation
• Unfathomable nanotech by 2040s-50s

7: Peril

• Nuclear weapons
• Biotechnology
• Nano-based weapons, gray goo
• AI risks:
  • 1. misuse
  • 2. outer misalignment: mismatch with programmer’s actual intentions
  • 3. inner misalignment: mismatch between learned goals and behavior
• Asimolar AI principles: 23 total
  • Non-competitive AI development
• “Ultimately, the most important approach we can take to keep AI safe is to protect and improve on our human governance and social institutions”
• “The best way to avoid destructive conflict in the future is to continue the advance of our ethical ideals”
• Need to take Luddite voices seriously

8: Dialogue with Cassandra

• Solution for meeting, purpose in life: extending the neocortex into the cloud