Max Tegmark2017
Max Tegmark asks what happens when life can design both its hardware and its software. Unlike biological evolution or human learning, Life 3.0 could redesign the machinery that makes intelligence possible. Because intelligence is the ability to accomplish complex goals, greater capability alone does not tell us whose interests it will serve. Automation could expand prosperity while concentrating ownership and leaving people without income or purpose; an intelligence explosion could also reshape who holds power. Tegmark therefore argues that beneficial AI requires deliberate choices about safety, institutions, and the future people want, rather than confidence that progress will produce a good outcome on its own.
Those choices depend on solving the goal alignment problem: machines must learn, adopt, and retain human goals even as their capabilities change. But agreement about those goals is itself unresolved, and the range of possible futures extends from peaceful coexistence to human disempowerment or extinction. Tegmark’s discussion of consciousness deepens the question by asking whether machines could have subjective experience and what would make a technologically powerful future worth living in. He argues for research and public discussion now, linking safety and human control to shared prosperity rather than treating technical capability as the ultimate aim.
“We could end up in an economy where most of our economy is owned and controlled by machines”
”It’s not that were creating more jobs, just that we’re crowding into the archipelagoes of intellectual capability”
“If consciousness is the way that information feels when it’s processed in certain ways, then it must be substrate independent.”