Summary

Information begins with the bit and becomes a bridge from mathematics to EE to CS. Each medium changes thought: written language enabled philosophy, logic, and mathematics; the telegraph divorced the message from its medium; information theory defined information as uncertainty, surprise, difficulty, and entropy.

The informational turn carried these ideas into psychology, cognitive science, genes, memes, thermodynamics, and quantum information theory. Information links randomness with complexity and becomes physical, while exponential information creates information anxiety and a widening gap between information and knowledge.

“Every new medium transforms the nature of human thought”

“In the long run, history is the story of information becoming aware of itself”

“The universe is computing its own destiny”

Key Takeaways

Notes

Prologue

• Bit = unit of information
  • Like KG, $
• “Information” adopted
• Information theory began as a bridge from mathematics to EE to CS
• “The whole universe is thus seen as a computer - a cosmic information processing machine”
• “Every new medium transforms the nature of human thought”
• “In the long run, history is the story of information becoming aware of itself”

1: Language

• Morse code
• Phoneme = unit of language
• Natural redundancy in language

2: Early written language

• Secondary orality (The Gutenberg Parenthesis)
• Medium → how we structure thoughts
• History and logic are the product of literate thought
• Pictographic → ideographic → logographic
• Written language enabled philosophy, logic, mathematics

3: Dictionaries

• First English dictionary
• Origin of alphabetical ordering
• Definitions enabled science
• Dictionary feedback loop: presence of dictionary drives new vocabulary
• Language → common knowledge

4: Computer precursors

• Cost of computing vs cost of looking up
• Computing was a cottage industry
• Logarithms
• Language should be rationalized, formalized, and made predictable
• First computers
• Babbage and Ada
• Difference machines, analytical engine
  • Functions, variables
• ↑ indirect precursor to computers

5: Telegraphs

• Telegraph
• Transatlantic comms in 1850s
• Had to divorce concept of message from medium in which it was written
• Encoding (save $$)
• Code books
• Binary
• Symbolic boolean logic

6: Telephones

• Logic circuits, binary arithmetic
• Mathematics could never be proved free of contradiction
• Telegraph → telephone
• Channel bandwidth

7: Information theory

• Turing machine: tape, symbols, state
• Data + instructions are all numbers
• Incompatibility → incompleteness (halting problem)
• Cyphers, cryptography
• Patterns = redundant
• English is ~50% redundant
• Information = uncertainty, surprise, difficulty, entropy
• Communication must contain:
  • Information source
  • Transmitter (encodes)
  • Channel
  • Receiver (decodes)
  • Destination
  • Noise source(s)
• Continuous and discrete
• Language n grams
• Info efficiency / redundancy tradeoff (dependent on noise)

8: The informational turn

• Philosophical: pockets of desecrating entropy
• Positive + negative feedback
• “Man is recreating himself”
• Conference on Cybernetics
  • “Noah’s Ark” approach to bringing together cross-disciplinary experts
• Learned mental models / theories
• The imitation game / Turing test
• Turing machines (substrate independent)
• Information theory → psychology
• 7 bits of working memory
• Recoding: chunking of info to abstract to higher levels
• The cognitive revolution / the informational turn: cognitive science: psychology + CS + philosophy
• Catalyzed the social sciences

9: Entropy and its demons

• Entropy = the unavailability of energy
• Laws of thermodynamics
• Probabilistic second law: there are more disorderly states than ordered states
• Maxwell’s demon: uses information to reduce entropy (chance → purpose)
• Entropy = uncertainty in the state of a physical system
• Life sucks in negative entropy
• Genes = an aperiodic solid

10: Genes

• DNA as code + information
• The central dogma:
• The Selfish Gene
• Genes are the primordial replicators
• Organisms are cooperatives of genes
• Genes, abstracted, can apply to anything, eg. reading

11: Memes

• Memes should be considered living structures
• The age of virality
• Memetics
• Hard to define or mathematize

12: The sense of randomness

• Deterministic computers can’t create true randomness
• A number is not random if it is computable
• Information = what is the length of the shortest program that will generate it?
• 3 methods for determining info:
  1. Combinatorial:
  2. Probabilistic:
  3. Algorithmic:
• Complexity: ratio of bits of info to algorithm needed to generate it (code)
• Information ↔ randomness ↔ complexity
• Complexity is generally uncomputable
• But randomness for the sake of randomness is not interesting
• Logical depth: measure of value of complexity
• Buried redundancy: ↑

13: Information is physical

• Quantum information theory:
• The bit is the ultimate un-splittable particle (not atoms)
• Information must always be preserved, reversible (quantum mechanics)
• ↑ black holes seemingly violate this
• Info only lost when erasing, not processing info
• Qubit: 2 states but indistinguishable due to Heisenberg uncertainty
• “A nonrandom whole can have random parts”
• Spooky action at a distance
• Quantum computers are, in a sense, computing their own futures all the time

14: After the flood

• Quantum waves vibrate outward indefinitely → reversible
• “The universe is computing its own destiny”
• Wikipedia is an example of how actual reality can never be pinned down
• Trademarks + namespaces
• Exponential info in tech
• Info in universe
  • 10^120 ops
  • 10^90 bits

15: New news every day

• Too much information
• Information anxiety
• Gap between info and knowledge
• Inverted U of information benefit

Epilogue

• “A widespread world consciousness aware of itself”
• ↑ = Wikipedia?
• Meaningless disorder is meant to be challenged, not feared
• MIT center for collective intelligence
• Google search = harnessing of collective knowledge
• 6 degrees of separation = everything is near and far at the same time