Summary

Hamming teaches an art / style of thinking for doing great work: concentrate on fundamentals, learn how to learn, choose important problems, and build a vision that biases your daily random walk. Technical mastery is necessary but insufficient; creativity, preparation, and sustained habits make you ready when opportunity appears.

Progress also depends on seeing beyond the local task. Systems engineering keeps larger goals in mind, reliable data is rare, you get what you measure, and new technology must be judged against legal, political, social, and ethical constraints. Great work is social: keep the open door, seek the right problems, and learn to sell ideas through clear presentation.

“Luck favors the prepared mind”

“In a sense, the main difference between those who go far and those who do not is some people have a vision and the others do not”

“When to doubt, when to examine the basics, when to think for yourself, and when to go on and accept things as they are is a matter of style - and I can give no simple formula on how to decide. You must learn from your own study of life.”

“What you learn from others you can use to follow. What you learn for yourself you can use to lead”

“Systems engineering is the attempt to keep at all times the larger goals in mind and to translate local actions into global results”

“Many many solutions solve the wrong problem correctly. In a sense, systems engineering is trying to solve the right problem, perhaps a little wrongly, but with realization that the solution is only temporary and later on, during the next round of design, these faults can be caught provided insight is obtained.”

“The open mind leads to the open door, and the open door tends to lead to the open mind”

Chappy’s Review

Such a good book! Lessons on art/style applicable to general self leadership + management written in exactly the way my brain works haha. Also, so fascinating to take a course on the thoughts of a scientific genius from 25+ years ago. Surprisingly prophetic and countless timeless quotes!

Key Takeaways

Notes

Foreword

• “This book is full of great people doing great work”
• “Science is a fundamentally social enterprise”
• Lamenting brightest minds doing startups now instead of PhDs
• Hamming’s greatness study included no entrepreneurial capacity
• Discoverers, not inventors
• Bell Labs + expectation to serve the public interest
  • “Science and engineering as public service”

Preface

• “Style of thinking”
• Art = style

1: Orientation

• Absolutely awesome course intro*
• Most effective to teach thru firsthand recounting, successes > failures
• Analogy to art is particularly apt: find a mentor and learn craft from them, but must create own style and adapt to the times
• Meta-education
• Doubling time of knowledge is 17y
• 1/2 of knowledge is obsolete every 15y
• ↑ answers:
  1. Concentrate on fundamentals
  2. Learn how to learn (new fields)
• Science: “if you know what you’re doing, you shouldn’t be doing it”
  • ie. creativity
• Engineering: “if you don’t know what you’re doing, you should not be doing it”
  • ie. practical
• Fields are blending and you’ll have to learn / self-teach faster than ever
• Future best understood thru understanding the fundamental underlying forces at play
• “In a sense, the main difference between those who go far and those who do not is some people have a vision and the others do not”
  • Eg. random walk with bias
• Correctness of vision less important - just need one
• Must distinguish:
  1. What is possible - science
  2. What is likely to happen - engineering
  3. What is desirable to have happen - morals / ethics / value judgements
• “The unexamined life is not worth living” - Socrates

2: Foundations of the Digital Revolution

• Analog → digital
• Transistors + ICs
• More of workforce making things → organizing information
• More and more…
  • Simulation > physical
  • What do we want to do > what can we do
  • Micromanaging (bad)
  • Computer + information superiority
  • Generalized chips (bitter lesson?)

3: History of Computers - Hardware

• Early analog computers
• Early punch card computers
• Lesson: 100x’s and can’t possibly project how it’ll change applications
• Moore’s law
• Parallelism
• State machine

4: History of Computers - Software

• Binary → hex
• First book on programming in 1951
• “Real programmers” rejected Fortran for binary purism - lesson to keep up with the times
• Trend: absolute → virtual machines (layers of abstraction?)
• “Almost everyone who opens up a new field does not understand it as well as its followers do”
• Universal Turing machines
• Prediction: by 2020 the experts in domains will be doing most of programming rather than experts in computers
• Programming is more an art than engineering - so many ways to do it
• 16% annual rate of improvement in programmer productivity in 1950s (100x)

5: History of Computer Applications

• History of computation → more and more applications as costs dropped
• Next frontier: pattern recognition + AI

6: AI I

• Rule-based systems
• Expert systems: extract rules / domain expertise via interviews
• AI: what burdens can computers alleviate for humans?
• Thinking: binary or to a degree?
• Human uncertainty of self-awareness, consciousness
• Aren’t computers learning machines just like us?
• Must undo your biases with AI and whether computers can think - write down and test assumptions

7: AI II

• “Maybe thinking is just an artifact of largeness”
• Most interesting: what can humans + machines do together?
• Most people are not equipped to do non-routine jobs
• Logical vs psychological novelty
• “In thinking, maybe we have confused what is done with the way it is done - and this is the source of much of our confusion in AI”

8: AI III

• First, clearly express your opinion and examine back and forth with counterarguments until you’re fairly clear as to what you believe and why you believe it
• Think and articulate your opinion clearly
• The more you’re clear on one side, the more you should probably argue the other side

9: n-Dimensional Space

• ↑ also L1, L2, L_infinity distance
• L_1 = sum of coordinates distance
• L_2 = Euclidean distance
• L_inf = single max of coord distance

10: Coding Theory I

• Shannon insisted on “information” > communication theory
• Source + channel encoding, noise

11: Coding Theory II

• Huffman coding

12: Error-Correcting Codes

• Error correction + detection
• “Luck favors the prepared mind”
• Establish the style of doing great things so you’re ready when the opportunity arises

13: Information Theory

• Information = surprise = -log probability of an event
• Entropy
• Channel capacity

14: Digital Filters I

• FFT
• Signal: equally space digital sequence of numbers
• Quantization
• Aliasing
• Sampling

15: Digital Filters II

• “People always like to think something new is just like the past. They like to be comfortable in their minds as well as their bodies”
• “There are opportunities all around yet people rarely reach for them”
• “You are famous only when your name is spelled with a lowercase letter”
• hamming window
• Teamwork is more and more essential

16: Digital Filters III

• The systematic decision of non-recursive filters*
• When you say something can’t be done, remember why so you won’t look like a goof when the answer changes

17: Digital Filters IV

• “To the extent you can, choose problems you think will be important”
• You must consciously doubt your own beliefs
  • But too much doubt and you’ll be paralyzed
• “When to doubt, when to examine the basics, when to think for yourself, and when to go on and accept things as they are is a matter of style - and I can give no simple formula on how to decide. You must learn from your own study of life.”

18: Simulation I

• Need domain experts
• Does stability or instability dominate the problem? Determines simulate-ability
• Start simple
• Jargon - blessing and a curse

19: Simulation II

• Simulations alter the real world, eg. stock trading

20: Simulation III

• Simulations = project into the future. Applies to life too
• Better to make decisions and iterate quickly rather than waffle

21: Fiber Optics

• Must anticipate / predict implications of technological shifts
• Actively lead > passively follow
• “Frequently what is technologically feasible and is even economically better is restrained by legal, political, and social conditions. Just because it can be done does not mean it should be done”
• ↑ key to predictions

22: Computer-Aided Instruction (CAI)

• Hawthorne effect: necessity for double blind experiments
• ”Programmed books” for education never worked (choose your own adventure style)
  • Lacked hard data as example
• “What you learn from others you can use to follow. What you learn for yourself you can use to lead”
• Must first create a vision for what an educated person in 2020 looks like when laptops are prevalent

23: Mathematics

• Must occasionally re-examine the basics
• Mathematics: “the language of clear thinking”
• Platonists vs formalists vs logicians
• His view: we are both masters and servants
• “Obviously, there’s more for the future to discover than ever we have discovered in the past”

24: Quantum Mechanics

• Newtonian → relativity + QM
• Wave particle duality not required before QM was useful
• “Man is not a rational animal, he is a rationalizing animal. Hence, you will find that what you believe is often what you want to believe rather than being the result of careful thinking.”
• Our brains don’t understand QM
• The future is getting more complex, so there will be more and more things we do not understand

25: Creativity

• Creativity vs originality
  • Multiplying 2 random numbers is original but we wouldn’t say creative
  • Thus, must have some value
  • Also needs diffusion
  • Brain must be in the right state, but no consistently “right” approach
    • Like sex - have to do it to know what it’s like
• General approach:
  1. Recognition of the problem in some dim sense
  2. Refinement + emotional commitment
  3. Insights + iteration
• Tips ↓
  • “Prepare your mind for success”: saturate the subconscious with the problem - deprive it of all else as best you can
  • Analogy is the most important tool
• Need a strong will
• “We are nothing but our habits”
• Most creative when young so get at it

26: Experts

• The Structure of Scientific Revolutions - experts operate within existing paradigms
• “In some respects, the expert is the curse of our society”
• “It is asking a lot of them to admit the very methods they used earlier to succeed are not appropriate at present”
• “We are all human, but that does not prevent us from trying to modify our instincts”
• “Civilization is merely a thin veneer”

27: Unreliable Data

• Data and model error
• Definitions (eg. poverty, unemployment) change over time which makes it challenging
• Stats like profit, inventory, GDP are often gamed / have feedback loops
• Skew
• “You need reliable data to make reliable decisions, but you will seldom have it with any reliability”

28: Systems Engineering

• Cathedral building vs brick laying
• “Systems engineering is the attempt to keep at all times the larger goals in mind and to translate local actions into global results”
• There’s always multiple layers of systems x time horizons x interests
• Risk of optimizing elements in disservice to the whole
  • Eg. course grade vs actually learning → use in vocation
• Telephones were really the first huge systems engineering problem
• “Specialists brought together are the basis of systems engineering”
• “Many many solutions solve the wrong problem correctly. In a sense, systems engineering is trying to solve the right problem, perhaps a little wrongly, but with realization that the solution is only temporary and later on, during the next round of design, these faults can be caught provided insight is obtained.”
• Deeper long-term understanding of the problem > the solution

29: You Get What You Measure

• Accuracy vs relevance of measure
• Eg. is intelligence normally distributed or is it just because our tests are calibrated to result in that?
• People tend to go into the fields that favor their peculiarities → reinforced → poorly balanced yet highly specialized
• Inbreeding + organization personality
• Eg. log scale vs linear
• “If the whole organization is working together to fool the top, there is little the top can do about it”
• Eg. counting lines of code is detrimental

30: You and Your Research

• 1. achieve your goals 2) set high goals
• “Luck favors the prepared mind” - yes it’s luck, but luck is up to you, too
• “Genius is 99% perspiration, 1% inspiration” - Thomas Edison
• Must have the belief you can do great / important things
• “If you do not work on important problems, then it is obvious you have little chance of doing important things”
• “Working with one’s door closed lets you get more work done per year, but I have observed repeatedly that those with the closed doors seem to work on slightly the wrong problems, while those who have let the door stay open get less work done but tend to work on the right problems”
• “The open mind leads to the open door, and the open door tends to lead to the open mind”
• “Intellectual work is like compound interest”
• Style = right problem, at the right time, in the right way
• Set aside time to “only think great thoughts” (10% of time)
  • Ask the larger questions and think in the future
• Learn to sell your ideas by force of clear presentation