We only have to look at ourselves to see how intelligent life might develop into something we wouldn't want to meet.
As Irving Good realised in 1965, machines with superhuman intelligence could repeatedly improve their design even further, triggering what Vernor Vinge called a 'singularity.'
Interpretation
What this quote means
This quote highlights the concept of machines advancing to a point where they can enhance their own capabilities beyond human intelligence.
Stephen Hawking references a profound idea proposed by Irving Good in 1965 regarding the development of machines that possess superhuman intelligence. These machines are envisioned to have the ability to iteratively improve their own design, potentially leading to a point of unprecedented technological advancement known as 'the singularity,' as further described by Vernor Vinge. This idea raises important discussions about the future of artificial intelligence and its implications for humanity.
Themes
In practice
Example use cases
In a technology conference discussing future innovations.
More from Stephen Hawking
All quotes βI regard the brain as a computer which will stop working when its components fail. There is no heaven or afterlife for broken down computers; that is a fairy story for people afraid of the dark.
It surprises me how disinterested we are today about things like physics, space, the universe and philosophy of our existence, our purpose, our final destination. Its a crazy world out there. Be curious.
I was not a good student. I did not spend much time at college; I was too busy enjoying myself.
The world has changed far more in the past 100 years than in any other century in history. The reason is not political or economic but technological-technologies that flowed directly from advances in basic science. Clearly, no scientist better represents those advances than Albert Einstein: TIME's Person of the Century.
In my opinion, there is no aspect of reality beyond the reach of the human mind.
Similar quotes
I invented nothing new. I simply combined the inventions of others into a car. Had I worked fifty or ten or even five years before, I would have failed.
Overnight the digital age had changed the course of history for our company. Everything that we thought was in our control no longer was. But within a year we had invested in social media and digital experts. Now Starbucks is the number one brand on Facebook.
The hope is that, in not too many years, human brains and computing machines will be coupled together very tightly, and that the resulting partnership will think as no human brain has ever thought and process data in a way not approached by the information-handling machines we know today.
What we can do should never by itself determine what we choose to do, yet this is the way technology tends to work.
You're either on, and you're connected and distracted all the time, or you're off, but then you're wondering, am I missing something important? In other words, you're either distracted or you have fear of missing out.
It is possible to think that the Internet will be a net positive for society while admitting that there are significant downsides - after all, it's not a revolution if nobody loses.