- Format: Kindle Edition
- File Size: 1174 KB
- Print Length: 83 pages
- Simultaneous Device Usage: Unlimited
- Publisher: Machine Intelligence Research Institute (1 Feb. 2013)
- Sold by: Amazon Media EU S.à r.l.
- Language: English
- ASIN: B00C7YOR5Q
- Text-to-Speech: Enabled
- Word Wise: Enabled
- Customer reviews: 32 customer ratings
- Amazon Bestsellers Rank: #1,377,252 Paid in Kindle Store (See Top 100 Paid in Kindle Store)
Enter your mobile number or email address below and we'll send you a link to download the free Kindle App. Then you can start reading Kindle books on your smartphone, tablet, or computer - no Kindle device required.
To get the free app, enter your mobile phone number.
Facing the Intelligence Explosion Kindle Edition
Would you like to tell us about a lower price?
There was a problem filtering reviews right now. Please try again later.
Top international reviews
It seems particularly designed to convince a skeptic why Artificial Super Intelligence (ASI) is not only possible but likely. It does this by carefully reviewing the proper reasoning/rationality required to see the logical inevitability of ASI and its dangers. However, it offers the "choir" not much more than clarity since we already get it.
I'm looking for deeper thinking and more examples on how ASI can go wrong, both from a programming perspective and in scenarios that could play out in our world—considering how the design of "friendly" ASI is so profoundly difficult. It did not deliver in this respect. In summary, I give it 3.5 stars. Since it's a quick and inexpensive read, I'll round up.
-There are legitimate reasons to have concerns about the development of an AI that is smarter than all of us and society needs to be more aware of these concerns, this book does contribute to getting this story out and is a good read for anyone not aware of potential threats from AI
-For some reason the author believes that being an atheist is somehow important or relevant to understanding this topic. I know some really brilliant Christians who can grasp these concepts easily and will not allow their beliefs prevent them from agreeing readily that danger may indeed lurk in careless development of an AI. If the author truly wants to rally awareness and support for his concerns he may want to consider leaving out his own beliefs
-This book and the "Less Wrong" site use what I think are silly examples of how an AI would misunderstand our desires and create a calamity, the book discusses how it could misunderstand saving mom from a building that's on fire, the website has a scenario where an AI decides to use all resources available to make paperclips and eventually burying us in them. It seems that something smarter than all of us combined that has access to the internet would be able to use "big data" analysis to understand we would not want mom killed when it removes her from the burning building or that people really don't want to be buried in paper clips!
-Truly, the author of this book and the contributors to "Less Wrong" are way more educated and smarter than I'll ever be, but in the interest of getting their message out to a larger audience they should consider presenting it in a manner appeals to a broader audience
And so Facing the Intelligence Explosion is equal parts discourse on the cognitive science of rationality, and problems with advanced AI. Both parts are persuasive. I could quibble with the long lead-in - nearly half the book goes by before AI rears its Medusa head. I also think the chapters about no God to save us (who thinks he would anyway?) and radical abundance and immortality (well trod Kurzweil territory) are distracting. But the overall effect is gripping nonetheless, and I hope readers not steeped in rationality and AI issues will bother to wade into the enriching links throughout the eBook.
Altogether Facing the Intelligence Explosion is a terrific primer on the most important conversation we could be having. You'll want to read it more than once, and recommend it to a friend.
The key problem (which I think is a brilliant, albeit short, discussion) begins in Chapter 12 (chapters here are very short), where the author begins to ponder how we would actually impart to an AI a set of instructions or rules on our values. "A different mind architecture," he notes, "one that didn't evolve with us, won't share our common sense. It wouldn't know what NOT to do. How would it make a cake? `Don't use a squid. Don't use gamma radiation. Don't use Toyotas.' This list of what not to do is endless." From this point the considerations become fascinating, with deep implications for the difficulty of imparting "values" to an AI. What the author fails to note over the course of these thoughts however is that all this is simply another version of the "frame problem," one of the most difficult and unsolved problems in AI, intimately correlated to the also all-but-given-up-on problem of commonsense knowledge - a knowledge achieved by humans only by interaction with the concrete world (not Watson-like by scanning Wikipedia text). For example, as a robot observes the coffee being stirred, the cup begins to bulge in and out, or the coffee surface erupts with small geysers, or there is a "snap, crackle, pop" sound. Are any of these an "expected" feature of the event? The robot (under one version of the problem) must check his (large) list of frame axioms for the event - the list of what remains unchanged in the world when one stirs coffee - a very computationally expensive task. Correlatively, if my wife asks me to stir the coffee, she would be shocked if I used a chair (or a Toyota) for the stirrer. The point being that we are equally dealing here with a problem in the knowledge of the concrete world, not simply with "values," and such knowledge (i.e., the same problem) is implicated in design, in problem solving, in invention and thus is equally and fully at the heart of the line of AI development of a "raw intelligence" which the author is so worried will be brought to fruition long before we figure out how to impart value. That is, his worry on the wrong winner of the race is unfounded - the problem is the same in both races.
Allow me to expand briefly, for the point is critical to bringing AI folks to reality. It relates as well to Muehlhauser's key feature of intelligence, to wit, "an ability to achieve goals in a wide range of environments," e.g., to get the coffee stirred. The concrete world is the most important of "environments." Coffee stirring is a perceived event in this concrete environment. Yes, per Muehlhauser's source of shared values concern, it is a "shared" experience among us all, but this sharing is the secondary problem. The problem is perception, this being the problem of the origin of our image (visual, kinesthetic, auditory) of this external environment - the coffee cup with swirling surface, the clinking/stirring spoon, the table surface, kitchen floor - and therefore of our experience of concrete events in this environment occurring over an extended, continuous time. The coffee stirring event is itself defined by a vast structure of invariance laws with invariants existing only over a continuous flow of time and embedded within a concrete physical dynamics (again, not within Wikipedia text, i.e., over any static data structure): there is the radial velocity flow field defined over the stirred coffee surface (why geysers don't "fit" or are "unexpected"), in the back and forth stirring motion there is an adiabatic invariant (a constant ratio of the energy of oscillation to the frequency of oscillation), and an inertial tensor whose elements define the resistance to angular acceleration as we wield the spoon (why wielding either a sledge hammer or a martini stirrer would feel anomalous), plus the normal resistance value of the liquid medium (which is why I don't need a cement mixer), and on. One must have a device that has solved the origin of the image of the external world, and further, is capable of registering the invariance laws that structure that world's dynamic events. No AI theory has even bothered to consider these laws. Without a "device" that has solved the origin of the image, therefore the nature of experience, therefore how this experience (with its invariance) is stored (or not), one has no grounded model of thought, most critically then no model of the ability to make analogies based upon this experience, where analogy, as Hofstadter ( Surfaces and Essences: Analogy as the Fuel and Fire of Thinking ) has just thoroughly demonstrated, is the foundational operation of thought (while it is clear he has little idea, and rather doubt, as to how a computer can implement this operation) and thus of problem solving, design, and the achievement of goals.
So, yes, it is critical to let the profound implications re value in this little book register, though, imo, its ultimate bet/fear on the race-winner is unfounded. Minus one star for the lack of this perception. But we should begin then to contemplate the actual nature of events, of our perception of these events, and therefore of what our knowledge actually must involve to "achieve goals" in this concrete world. AI must finally acknowledge that there is a science being resolutely ignored, the science of the ecological laws defining events. For this, I would point to J. J. Gibson ( The Senses Considered as Perceptual Systems ) and ecological psychology, to Kugler and Turvey ( Information, Natural Laws, and Self-Assembly of Rhythmic Movement (Resources for Ecological Psychology) ), and if curious, ] for an attempt to elucidate and place Gibson in this larger context of consciousness and cognition.
Since the book is written by a human, it should not be surprising that it argues for the survival of the human in modified form over Nature's more traditional path of "extinction and replacement".
Were a machine to write a similar book, the emphasis might be more on maintaining stability during a transition from a human/organic dominated planet to a machine/inorganic dominated solar system. After all, during the transition there will be an extended and unstable period of time during which we (both humans and machines) can view advanced forms of intelligence as a form of weapon wielded by humans, rather than as a set of replacement species.
Perhaps transhumanists will at least smooth out the rough road we humans and machines face over the coming decades as humans pass the baton to machines.
So, to the book. Much of the intext links it contains are to various parts of LW, which is right and proper, given the source. What was a pleasant discovery was how smoothly it flowed together. Where the Sequences are something of a Rube Goldberg arrangement of mental concepts, this book presents a sort of path blazed through them all for the sake of an argument, in this case, the importance of FAI as an existentially critical goal. This even helped my own understanding, as I got past the Hands vs Fingers reductionsim regarding free will that has dogged my mind for more than a year.