Too Big to Know: Rethinking Knowledge Now That the Facts Aren't the Facts, Experts Are Everywhere, and the Smartest P [Kindle Edition]

The premise of this book is that somehow networked organizations and networked thinking will lead to better, smarter decisions. As long as we include a sufficient diversity of opinions and experience in the networks helping us make our decisions we will arrive at better, more informed answers. In fact, as the amount of information explodes, these networks will be the only way to manage all the information we are creating.

Here's the problem. I don't think anyone will dispute that reaching out the to internet to search for knowledge can get reasonable answers quickly. Also, running contests where many experts are involved can get good results. The problem is, if you are solving a real problem at the end of the day somebody actually has to do the work to get an answer. A "network" isn't going to magically come up with an answer. Also, reaching out to a wide group on the internet often results in the same stupid *wrong* answers to a problem being circulated around and around and around. Networks can just as easily work in a negative direction recycling stupidity rather than knowledge. There doesn't seem to be much of a role in this book for sustained critical and deep thinking about a problem to arrive at a solution. This doesn't make sense to me since much of human progress continues to come from sustained hard work by individuals working to achieve expertise in an area and focusing on a single problem at a time. This book makes some good points about how our relationship with information is changing to rely more on networks of our colleagues or friends to filter and absorb the massive amounts of information created every year. However, the author's confidence that networked thinking and organizations will magically solve many of our problems is happy nonsense, in my opinion. Also, while the author claims that networks can make better decisions, he never really gives any detailed research supporting this assertion or showing under what conditions networks are better or worse at solving problems. This makes the book more of an exercise in faith rather than something you can use to decide if a network would be helpful in a particular problem or not.

Having his background in philosophy, perhaps no one had expected David Weinberger to write a book on a topic that is at the heart and soul of librarians, i.e. cataloguing and classification. In the modern notion of the term this is called metadata. When Everything is Miscellaneous was published in May 2007, at first it was as if some war was waged against Melville Dewey's classification system, especially the class 200 for Religion. Some protagonists in the field such as Peter Morville responded with an apt blog entry arguing that "Not Everything is Miscellaneous". In his book, even more in his several book talks, Weinberger mocked not only Melville Dewey and Michael Gorman but also Aristotle, albeit with a great caution. In many ways though, the book has slowly been well received and cited widely in the library and information science literature. The book would be considered as disruptive in its argument against some of the conceptual foundations of library and information science, mainly classification and categorisation systems. In Everything is Miscellaneous, Weinberger called for a total rethink of not only the notion of classification systems but also the very definition of metadata. For him, "metadata is what you already know and data is what you're trying to find out" (Weinberger, 2007, p.104).

Now his new book is out as of early January 2012. Too Big To Know: Rethinking Knowledge Now That the Facts Aren't the Facts, Experts Are Everywhere, and the Smartest Person in the Room Is the Room. In this book, Weinberger offers yet his staunchest critique on well established conceptual and theoretical foundations of knowledge including the DIKW (Data- Information-Knowledge) pyramid that most computer science and library science have incorporated in their curriculum in their foundations course.Another concept he took aim is information overload. A typical Google search on the phrase information overload returns more than 6 million results (doubled even since Weinberger records this statistics). Popularised by the technology futurist Alvin Toffler, the phrase resonates in the minds of librarians who for so long have hinged their value proposition on solving the problem of having too-much-information. As Weinberger notes, information overload, also called info glut, data smog, or information tsunami, is a problem so serious that it has become a topic for a whole body of work. Not only that, the problem also warranted inclusion into the scientific and psychiatric dictionary with its nomenclature such as information anxiety or information fatigue syndrome. In Too Big To know, in what seems a disruptive argument, Weinberger tells that too much information is actually a good thing. To support his argument, he cites Clay Shirky, who argues that "it is not information overload. It is filter failure".

By providing several examples and writing rather beautifully, Weinberger contrasts the long-form argument of the Age of Books with the loosely connected webs of the Age of Networks in which he argues, the long form argument is a constraint inherited from the medium of print. Our thought process, nonetheless, works not in a simplistic, linear and long form ways but in an intricate web of links and associations which is better reflected in the Age of Networks. Scientists work in private in the Age of Books, after-the-fact peer-review is the norm, but in the Age of Networks, he argues, the filtering process is immediate, open and on the cloud. In short, he argues the abundance of crap and good that is generated through the network gets filtered by the network itself.

Reading this book, one can surmise that Weinberger is for Open Access. He is for Open Internet. He is for Open Data. He is for Linked Data. Such an open ecology, Weinberger argues, provides a fertile ground for innovation and creativity. Overall, influenced by less baggage from the disciplines of either computer science or library science, Weinberger seems to suggest that the influence of the Age of Books is fading and the time has come for the Age of Networks. Hence, he argues, knowledge is now residing in the network, not on any one or even genius skull.

In many respects, Weinberger's Too Big To Know, is in agreement with arguments put forward by James Surowiecki's The Wisdom of Crowds and Clay Shirky's Here Comes Everybody. Perhaps a slight oversight in the book may be the notion of `information overload as good and inevitable' was first discussed by David Allen in his book Getting Things Done (2002). We will await reflections from other authors such as Andrew Keen who may argue against some if not most of the views espoused by Weinberger.

The book is relevant to the field of library and information science in many ways. The discussion on the value of metadata, especially on metadata that is generated by users in terms of user tagging, ratings, reviews, filtering, and recommendations (pp.186) is crucial for librarians who are at the cross-roads of choosing between old and new metadata paradigms. This book illuminates new ideas on library collections and provides a glimpse of what the future of libraries would look like, albeit Weinberger's discussion is at a philosophical level. Metadata is ultimately one of the solutions to the filter failure in which he asserts "the solution to the information overload problem is to create more metadata". This is a notion, I believe, we in library and information science should develop further.

First the disclosures. I asked for a copy of this book to review, David Weinberger being one of my heroes and I being unemployed at this time. They gave it to me and now that I have read it, I will be donating it to the Oakton, VA public library.

Second, the subtitle. The subtitle of the book captures the entire field perfectly, and richly merits emphasis: "Rethinking knowledge now that the facts aren't the facts, experts are everywhere, and the smartest person in the room is the room." This is the final nail in the coffin of secret intelligence communities and companies devoted to proprietary software. There is nothing intelligent -- nor substantively valuable -- about "closed" environments if ones purpose is to optimize both the allocation of resources and outcomes beneficial to the public.

Third, the historical context. Many people have been focused on the changing role of knowledge coming into the 21st century, and I list just five of the books below to make the point that in the context of all else, this book says it better, more easily graspable for the non-digital leaders struggling to decide where to go next --this book is highly relevant to the 1950's mind-set leaders of all eight tribes of intelligence: academic, civil society, commerce, government, law enforcement, media, military, and non-governmental / non-profit.

The exemplar: The exemplary performer in the age of productivity
Radical Man: The Process of Psycho-Social Development
The Knowledge Executive
Infinite Wealth: A New World of Collaboration and Abundance in the Knowledge Era
Powershift: Knowledge, Wealth, and Violence at the Edge of the 21st Century

Summarizing the book concisely: everything we do now with hierarchical organization, hoarded information, restricted accesses, and isolation from the full range of external sources and methods, is wrong for the times.

Here are the five recommendations the author discusses in his last chapter, every single one of them poorly addressed by most organizations, and especially those that are highly bureaucraticized:

01. Open up access
02. Provide the hooks for intelligence (meta-data)
03. Link everything.
04. Leave no institutional knowledge behind.
05. Teach everyone.

This is a provocative book with a strong message at four levels:

Strategic: "The more one looks at the question, the clearer it becomes that we don't have an agreed-upon explanatory [or even exploratory] framework within which the question might be resolved." [page 114]

Operational: It not about what you know or even who you know, but about the network you create so as to be able to access the right relevant knowledge when you need it, in the form you need it.

Tactical: Centralized authority rots -- decentralized authority not only has the agility to be effective in situ, but is much more likely to "see" and integrate local knowledge.

Technical: Citing Clay Sharky, "It's not information overload. It is filter failure."

While I can see where those deeply familiar with the literature on collective intelligence might find the book basic, I am always on the look-out for books that can explain my complex views to busy leaders that can barely compute three colors (red, yellow, green) and this book fits the bill. We are suffering an entire generation of leaders not only brought up with Weberian administrative and hierarchical standards, but never held accountable for failure. For those that would like to use their last couple of years doing some good instead of more damage, this book is a perfect beginning to what might be a fruitful conversation with those of us who "got it" in 1988 and have been trying ever since to help others "get it."

KEY CONCEPT: Knowing by reducing / filtering / weighting is out of date. The best knowledge is linked / inclusive / open.

Today's networked knowledge -- and the digital natives that excel at creating and interacting with digital multi-media knowledge -- is directly antithetical to every possible attribute of the Epoch A top-down, "because I say so" authoritarian hierarchies. When even the Harvard Business Review acknowledges that CEO's not only don't know what they need to know to make good decisions, but are also not able to to make those decisions by themselves, the times, they are a'changing.

The 21st century institution, like the 21st century network, must be very wide, boundary free, populist, respect those who are credentialed by the network rather than diplomas, and comfortable with a constantly changing "unsettled" landscape of culture, history, and local knowledge.

Old knowledge focused on FACTS. Intermediate knowledge has focused on CONTEXT. Now the new knowledge is focused on RELATIONSHIPS.

In all of this, TRUST is what determines the successful collection, processing, analysis, and decision-support rendering of information into intelligence. I speak here of informed trust, not blind trust. I cannot link to my own books, but I read this book in time to get several mentions of it into my forthcoming book, now listed on Amazon, THE OPEN SOURCE EVERYTHING MANIFESTO: Transparency, Truth, and Trust (Evolver Editions, 5 June 2012) and would also point everyone to the work of Robert Garigue (RIP) on security as a trust-building network rather than a lock-box.

The author points out that in the absence of a properly designed network, the massive amounts of information that are accessible, and the new ability of every nut-case on the planet to be "co-equal" to more measured professionals, makes it very (VERY) easy for even the most well-intentioned researcher to go off the deep end. In other words, not only do you have to have the world's best OPEN network at your finger-tips, you ALSO have to have a profoundly professional combination of automatic, social, and professional filters and ingestion / visualization capabilities -- see the still not existing Computer Aided Tools for the Analysis of Science & Technology (CATALYST) -- four of us knew what we needed in 1986, and we still do not have it because several generations of "leaders" refused to be serious about the information revolution and being accountable for actually "doing" intelligence useful to all elements of government at all levels. And as Ben Gilad points out in his still relevant, Business blindspots 2nd edition: replacing myths, beliefs and assumptions with market realities, CEOs are no better off -- surrounded by sycophants terrified that the next new idea from OUTSIDE might prove them to be obsolete (they already are, but keeping the Potemkin village alive is more important to them that actually serving the bottom line).

The author spends time on distinguishing between classic facts (in isolation), databased facts (in context, sort of), and networked facts (which have many different contexts, and as he demonstrated in his earlier book, Everything Is Miscellaneous: The Power of the New Digital Disorder, both visible to different sets of people, and useful in different sorts of ways.

The author speaks to science and how it is being affected by the network. While I would have preferred the inclusion of religion and philosophy as well, the following are worth contemplating:

01 Reality is too big for small theories [my interpretation is that one has to do whole-systems and real-time science now, the era of knowing everything about nothing that has characterized academia is now over -- the PhD's of the future will be constuctive constructionists, not destructive deconstructionists.]

02) Science is flatter. One no longer needs to spend 8-16 years as an intellectual serf. To this I would add that science is now also real-time (that is to say, serious science versus show science). Changes to the earth that used to take ten thousand years now take three. Both science and politics are so removed from current reality and from an integrated public perspective as to be very dangerous to the human species.

03) Network is continuously public. This is the part the secret world has difficulty with. They obsess on secret sources and methods as inputs, when all this time they should have been adapting to the prospect of being able to provide continuous decision-support to a full range of customers, in the process creating a Smart Nation. Instead they have a secret black hole.

04) Open filters. Buy the book.

05) Science with a difference. Buy the book. Cybernetics is now going in this direction, and we may be on the verge of a multi-disciplinary and humanities break-out in which the academic and government paradigms for thinking and studying experience an order of magnitude topsy-turvy "do the right thing, not the wrong thing righter." This is a great time to be an intelligence professional.

06) Hyper-linked science. I knew about citation analysis cabals in 1970, when the senior reference librarian, Diane Guldner, at Muhlenberg College, took the time to show me the stuff no one else really used. Science and the humanities today are a travesty. I just ripped apart some psycho-babble about "Intelligent Management of Intelligence Agencies" at Phi Beta Iota the Public Intelligence Blog, and I am still irritated. If we do not get serious about creating the World Brain and Global Game as Earth Intelligence Network has conceptualized the immediate possibilities, governments and corporations are going to continue a path that is tantamount to mass suicide for the human species.

Chapters eight and nine focus directly on leadership. Here also today's "leaders" are going to struggle. Citing Jack Welsh and his success at the time, the author points out that the leaders primary responsibility is to connect to reality and to make hard choices. I know too few who have even a clue about what reality really it (hint: poverty is the greatest threat, not terrorism), or the disposition to make hard choices (hint: close down half of the secret technical collection capabilities, invest in an Open Source Agency under diplomatic auspices -- Alec Ross got the email [visit Open Source Executive Access Point at Phi Beta Iota for the details], evidently someone scared him away from thinking independently and Hillary Clinton has no idea that the Office of Management and Budget is ready to give her $150M for year one going toward $2 billion at FOC).

Especially meaningful to me are the author's emphasis on the changing role of leadership -- it is no longer about making decisions and overseeing the process that brings the information to the decision point -- it is more like Ike Eisenhower assembling the force that crossed the channel -- the leader's greatest role is three-fold: pick smart people that are self-starters; create and nurture the OPEN network that allows those self-starters to be all they can be; and finally, be the catalyst for integrating diversity with clarity and integrity. The greatest moment in the Tom Selleck version of Ike - Countdown to D-Day is the moment after he says "GO." In that moment, he has fulfilled his leadership mandate, and the future now depends on every single Private and Corporal hitting the beaches, AS INDIVIDUALS within the NETWORK that he mustered over years toward that one decisive encounter. In that moment, his "leadership" is manifest in all others, not in himself.

I put this book down very pleased with the time/energy to reward ratio. I certainly do not agree with those that disparage this book. One has to remember that most leaders are 20-40 years away from their formative education, most of which is now out-dated; they do not have time to read for their continuing education; and they are surrounded by "mini-me" sycophants who dare not speak truth to power for fear of being reassigned or worse, sent into long over-due retirement. It may be that we have to wait, as my middle son pointedly told me in the car one day, for all of us old guys to die and get out of the way. I for one do not know of any leader with authority that is actually interested in doing the right thing instead of the wrong thing righter. If there is such a person out there, I am available, mobile, and cannot be stopped from doing the right thing...as I like to say, "the truth at any cost lowers all other costs."

With my last two links:
Ideas and Integrities: A Spontaneous Autobiographical Disclosure
Reflexive Practice: Professional Thinking for a Turbulent World