Larry: Jeff, I was most interested by this concept from your book:
"The biggest portion of information that is used daily in business routine has never been captured. It is so-called “Tribal Knowledge”. My conservative estimate of the ratio between structured, unstructured and “tribal” knowledge is 10%, 20% and 70%. For many companies, more realistic numbers are 5%, 15% and 80%."
I run into this over and over again in my work experience. Decisions tend to get driven almost entirely by structured knowledge, while unstructured and tribal knowledge ----- or what I would label as "qualitative" or "contextual" knowledge --- is assigned a secondary role or dismissed altogether, often with disdain. After reading your book, it strikes me that structured knowledge is the "native language" of computers ---- they are far more advanced at capturing and manipulating structured knowledge (i.e., numerical data), than at capturing qualitative or contextual knowledge. Because of this, structured knowledge tends to be the native language of decision makers who rely on computer-mediated knowledge.
Let me offer a couple of examples.
When I am developing a museum exhibit, there is inevitably a large body of structured data about the audience ---- numerical research that is based on surveys, focus groups, market analyses, etc etc. Sometimes this research is handed to our team in completed form. In other cases, we develop and execute the research on behalf of our client. In either case, this type of information can be helpful to the creative team as we make decisions about many aspects of the exhibit --- the title, the subject matter, the physical design, the visual presentation, the style of writing, etc etc. However, the most creative and compelling insights usually come in unstructured / tribal form ----- from the museum staff, or the volunteers, or perhaps a case study from a similar museum in another city or state. It's very difficult to convince a museum administrator to trust the unstructured / tribal insights OVER the conclusions yielded by structured data. But when we are successful at persuading the decision maker to take a chance on an "unstructured" idea, it is almost always a huge benefit to the exhbiit. We usually realize positive outcomes that were not predicted by the structured data, and which we could not have achieved if we relied solely on structured data.
Was it clear so far?
A second example is in professional sports, which I follow closely and have written about extensively. Sports generate a large amount of structured data --- statistics like batting average in baseball, shooting percentage in basketball, pass completion rate in football, etc etc. Computers have made it possible to capture vast amounts of this structured knowledge, and perform advanced analysis of it. Nearly every professional sports franchise now employs a team of data analysts to use structured knowledge for player evaluation and analysis. However, the teams continue to employ traditional talent evaluators --- scouts, coaches, etc. ---- who provide unstructured knowledge. These individuals evaluate talent by observing the players directly and grading their technique, their motivation, their intelligence, etc etc. The most successful sports franchises integrate BOTH types of knowledge --- structured and unstructured ---- into their analysis. Many of them are working on proprietary computer systems that integrate both types of knowledge into one body of data.
Jeff: Larry, you point to an interesting area where combination of structured and unstructured knowledge under a semantic umbrella might be very fruitful. Your response provides a great deal of analysis in the area of your expertise (journalism, sport, evaluation, decision making) and gives real examples of how this combination can work in the complex decision-making process.
This technique can potentially be used to evaluate quality of content.
I think that one of the biggest problems created by the Internet is content quality.
The Internet made information “plenty and cheap”. But the pearls of content, which sometimes present by real journalists, are not distinguishable in this ocean of data. Combination of expert values with computer-based semantic analysis and editorial services conversing with the knowledgebase can elevate the best pieces to the surface. The brilliant content will shine with its real value giving professional journalism the second life.
Larry: Your summary of my response ---- "combination of structured and unstructured knowledge under a semantic umbrella" ---- is exactly what I am trying to describe.
It seems to me that having this type of tool could be incredibly powerful in expanding our range of solutions to various problems. It would mean, perhaps, that knowledge which is currently overlooked would become more prominent in our thinking.
