Human Limits

It has been a few weeks since I posted on larger societal issues like longevity or the U.S. Federal Budget.  However, a recent column by David Brooks in the New York Times on “Big Data” caught my eye.   The idea is that in the electronic world there is all sorts of data out there that can be analyzed and that many of the ways we think and many of the assumptions we make don’t hold up when analyzed.   As early as the 1980s people started to evaluate the idea of the “hot hand” in basketball and found the widely accepted idea that people go on hot streaks is not accurate.  Brooks goes on to highlight a number of areas where the data from various sources is being used to challenge long held ideas or conventional wisdom.

One problem with the current infatuation with “data” is that we don’t always know much about the quality of the data.   This has been acknowledged by IBM, a company that has made a major push into the business of what is generically known as analytics:

 

“The volume of data produced today isn’t just increasing—it’s getting faster, taking more forms and is increasingly uncertain in nature. Uncertainty arises from such sources as social media, imprecise data from sensors and imperfect object recognition in video streams. IBM experts believe that by 2015, 80 percent of the world’s data will be uncertain.”

 

We see this all of the time in medical research where information gleaned from clinical records has to be scrubbed before it can be used in a meaningful way to address a scientific question.   It is also possible to be “led astray” using observational or easily available data.   There are a number of high profile examples of medical practices based on observational data did not hold up or were reversed when they were tested in a more rigorous fashion using a more controlled approach.   One of the best examples is post-menopausal hormone replacement therapy.   However, this remains controversial and full of nuances, so the more we know sometimes the more we don’t know.

The other issue here is that all the data in the world is no substitute for judgement.   I have used the example of Robert McNamara in the past and his “data driven” approach to “managing” the war in Vietnam.  I would urge everyone interested in the limits of number crunching and metrics to watch the Fog of War.  As the clip below points out, if your fundamental assumptions are wrong all of the data in the world will not help you get it right.

 

 

The final and perhaps more optimistic point I want to make is that sometimes an off the wall anti-metric approach is so disruptive that it wins and is incredibly successful.   The ideas about a hot hand in basketball aside, for those of you interested in really thinking differently the ESPN 30 for 30 “Guru of Go” on Paul Westhead and Loyola Marymount’s ultra-fast style of basketball is instructive.   Westhead was a traditional coach focused on defense, ball-control, getting a good shot and minimizing turnovers until he realized there was a faster and more chaotic (yet organized) way to play.

 

 

Summary:  The current obsession with metrics, data driven approaches, analytics and big data need to be taken with more than one grain of salt.   Sometimes it is the stuff you can’t measure that really counts.  All of us who value freedom and democracy are lucky that Winston Churchill was not concerned with the “metrics” of World War II when Britain stood alone and all seemed lost in 1940.   As the psychiatrist/historian Anthony Storr pointed out:

 

“In 1940 Churchill became the hero that he had always dreamed of being.… In that dark time, what England needed was not a shrewd, equable, balanced leader. She needed a prophet, a heroic visionary, a man who could dream dreams of victory when all seemed lost. Winston Churchill was such a man.”

 

If history is any lesson, then perhaps the important questions for the future are less about data and more about judgement.

 

This entry was posted on Thursday, February 7th, 2013 at 6:08 am and is filed under Current Events. You can follow any responses to this entry through the RSS 2.0 feed. You can leave a response, or trackback from your own site.