I was at a very interesting Churchill Club event recently about the use of sensors in electronic devices and software applications when I heard two very interesting statements:
- “There are too many data being collected”
- “Too much information is confusing”
I mention this because I always bristle when absolutes are used. OK: correction, I usually bristle when absolutes are used.
So when someone says “… too many data …” and “… too much information …” my instinct is to challenge the absolute and try to find the right adjective to describe the, in this case, proliferation of data and information.
It’s true there are certainly a lot of data and information out there. But is that a bad thing?
Imagine that we implant a sensor in our hip that monitors our body temperature. It then relays this to our doctor every minute. In the course of a year that is over a 500,000 data points per year. And what if the daily average is a constant 36.9° Celsius – normal? Is the fact that we collected samples every minute “too many data”, is it “too much information”? Not if we look at all the data and see a 5 degree increase for 5 minutes after meals and a 5 degree decrease in the morning before waking.
The data aren’t inherently the problem. The information we present from the data aren’t the problem either.
It’s the analysis stupid. It is how we aggregate and summarize the data and present it that matters the most. We claim to make decisions based on the data but, in fact, we make them on the analysis of the data.
Instead of asking what the average body temperature is we should ask what are the mean and the standard deviation? Then we would see there are fluctuations. Or we could map the daily averages at the same time of day and we’d again see the fluctuations.
Storage is cheap. Let’s collect all the data we can. I, for one, want to see a time in the future where some surgeon can look at all the historical data about my vital signs and, applying some insight to those data, predict and prevent bad things from happening to me.