Data, data, everywhere and no one stops to think


Data everywhere

Who would have thought just 0 and 1 would define who we are

I was at a very interesting Churchill Club event recently about the use of sensors in electronic devices and software applications when I heard two very interesting statements:

  • “There are too many data being collected”
  • “Too much information is confusing”

I mention this because I always bristle when absolutes are used. OK: correction, I usually bristle when absolutes are used.

So when someone says “… too many data …” and “… too much information …” my instinct is to challenge the absolute and try to find the right adjective to describe the, in this case, proliferation of data and information.

It’s true there are certainly a lot of data and information out there. But is that a bad thing?

Imagine that we implant a sensor in our hip that monitors our body temperature. It then relays this to our doctor every minute. In the course of a year that is over a 500,000 data points per year. And what if the daily average is a constant 36.9° Celsius – normal? Is the fact that we collected samples every minute “too many data”, is it “too much information”? Not if we look at all the data and see a 5 degree increase for 5 minutes after meals and a 5 degree decrease in the morning before waking.

The data aren’t inherently the problem. The information we present from the data aren’t the problem either.

It’s the analysis stupid. It is how we aggregate and summarize the data and present it that matters the most. We claim to make decisions based on the data but, in fact, we make them on the analysis of the data.

Instead of asking what the average body temperature is we should ask what are the mean and the standard deviation? Then we would see there are fluctuations. Or we could map the daily averages at the same time of day and we’d again see the fluctuations.

Storage is cheap. Let’s collect all the data we can. I, for one, want to see a time in the future where some surgeon can look at all the historical data about my vital signs and, applying some insight to those data, predict and prevent bad things from happening to me.

About Kevin

In the past year Kevin has spoken at 20 conferences and seminars on a range of leading IT topics, including methodologies, business analysis, quality assurance techniques, governance, open source issues, tool interoperability, release management, DevOps, Agile, ITIL, from the mainframe to distributed platforms to the web, mobile, wearable and embedded systems. He is a much sought after speaker, recognized around the world for his provocative and entertaining style. Kevin is a 40 year industry veteran, holder of three technology patents and today is VP of Worldwide Marketing and Chief Evangelist at leading Application Development and Deployment vendor Serena Software. In the past decade he has been crossing the globe and has met with over 4,000 people. At Serena he works closely with industry analysts, the press, customers, partners and employees to exchange ideas about industry direction and business issues. He was born and educated in the UK and lives and works in the Bay Area, California.
This entry was posted in Business and Technology, Personal experiences and tagged , , , , , , . Bookmark the permalink.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s