Big Bag of Data
·
So data < information < insights
Having known this, why would we think that more data would help bridge the gap between data and information?
That is in essence the fallacy of big data. Big data will expand the range of hypotheses, it will also extend the range of the search space. Nothing gets easier with the addition of more data, especially when you are in the business of data exploration. The main job of a data scientist is to find possible hypotheses that describes what is happening with the data. The data does not lie, or at least lies less than the human :)
Should we collect more data? Should we care?
Just because data has grown exponentially, does not mean we keep collecting it. We care if the data is interesting. If the data fits snugly into an equation, we have figured out the data. There is no mystery left with the data anymore then we should not care about it. It is with figuring out the various "Whys" of the data that we can find true insights. We should only care when the data puzzles us.
Where do insights come from?
Not from no data. That double negative is just meant to express the fact that there is no accepted answer. Data helps and some data is better than no data. But where we should draw the line is highly debatable. The storage costs per gigabyte reducing by a factor of 6 has meant that we can safely store information without having to worry about cost overhead but it is in the utility of this data that we are confused. Big, huge quantities of data are not necessarily high yield in terms of insight. And it is here that having a lean approach towards storage can certainly help.
In conclusion
Store the data to figure it out. Once it fits, its time to move on to another data set. It is in these attitudes of exploration that we can truly drive insights or at least a mindset that helps achieve insights.
Riffed off of and thought credits to
http://techcrunch.com/2012/11/25/the-big-data-fallacy-data-%E2%89%A0-information-%E2%89%A0-insights/
Having known this, why would we think that more data would help bridge the gap between data and information?
That is in essence the fallacy of big data. Big data will expand the range of hypotheses, it will also extend the range of the search space. Nothing gets easier with the addition of more data, especially when you are in the business of data exploration. The main job of a data scientist is to find possible hypotheses that describes what is happening with the data. The data does not lie, or at least lies less than the human :)
Should we collect more data? Should we care?
Just because data has grown exponentially, does not mean we keep collecting it. We care if the data is interesting. If the data fits snugly into an equation, we have figured out the data. There is no mystery left with the data anymore then we should not care about it. It is with figuring out the various "Whys" of the data that we can find true insights. We should only care when the data puzzles us.
Where do insights come from?
Not from no data. That double negative is just meant to express the fact that there is no accepted answer. Data helps and some data is better than no data. But where we should draw the line is highly debatable. The storage costs per gigabyte reducing by a factor of 6 has meant that we can safely store information without having to worry about cost overhead but it is in the utility of this data that we are confused. Big, huge quantities of data are not necessarily high yield in terms of insight. And it is here that having a lean approach towards storage can certainly help.
In conclusion
Store the data to figure it out. Once it fits, its time to move on to another data set. It is in these attitudes of exploration that we can truly drive insights or at least a mindset that helps achieve insights.
Riffed off of and thought credits to
http://techcrunch.com/2012/11/25/the-big-data-fallacy-data-%E2%89%A0-information-%E2%89%A0-insights/
Subscribe to:
Post Comments (Atom)
No comments:
Post a Comment