In many industries has
become the buzzword. Scarcely a discussion seems to happen without touching
upon the perceived advantages of big data. Firms seem to outcompeting with each
other in collecting reams of data. The question however is the effectiveness of
this data. The firm’s outcomes are determined primarily by the utilization of
big data rather than collection of data per se. Ferreting out big data is a
challenging task.
Although the big data
presents a data set that shows 10X or 100X in relation to existing mechanisms,
it does not necessarily convey 10X or 100X worth of increase in insight. While
the implication of big data is that quantity is paramount, the returns
generated do not match the quantity of data generated. Big data too is subject
to diminishing returns.
Experts point out, it is
not per se the data that should be big, but the primary factor that counts is
the diversity of data. Even if datasets may be small, the amount of richness
they provide when they are diverse, they have the potential to create an output
subject to increasing returns.
As Hal Varian points out,
the statistical limit of accuracy with which one can measure data as it
increases is the square root of its sample size. To obtain an estimate that is
twice as good as current estimates, a firm needs four times more data than the
current levels of data. For a company
like Google, the demand side ( query traffic) is growing at around 40% per year
( in other words doubles every 2-2.5 years), the supply side (increasing the
relevance of search) is experiencing the law of diminishing returns. Big data was crucial to Google in the initial
years to mine the user behavior and the links used but the passage of time
necessitated better algorithms to mine the data than data alone per se
Source: Nicholas Carr, “The diminishing returns on data”, Posted August 14, 2009, Accessed
on March 31, 2012, http://www.roughtype.com/archives/2009/08/the_diminishing.php; In Defense Of Small Data.
By: Upbin, Bruce, Forbes.com, 3/30/2012; Nathan Eddy, “Diving into Data Pool
Demands Solid Business Strategy”, Posted April 12, 2012, Accessed April 15,
2012, http://slashdot.org/topic/bi/diving-into-the-data-pool-business-strategy/;
Comments
Post a Comment