Sunday, October 24, 2010

Big Data is less about Size and more about Value

Big Data is the talk of the town, specially in the Silicon Valley start-ups. As far as I know Big Data is around for a long time at least for last 5 years. Companies like eBay and Amazon had massive amounts of user's data that they were using for their advantage. Credit card companies have been mining the huge customer's data for their competitive advantage. So what's up with Big Data now, that every small startups as well as Big enterprise companies want to jump on this. With my background in data warehouse and business intelligence and little research of what some of these new startups doing, its easy to summarize the reason for the big push for Big Data:

  •  Big data is democratized by advances in cloud computing - Start ups do not need to have expensive hardware and software to crunch large amount of data. They can use cloud provide and get large number of clusters running in a matter of days with low cost. Open source software like Hadoop and HBase paved the way for creating applications cheaply. Flightcaster is an excellent example of a startup which processes massive amount of data to predict Flight delays.
  • Big data is less about the size and more about the value - Matured companies like eBay, Facebook and Google showed the world, how to unlock the potential of massive data they create and store. Once infrastructure becomes easy to setup, everyone wants to get the real value out of that big data. Trending Topics is a good example of the project undertaken by an individual, that process such a huge amount of data.  AdKnowledge uses predictive analytics on big data to connect advertisers to the consumers. Cooliris uses hadoop and coverts your web browser into a place that is cinematic and fast. You can browse photo and videos on web as well as from your local computer. Datameer provides big data analytic solutions for big businesses. Deepdyve is the big data mining solution for large scientific community for journal and article search. Enormo generates and filter real state listings based on huge data. Gumgum does image analytics to provide In-image advertising and plenty more... These start ups are all about getting unique value out of big data.
Going forward, it will be interesting to watch and learn from how big data apps in the cloud can be commercialized. Hope some of my "big data" professional friends will share their thoughts by commenting here.

No comments:

Post a Comment

Make Everyone Smile

Hey there! Just wanted to let you know that today is officially National 'Make Everyone Smile' Day! So, consider yourself officially...