"Big Data", and "Cloud"


What is “Big Data”?

Data captured on web, Data available in firms as part of the data capture of the customer, social networking sites.

Big Data in four dimensions:

1. Volume : Peta -bytes of data
2. Velocity: Tweeter , Face book and will have per sec n you will get tonnes of data.
3. Variety: Images, documents, graphs , structured/un-structured data
4. Veracity: Information which can be trust-ble/analyzable
Artwork: Tamar Cohen, Happy Motoring, 2010, silk screen on vintage road map, 26″ x 18

This data can be analyzed to get following trends/use:

  • Revenues
  • Business generation
  • Revenue leakage
  • Research and development on new area, to solve the existing problem

How do you program/Analyze the data?

Mostly the data is un-structured, and we can not use SQL. So we have to “No-SQL”. 
 
Some of the API”s /Web apps used
1. MapReduce
2. Hadoop
3. Cassandra
4. Dynamo
5. BigTable obviously.
6. CouchDB etc

Where do you store?

For storage, you need so many servers/cluster, which is very impractical  as you have build them to test. Instead you can go for the “Cloud”

Why you want analyze?

Analyze and come-up up with use-case, so that you can analyze and peta-bytes of data. Also have ROI plan

Big-Bang approach?

Build a small POC/project, before you go on large scale implementations.

What skills we need?

Organization to be geared up and build on essential skills. Need build to small team to kick-start.
1. Managers: Should able to come with 5 year plan and convince management
2. Developers/Architecture : To be familiar with MapReduce, Hadoop, and cloud principles, Data Analytic areas.
Hope the above stuff is useful.. Please feedback 
Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s