Google Search Architecture (part-2)



Thanks for response on the first article on the “Google Search Architecture”. Even this is quite old topic, but enjoyed your comments on blogspot/wordpress blogs.  Also this motivates me to write more of this kind.

First part can be found in http://blog-bhaskaruni.blogspot.in/2012/11/google-search-architecture.html

We have seen some components MapReduce, GFS (Google File System) API/software abstractions.
Google published the architecture. And based on this major software firms “Facebook” and others use same architecture. 

Major Modifications in Google Architecture
1. MapReduce is time consuming and major revamp in architecture is made. This  replaced with  Caffeine. 

2. GFS is also replaced by Colossus (means large statue), this is used across the all the “Google web Services” . GFS was using batch operations logic. Colossus is built for “Real time services”[1] . Colossus is also used for “Google Cloud Service” just like Amazon “S3”


Incarnation of Google Architecture

Hadoop cluster
Hadoop Architecture
Hadoop Logo
Hadoop Apache foundation

1.  Hadoop  and  its “Hadoop Distributed File System” (HDFS), an open source Java product similar to GFS.
 2. Hadoop is open source implementation of the GFS

Hadoop was created by Doug Cutting and Michael J. Cafarella.Doug, who was working at Yahoo at the time, named it after his son’s toy elephant. It was originally developed to support distribution for the Nutch search engine project.
Besides Facebook and Yahoo!, many other organizations are using Hadoop to run large distributed computations.  

References:
1. Google Remakes Online Empire With ‘Colossus’ (CADE METZ)
2. Simplified Data Processing 
3. Todd Hoof’s (Google Architecture)
4. Hadoop on wikipedia

Advertisements

Leave a Reply

Please log in using one of these methods to post your comment:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s