How big MNC’s like Google, Facebook, Instagram, etc store, manages, and manipulate Thousands of Terabytes of data with High Speed and High Efficiency?
BIGDATA:- It is problem for companies to manage more than the capacity of data which is increasing day by day.
Let’s understand in my words …..
Can you ever think about that every day may be whole world uploading photos or video, making texts to each other, uploading lots of data on drive which causes increasing a data in MNC like Facebook, Google, etc
which creates Bigdata.
In this Bigdata there are two problems that the company is facing which are volume and velocity. It means the capacity of storage and i/o speed for storing data respectively.
According to research, there are more than 500 terabyte data come to Facebook each day which is increasing day by day.
Like every time whenever the problem is human to try to find its solution. So, for that, some genius creates a new concept known as a distributed system.
In this concept, they manage data by using other systems as their own as virtually. It works like For Example If the system needs to store 100TB of data but the system doesn’t have that much capacity they will cut out data into several parts and they make it into blocks and they send it into that’s other systems which are known as their nodes or slaves.
Let’s explain in the diagram.
If Bigdata is coming towards master which is connected with 4 nodes having 25TB per slave capacity .then master virtually having 100TB of the capacity of storage then the problem of volume is sorted here .then velocity is coming. when it wants to transfer than 100TB of data it will transfer much faster because of the slave having its own i/o capacity of storage then the problem of velocity is also sorted out here.