While we know that Google runs millions of servers, and while we know that the company aims to eventually store the world’s information in its data centers, there is little information how many pages are stored and how much storage space these pages require. With its latest search engine update, Caffeine, the company revealed a few more bits of information: To match Google’s current index, you would need 625,000 160 GB iPods. Each month, Google crawls 85 billion new documents.

What does it take to index the world’s information? It’s clear that you would need massive amounts of storage capacity, beyond the imagination of the average computer user. But how much exactly?

Google released some information as part of its Caffeine search index release, which is a substantial upgrade to Google’s former silo index, which layered different types of information such as text, images and video. Caffeine does not differentiate between those information types anymore and treats them equally. It analyzes smaller portions of the web at a faster, continuous pace. New pages are added immediately to the index. With the silo structure, there was a “significant” delay between the time a page was found and it was added to the index. The result: Google now claims that its search results are “fresher” than ever before.

There were interesting notes about the size of Google’s search index as well. Google said that the index currently has a size of almost 100,000,000 GB , which translates into roughly 100,000 TB or 100 PB (petabytes). The storage capacity is equal, according to Google, to about 625,000 of the largest iPods. If you stacked them together, they would stretch out to a length of more than 40 miles. Each day, Google adds “hundreds of thousands of gigabytes per day” to its index. If you were to print all those pages that are processed by Caffeine on paper (“hundreds of thousands of pages in parallel”), the pile would grow three miles taller every second, the company said.

If you consider the average thickness of a standard piece of paper of 0.081 mm, then we know that there are about 314 pages in each inch or 3756 pages in each foot. Since a mile has 5286 feet, it means that Google’s infrastructure crawls about 60 million new pages every second. That number translates into more than 85 billion documents every month.

Google is believed to run 2% of the world’s installed server base.

You can leave a response, or trackback from your own site.