John Wilkes says that joining Google was like swallowing the red pill in The Matrix.
“I’m an old guy. Megabytes were big things,” Wilkes says, in describing the experience. “But when I came to Google, I had to add another three zeros to all my numbers.” Google is a place, he explains, where someone might receive an emergency alert because a system that stores data is down to its last few petabytes of space. In other words, billions of megabytes can flood a fleet of Google machines in a matter of hours.
Then, as he was still trying to wrap his head around the enormity of Google’s data-center empire, John Wilkes went to work on the software system that orchestrates the whole thing.
This software system is called Borg, and it’s one of the best-kept secrets of Google’s rapid evolution into the most dominant force on the web. Wilkes won’t even call it Borg. “I prefer to call it the system that will not be named,” he says. But he will tell us that Google has been using the system for a good nine or 10 years and that he and his team are now building a new version of the tool, codenamed Omega.
The Borg moniker is only appropriate. Google’s system provides a central brain for controlling tasks across the company’s data centers. Rather than building a separate cluster of servers for each software system — one for Google Search, one for Gmail, one for Google Maps, etc. — Google can erect a cluster that does several different types of work at the same time. All this work is divided into tiny tasks, and Borg sends these tasks wherever it can find free computing resources, such as processing power or computer memory or storage space.
source:
http://www.wired.com/wiredenterprise/2013/03/google-borg-twitter-mesos/all/.
Author