A great deal has been written about the ideas, logic and motivations behind Facebook’s Open Compute Project. Last week I spoke with Julie Bort at Business Insider, who wrote a fantastic piece with the headline “How Facebook is eating the $140 billion hardware market,” which covered the backstory from many of the players along with forecasting the industry impact of OCP.
Through the fullness of time it’s easy to see the Open Compute Project portrayed as “eating the $140 billion hardware market.” However, 6 years ago, our original goal was far more modest: improving every Facebook user’s experience with abundant capacity and managing the costs of growth. Controlling our destiny by relentlessly focusing on creating some of the most energy-efficient, advanced data centers in the world, Facebook saved over $2 billion in infrastructure costs over the past few years.
When we — it’s self-aggrandizing for any one individual and even more so for me to take sole credit for an idea in a company of brilliant collaborators — concocted the Open Compute Project, the world was a different place. It was circa 2009, and Facebook’s infrastructure organization had its foot so hard on the gas pedal it blew through the floor! User and usage growth were accelerating rapidly. There were days — even weeks — when the site was literally teetering out of capacity, causing slowdowns or instability. This was bad — not having capacity meant no headroom to launch new features and made some engineers fear implementing changes. Most importantly, we didn’t want to suffer an outage and disappoint the people who trust Facebook to be an always available resource for connecting with friends and family around the world. Forecasting capacity demands for a fast growing (and rapidly changing) Internet service was challenging to say the least, and 6 years later remains more a work of art than science.
We had multiple projects underway to keep up with growth, including the particularly successful HipHop for PHP led by Haiping Zhao, Keith Adams and a team of truly exceptional engineers. By using HPHP as a source-to-source compiler, the PHP code that Facebook is built on got translated into C++, compiled into a binary and run as an executable. When HPHP was open sourced in 2010, performance increases of up to 6x had been observed across Facebook’s Web servers. HPHP had a spectacular impact on Facebook’s ability to scale capacity and manage infrastructure costs.
(Well after OCP was launched, in 2013, HPHP was replaced by HHVM, a process virtual machine based on just-in-time (JIT) compilation. HHVM enabled further quantum leaps in application performance.)
Despite all these performance gains, we were still plagued by the inflexibility of server and datacenter manufacturers not meeting our requirements. These limitations led to the spark that became the Open Compute Project.
OCP was inspired by three fundamental observations:
- Designing the datacenter, server & network hardware and software application stack in unison would lead to massive efficiency opportunities. Google was the first significant Internet company to innovate at this scale and we stood on the shoulders of their ideas to imagine a new path.
- No matter how hard we negotiated with our vendors every quarter, their prices would only change so much. We had to reengineer core elements to meet our needs and attain better cost savings.
- Everyone builds software using all kinds of weird techniques. Why not hardware? We believed other companies had similar challenges and by sharing our inventions with the world, it would inspire other companies to collaborate.
It’s this relentless focus on optimization and clever problem solving that enabled apex Internet services such as Facebook to scale rapidly. By focusing on the essence of scale, ignoring the clamor of vendors and cloud providers, and making bold bets on technology that has never existed, other companies can reap similar rewards.
The same old solutionism can’t be applied to unanticipated problems. The Open Compute Project is just the beginning of a broader movement, one that disturbs the conventional business model of the hardware industry. Fledgling ideas that today sound audacious or risky are the reason In Sik Rhee and I started Vertex Ventures. There are few people courageous enough to spend time exploring non-obvious ideas that can change multi-hundred billion-dollar industries. To us, those ideas are simply good business.