Facebook, Microsoft Want Improved Infrastructure
Article By : Rick Merritt
Advances in optical and flash technology on the wishlist of the major players
SAN JOSE, Calif. – Web giants called for significant shifts in networking, storage and software at the Open Compute Summit here to keep their heads above a rising flood of data.
Facebook said it will need switch ASICs with optical interfaces within three years for its next big network upgrade. Microsoft announced a new low-level interface for NAND flash drives to make better use of storage. And the Open Compute Project (OCP) struck a partnership with the Linux Foundation to plug holes integrating and testing systems and software together.
Interestingly, little was said about the adoption of machine learning, the latest force driving the data explosion. Security, however, is a rising focus with increasing support for a root-of-trust module announced last year and a broader security project just getting off the ground.
More than 3,000 people registered for the ninth annual event sponsored by OCP and launched by Facebook in 2011. The group now has 175 members that have created 375 specifications related to open hardware for data centers.
In recent years, OCP has attracted Google, China’s Alibaba and Tencent and carriers such as AT&T and Verizon. However, vendors say Web giants show little unity in their technical demands.
Indeed, even OCP’s most vocal members — Facebook and Microsoft — showed separate flavors of server and switch designs here. The largest data center operator, Amazon, continues to quietly go its own way.
Facebook patches burgeoning nets with cables
A Facebook engineer described rack-scale switches it was forced to lash together with cables last year when network demands overwhelmed its gear from third-party vendors.
To keep up with traffic, the company expanded some of its data center regions from one or two buildings to as many as six buildings in some cases. To link the centers together it needed switches with as much as three times the 512 100-Gbit/second ports of high-end products.
“We were already using the largest switches available and there was nothing else we could buy readily,” said Sree Sankar, a Facebook project manager.
Her team wound up stacking dozens of its Wedge 100 switches in a rack, tied together using a backplane of cables and patch panels. The result was Fabric Aggregator, a 100 Tbit/s switch that went from concept to production in five months with four configurations all running Facebook’s FBOSS networking software.
To make the next big leap to 400G interfaces about two years from now, Facebook is calling for work to bring optical interfaces directly to merchant switch ASICs.
It’s a big and technically difficult leap that vendors thought was still many years ahead. At a recent optical networking event, a group of vendors led by Microsoft’s data center experts released their first spec for a half-step in that direction, bringing optics to a motherboard module.
Sankar said the COBO proposal won’t fit Facebook’s need for 400G links on all switches from the top-of-rack on up. At 400G, she is concerned engineers will not be able to cool separate ASICs and optical modules.
In a separate talk, networking veteran Andy Bechtolsheim joined Facebook in calling for work on co-packaged ASICs and optics at 400G. Such designs could slash power consumption 30 percent, he said.
Bechtolsheim proposed using 2.5D chip stacks with switch ASICs and optical modules riding next to each other on a silicon interposer. He noted technical and business challenges creating an interface that’s both low power and reliable enough to survive handling. The design also would redefine who does what in the networking supply chain.
“It’s a multi-year project, so don’t expect this anytime soon. But let’s get started now,” he said.
Bechtolsheim also called for engineers to rally around 400G multi-source agreements (MSAs) for the next generation of pluggable optical modules.
Separately, Intel said its 400G module based on the CWDM8 MSA will sample later this year for distances up to 10 km. Several vendors are expected to ship 400G pluggable modules in time for adoption by as many as half a million ports in 2019.
Bechtolsheim called for a 2.5D stack (top) and showed research by Luxtera on an switch board (below).Source: OCP
Microsoft wants more control of flash
Microsoft announced Project Denali, creating a so-called pBLK interface. It would let Web giants control on CPUs, SoCs or FPGAs functions on solid state drives ranging from garbage collection and wear leveling to where applications run and how they are accelerated.
The project is essentially a compromise, stepping back from a more extreme call for SSDs without firmware. Under Denali, SSD vendors would use controllers to verify and manage media and power failures. Broadcom, Intel, Marvell, Micron, Samsung and SK Hynix expressed support for the new interface, which was shown on a terabyte M.2 drive from Cnex Labs.
While Web giants only use about a sixth of all flash chips, they still spend billions on SSDs, “so we want to drive efficiency,” said Kushagra Vaid, general manager of Microsoft’s cloud hardware. “We’re not exposing the performance available in flash,” he said, noting the interface also is geared for emerging storage-class memories.
Drives using the new interface will be running in production networks this year, Vaid said, inviting feedback on the new spec. Facebook, Google and Toshiba are expected to back the project eventually.
“With conventional standards groups, something like this could take years, but I hope we can get this done in a couple months,” said Bechtolsheim, who sits on the OCP board.
Separately, Vaid gave an update on Project Cerberus, an MCU-based root-of-trust. The CPU- and OS-agnostic effort has gained backing from Broadcom, Cavum, Intel, NXP, Mellanox and Qualcomm.
The OCP started a broader security initiative about three weeks ago co-chaired by Google and Microsoft, but few details were released on it. The effort aims to embrace threats that extend across the supply chain and the full spectrum of data center products through their lifetime.
Denali would let cloud operators embedded accelerators to speed operations on SSDs.Source: OCP
Partnership takes a deep dive on software
Amid an explosion of open source software projects, the Linux Foundation and OCP struck a partnership. They will collaborate to automate and test software loaded on open systems and explore new projects.
“We have to provide tests that cover hardware and software like OPNFV,” said Arpit Joshipura, general manager of networking at the Linux Foundation, citing his group’s work on open code for network virtualization.
An explosion of rival projects in open networking software is raising concerns about interoperability. For example, AT&T provided code to start dNOS, Google contributed code for the recently announced Stratum project, Facebook backs its own FBOSS and more projects are in the pipeline, he added.
The efforts are among more than 20 open source projects just for networking currently active at the Linux Foundation. The group also has expanded lately into areas as diverse as software for IoT devices, storage and blockchain.
It’s too early to tell whether the OCP collaboration will try to enforce interoperability through certification programs. For now, the motivation seems to come from carriers demanding testing of fully loaded network systems before they are delivered.
Among other news at the event, Facebook showed its Big Basin GPU server updated with eight Nvidia Volta chips, drawing a whopping 2.5kW. A Facebook engineer said the company tries to train many of its neural networking models on its vast supply of CPUs during times when they are relatively idle.
The Linux Foundation is managing an explosion of open source projects, especially in networking. Source: OCP
— Rick Merritt, Silicon Valley Bureau Chief, EE Times
Subscribe to Newsletter
Test Qr code text s ss
