142

Good afternoon, everybody.My name is Allan Cantle, and I'm the technical lead for the OCP High Performance Compute Subproject, and I'm going to talk about HPC, the next generation, a sustainable energy-centric first innovation.So basically I'm going to give you a quick overview of the thinking behind what we're doing here.

So firstly, if we pay attention to what's going on in our industry right now, we see that we're getting proprietary compute architectures coming from every direction.It's very prolific, like these ones here, NVIDIA's DGX, Google's TPU, Tesla Dojo, and even the startups.And they're all very proprietary and not accessible to the rest of the industry.The rest of the industry, we've just got regular old servers based on the IBM PC from 1981.So clearly we need these.

What's really driving this computing architecture change?Well, first of all, we have the drive of sustainability.We want ever-increasing performance.Naturally, that means we have to focus on energy anyway, and we now need to double down on that area.But we also need to start looking into energy recovery and reuse.

Specifically in the area of emission reduction to achieve net zero, there are three scopes.We all know about the scope two, the actual energy that our computing architecture consumes.But you've also got scope one, which is the facilities, energy used to power the buildings, et cetera.And scope three is the big one that people are paying most attention to right now, is the embodied carbon side of things as well, in terms of the actual energy to make the computing components when we build the computers.So that's a big factor where we want-- basically, we renew our computers every three to five years and throw the old ones away.So that's a big issue from an embodied carbon perspective.

And then we've got this composability, domain-specific architecture composability.We want to configure these machines, the components, in different ways for particular applications.

So I've put the four categories here as I see them-- fixed architecture, the likes of the Apple iPhone chip, that you've got a piece of silicon rammed full of heterogeneous architecture all in one piece of silicon.It's very fixed.It's not composable.But it's a very large market.At the other, we've got our classic server, which is a very pluggable physical composability.Everything's pluggable-- CPU, memory, power, fans, everything's pluggable, which is great.But the resources can get stranded once it's been configured.And we've now got what I've coined hard physical composability with these chiplets.We're basically cramming the entire server into the package of what used to be just the CPU.So that's hard physical composability.And at the opposite end of the spectrum, we've got dynamic software composability, where we're separating out all of the separate components-- compute, storage, memory, I/O.And then we're recombining these as the application demands.And then we don't get stranded resources.When we look a bit closer at these, the chiplet approach is all about energy efficiency.This data movement is the biggest power consumer.And so bringing these components closer together in this hyper-converged architecture allows us to save a bunch of energy.But on the opposite end of the scale, this dynamic software composability is very energy inefficient.It's basically moving all the pieces apart with these disaggregated racks.So that presents a challenge.The good news is we've got standardization going on in the disaggregation field, driven by CXL in the Ultra Ethernet Consortium.And in the chiplet world, we've got standardization with OCP's ODSA subproject and also the UCIE consortium that Intel kicked off a few years ago.So we've got some standardization going on.So that's excellent.

But we really need to bring these two worlds together.You would think that the hardware engineers making these chiplet packages don't talk to the software engineers who want this disaggregated composability.And you'd be absolutely right.So we need to try and get everyone back in the same room, especially from an energy efficiency perspective.

To put it succinctly, Google, when they announced their TPU-v4 earlier this year, said system architecture matters more than microarchitecture.And like I said, with these proprietary architectures, you can see that the companies are actually doing this themselves, not just Google.

And finally, really, we want to drive down the cost.And if you look at the cost of these machines, they're very proprietary.They're getting very, very expensive.We need an open architecture to do that.And that's what we're all here in OCP we're trying to achieve.And in the Open Compute High Performance Compute project, we're looking at compute from the ground up and saying, how do we make an energy efficient solution if we had a clean sheet of paper?So this is pretty radical, what we're doing.

So this is an abstract view of computing as I see it from a domain-specific architecture template.Basically you have a data plane continuum.You have all of these different sorts of processors and accelerators attached.But you've got the cloud, the edge, and the intelligent IoT.And but we should treat this as one computer.And we should put the software and the compute wherever is most efficient from an energy perspective.

If we look at an individual high-performance processor, what we really need to do-- and we know about this with the composable memory systems we've been working on-- is we want to couple as much memory as possible with as much bandwidth and as low energy as possible to these high-performance processors with this local memory interface.But we also want to be able to share that memory with others when either we don't need it or if we are just sharing data that is in that memory between processors.So we need a high-bandwidth memory pooling data plane.So that's our fundamental building block, so to speak, for our domain-specific architecture.

If we map that onto my previous diagram, we see we've got one processor with its own local memory.And we have this fabric, which makes that memory accessible to all the other processors in the network.So it's pretty straightforward to say you could step and repeat that.You have this building-- this Lego block building block.And you can step and repeat that across the cloud and the edge.Wouldn't it be nice if that was just one single building block?And that's what we're trying to do in the OCP High-Performance Compute Subproject.

So I've got a little animation here.I can't get into details.But this is the module.You have media plugged in.That's E3.S modules.And this is a water-cooled jacket.We liquid-cooled-- direct liquid-cooled this.And inside, we have-- it's similar to the OAM, but we have a backplane of connectors.We're not mezzanine.There is no motherboard.We connect everything together with topology connectors and cabling.As you can see in many servers today.And we have the big, powerful processor, just like in an OAM module.And that can take many forms-- some CPU, GPU, AI, et cetera-- including switches and optics and even any startup that makes a silicon.They can actually basically just make this module.They don't have to make an entire data center like Graphcore and SambaNova have had to do.So they could build this and be part of this ecosystem.

So just putting this module back together again, reattaching.So we have conductively cooled and water-cooled.We have a mini-immersion, and the water comes in and out there.Reassemble the media.

We then plug it into a wall.And we call this a wall of compute.So the wall will carry the water and the 48-volt power.So in your data center, you would have no 19-inch racks.You would just have walls with carrying the water and power.And you would blind-make plug these modules in, as can be seen here.So on the right now, you see that we've got a wall constructed with just no modules populated.And we would be able to blind-make populate those modules, as shown here, into the full grid.Once you've got that, we can construct different nodes.So if you know the frontier node with the AMD CPU and GPUs, the 4 to 1 ratio, we can actually make that ratio.And we can cable those modules together to the exact topology of frontier.Likewise, if you wanted to make a Grace Hopper type one with 7 times PCI bandwidth between a CPU and GPU, you could connect those together directly, again, with cables between two modules.So you could emulate that.If you want to construct the older supercomputer, IBM and NVIDIA Volta, that's a summit node.That was a 3 to 1 ratio with a different interconnect.Remember, these nodes in the supercomputers are fixed topology, whereas you can rewire these to any topology.If you wanted to make an AI-optimized GPT-3, you could just use a bunch of AI chips.Or maybe if you wanted a disaggregated pool optically connected, you could use an FPGA and some optical modules.Or finally, Burger King, you can compose it your way.So hopefully that gives you an idea of the flexibility of what we can potentially do here.

So what are the sustainability aspects of this, then?Well, first of all, we've got reduced scope one.So we're bringing cold water in, and we control the flow over each module so that we have 65 degrees C hot water return.So that can actually be used for energy reuse.About 90% of the energy of our computers is exhausted as heat, and so there's tremendous opportunities there.This network can be used on the edge as well as in the data center.So if you imagine a telecom box on the side of the road, that could have maybe 20 modules being fed with water from a city heat network, for instance.It could feed the hot water in there and get the power there as well.From a scope two perspective, the domain-specific architecture, the ability to wire this exactly to your application needs between these different nodes, really gets you some tremendous energy efficiency gains.Also, everything's much more tightly coupled, so we have a lot of opportunity to have less data movement and save energy on the data movement.And the delivery of the power, which we all know is getting higher and higher, is far more efficient as well in terms of losses.So that's how we lower the power reduction with this architecture.Finally, in scope three indirect emissions, we're using less raw materials because this is a much, much tighter module.And these raw materials are actually very reusable.The building blocks are the same.If you decide to upgrade a CPU, you could reuse the metal infrastructure, et cetera, and for a different type of module because it's a common footprint for everything.So that's that.

A couple of other areas as well is to begin with, we've obviously got a long way to go, but to begin with, this can be used as a very good R&D type platform for people experimenting with different domain-specific topologies for different applications.So I've just shown a little Bench Top example here with four modules.You may have an AMD MI300, an FPGA, an Infabrica switch chip, an Intel Agilex FPGA, some Samsung CXL memories or Micron or NVMe SSDs, et cetera.And you would wire that together to your specific needs.And that could be copper or near-packaged optical or even potentially co-packaged optical we've been thinking about as well in this architecture.So this could give you an actual nice migration path rather than just a complete switch over from copper to co-packaged optics.Also we're looking from a chiplet in a photonic enablement platform as well.

So if you look at what's going on in these chipletized packages, they are getting huge now.The 78.5, I believe, is maybe the MI300 or one of the other chips.And you start to realize that the actual chip is not much bigger than the module.And these modules really only carry power and high-speed interconnect.So you sort of think, well, why not cut out the middleman and just actually-- certainly this could work with CoWoS-- get rid of the substrate package and actually make the module itself.That can give you signal integrity and power integrity benefits as well.It would also allow you to move some of our chiplets around and actually maybe-- you might be able to see some small corner dots.Some of the smaller devices could be directly behind the connectors.And maybe you could have backside optics modules for actually doing a co-packaged optics implementation.

So in summary, we've got an opportunity to redefine open computing architecture that really can deliver on sustainability, the easy construction of any domain-specific architecture.And we're going to democratize the industry by lowering the barriers to entry for startups, et cetera, and reducing the costs for all of us.And we have an opportunity to create a CXL and a chiplet enablement platform-- and photonics, actually.And HPCM really does bring multiple sustainability benefits-- reduced embodied carbon with less raw materials and improved reusability, improved power and signal integrity and efficiency, seamless silicon photonic enablement, power from copper to NPO and CPO, and a simple and effective heat energy recovery and reuse.

So my call to action for you all is we are still in concept stage now.We actually have a commercial interest.And we're beginning to actually turn this into a real product.So the wheels are beginning to turn at a much faster clip.We do need a good bunch of support.We need broad industry cross-discipline collaboration.This is a challenge, is cross-discipline, from thermal to power to electrical, mechanical, all of them.And we're going to be targeting some Bench Top examples.We'd like to work with people on that.So come and join us in the OCP HPC subproject.I've listed some details there of how to find us.Thanks very much.