Enhancing efficiency with 48V solutions: topologies, benefits and applications | Infineon
In the last eight years the computation capabilities has increased by 1000 times in the latest chip. That means from given numbers has moved from 20 teraflops to 20,000 teraflops or let's say 20 petaflops, which is really revolutional. This is the podcast for engineers, the podcast you just have to listen to if you're interested in what's going on in the semiconductor market. Today, we are diving deeper into the world of We power AI. We're going to talk about the benefits of 48 volt architectures and the different topologies. I'm joined today by Doctor Roberto Rizzolatti, he's a principal engineer at Infineon.
Roberto, welcome. Hi Kelsey, thank you. Happy to be here with you. Looking forward to deep dive on the challenges in powering AI from 48 volt.
Yeah, me too. So today, as you mentioned, we're diving a little bit deeper into the We Power AI story. We're going to talk about how different architectures support these increasing demands in artificial intelligence and the increasing demands of data centers.
So Roberto you're going to give us a clearer picture on both architectures and topologies, both the physical and electrical architectures. But let's start with the AI trend. So we've talked in previous episodes about the increasing demand for computation power.
How much computation power are we actually talking about when we talk about AI chips? It's really a revolutional trend in the last years. Just you can imagine that in the last eight years the computation capabilities has increased by 1000 times in the latest chip. That means from given numbers has moved from 20 teraflops to 20,000 teraflops, or let's say 20 petaflops, which is really revolutional. So this is clearly translating into challenges for powering these chip because the computation capability has increased, but also the power consumption has increased. This is really important to consider. And how can 48 volt architecture help with this problem? It's actually very simple.
From one perspective are I2 losses. So the power goes into this board from bus rack. So if you have a reduction of four times of current, you have 60 times less I2 losses, very simple calculations. Moreover, the filter design is pretty much more simple from 48 volt and in general this turns out to be a lower total cost of ownership for our end customer. So 48 volt is really important, and it is becoming a standard.
So it all sounds really great. But are there any bottlenecks with the 48 volt architecture? Any challenges? Yes, clearly. From 48 volt you need to supply the ASIC in a two stage approach. You have a first stage.
You have a second stage and then you have challenges at any level. So let's start from the second stage which is that the most important part. So the second stage is called VRM voltage regulated module. There are several challenges which are enabling AI for the VRM. So the main challenge is you need to take on is the current density primarily because you need to keep up with the increase of current, but also you need to at the same time keep high efficiency because you are increasing the power level, so you will need to keep same thermal performance increase the actual efficiency. Not only that, transient is also very important because the transient performance are actually limiting the system.
You will need to place a lot of capacitor otherwise. So you need also to take care about the transient performance at the VRM level. In general, you can address these challenges in different ways.
The easiest one is to increase the switching frequency, lower the input voltage to get the better figure of merit for the device. Then let's move on the first stage, which is called intermediate bus converter IBC. So here we will need pretty much to reduce the losses of the intrinsic topology we have nowadays. For example, we have a state of the art which is LLC converter. Let's talk about, for example regulated DCX converter. There are ways to reduce the current stress in both the silicon and magnetic and we came actually with some ideas and also products out to reduce those losses and we´ll deep dive later on.
We also very important and we are really good in that improving the figure of merit for all the devices. When it comes to 40 volt IBC we talk about 15 volt recently delivered 25 volt, 30 volt, 60 volt, 80 volt and 100 volt OptiMOSTM FETs.. In this kind of solution you might find attractive 25 volt when it comes to 8:1 unregulated, 40 volt when it comes to 4:1 unregulated, then for all 80 volt for the primary side. Now the magnetic also we are really we are also focused in this field because we are looking into all the system meaning that we are also proposing new magnetic structure to improve the performance of the system and actually improve the actual performance of our devices.
Meaning is very important to have a structure, we try improving the current flow, meaning reducing also the RMS current within the magnetic and into the devices. Okay. so you painted a picture of 48 volt power architecture there. What types do we actually currently see on the market? Yeah. There are actually main two architecture currently the market. So we have identified two main category.
One is an accelerator card basically where you you attach the substrate of your ASIC and this is powered from 48 volt. Let's give a rough description how look like this board. Basically you will have the connector which are normally metallic connector, which are, connected actually to your universal baseboard. So from there you get all the ISB signal but also you get power, you get the 48 volt rail. In those boards since you don't need any type of other regulations, normally use an unregulated converter first stage of which is actually very efficient and very power density.
And on the second stage you have a VRM. What are the challenges? So there are two levels one is mechanical because you have a very tight environment given by the thermal constraints, meaning that you cannot have, you cannot exceed a certain tight for the component, but also the footprint is very limited where you place the foot, the components you place on the bottom, you place on the top, these are really challenges, but also electrically, you need to place the 48 volt and the IBC somewhere in your system, but you need to place very close to your ASIC the VRM because the longer is the distance, the higher are the I2 losses, but also the poorer is going to be the transient performance. So there are tradeoffs you need to consider when you design a power supply in these cards. But also you need to consider that the trend is also moving into a bigger ASIC. Computation is going up but also the size of these ASIC is increasing, meaning that if you stick on these given sides, then you have a less space for higher power.
So those are also the challenges. Indeed, we see different standards going on into the market. Then we can identify another standard, which is basically based on the power delivery board. So the main idea is to adopt, and this is very simple, to a legacy 12 volt system. So you have a front end of 12 volt card basically, which convert from 48 volt to 12 volt mechanically is much more recent in this environment, because you stick on the one you of your, of your switch AC and then you basically convert a 12 volt regulated bus because is not only needed for the VRM of your ASIC, but it's also needed for all the auxiliary, for the fan, also for the memories, and so on.
Electrically it looks like a board, and also mechanically is simply a board. You place this down solution module and that´s it, it´s actually very straightforward, cost oriented but the real limitation is that in the unregulated converter you will never achieve the efficiency you will achieve it in the regulated converter like in the first case. So that is really a kind of bottleneck when it comes to increasing the power level and then adopting such a method.
Okay. Yeah. You mentioned the word cost. So total cost of ownership for these two options. What does that look like? Definitely the first option is the efficient one because you are loading the high current path. Think about you need to cover a long distance into this data center board with a 12 volt rail.
Clearly you have four times the losses in those rails compared with the 40 volt solution, that is pretty much clear. But also you cannot go into different conversion ratio, for example you cannot explore 5:1, 6:1 and 8:1 because it's simply impossible to deliver very low voltage for high power current along the board, meaning that, the first one is for sure is definitely the best performing one. Okay. Yeah. So we've talked about IBC and VRM.
Can you and how they're both very, very important for enabling these high performance artificial intelligence data centers. Can you tell me a little bit more about the topologies for each? Yes, sure. So, in the second one, actually we we were focusing on the regulated converter, right, in the first one on the regulated converter. So you can see these two main categories. One is establish the regulation in this intermediate bus voltage. The second one the regulator one it doesn't.
So let's maybe go through the regulated one. Regulated is typically used when is required an intern bus regulated pretty much in this legacy 12 volt you need to adopt a legacy 12 volt you go for that. But the definitely is not the best fitting for AI Power and give you the answer is simply spoken because, you need to follow a voltage target.
You need an inductor. If you have an inductor your size depends on your output current and once this size depends on your output current, it´s difficult to keep up with the power density. However, we came up with the new solution within Infineon. So basically we will have different paradigms for this converter. So we can identify three categories.
One is a kind of buck converter, inductor-based, which cannot switch a very high frequency and all the current is processed by an output inductor. The voltage faced by the inductor is actually the entire input voltage, which is definitely not efficient. We proposed in the past but also academia and the other companies propose a switched capacitor based topology, which are more attractive because you step down the voltage in front of the inductor. so less voltage lower the frequency. You might pretend to switch at a lower frequency, meaning that you can also increase the power density eventually but also the efficiency.
There is also a third option which is a partial power flow. So think about you process the energy 80% through an unregulated converter and then through a regulated converter you process only 20% of the energy. Sounds efficient because you are basically having one converter which is not very efficient but is processing just 20% of the power, meaning you can increase power density, efficiency, and eventually you can also pretend to keep up with this high current trend. Okay, so that was the regulated side of things. What about the unregulated side? In the unregulated side we can identify actually different topologies. The main idea behind the unregulated is to divide the voltage and multiply the current.
What it means for the actual topologies? So, let's assume you come from an LLC running in a DCX operation. This is very good topology soft switched topology. You can pretend to switch to 1 MHz very good because you have a high frequency from one side, meaning you can shrink the size of your actual magnetic. On the other side, you have a low voltage operation on the output part where you have a high current, which is very good because you can use a 30 volt, 25 volt MOSFET.
However, there are also way to improve it, meaning you can use a switched capacitor based topology which are very fitting in a down solution. Basically placing this converter on the actual customer substrate and with that you can list the STC converter, very famous from Google but you can have the ZSC converter as Infineon has proposed, which is a zero voltage switching converter. And also very interesting is the third category hybrid switched capacitor converter.
So that is basically a combination between the first LLC and the second switched capacitor combines the benefit of the two worlds. Since you don't need an isolation LLC is clearly an isolated topology you are actually merging the two worlds. You lower the losses within the magnetic compared with an LLC but also within the fed. So you can extend power capability of a given footprint with these topologies. You mentioned earlier that we have products that are designed specifically for unregulated families. Can you pick out a couple of highlights that you want to share? Yes sure.
Not only, of course, for our unregulated because our general purpose of products but are really shining into these unregulated converters, for example the source-down MOSFETs. So the source-down package is actually helping to keep up power density and efficiency thanks to its superior thermal performance and cooler than a current PQFN three by three. In a drained down configuration, which actually we have in 15 volt, 25 volt, 40 or 68 and so on. So all these voltages are actually used in IBC converter. This package offers best thermal performance, but also best in class R on A I guess, each application has some slightly different needs. Yes. So, can you say something about
picking the right product based on your application needs? Yes, you need to actually to tailor a given product into a converter, which actually has been defined from the application. So product to system is really important and at Infineon we are really working in that because you need to understand the problem of our customer, you need to propose a new topology, indeed we have proposed a lot of topology, but also eventually, since is the most important part in enabling high performance, you need to design your switch or your driver or your controller based on what you actually need. And the answer is actually, that we design our products based on these specifications. Okay, great.
So let's switch gears a little bit. And you talked a lot about unregulated converters and how they're so important. Let's talk about ratios.
Why are different ratios important in the second stage. Well that is really important because this is also the trend we are seeing out there. So today but also in the future 4-1 is the most used ratio. There is a simple answer. 48 divided by 4 is 12.
So you are dividing the voltage, multiply the current. You can see that you can do in very different ways. You can do in an LLC, STC, HSC type of topology. ZSC...
So there are a lot of products available for this ecosystem and this is very efficient. So this is for sure a ratio we see out there. But of course it has pros and cons compared with other ratio because with the 12 volt you will need to use 25 volt technology for the second stage. Meaning that you cannot push very high frequency for the second stage and then the power density cannot further increase and then also the limitation in transient might be a problem.
Indeed there are also ways and strengths to going further down, for example 5:1, 6:1 and 8:1. 5:1 is something that as been considered in this system is not really popular nowadays, but we are actually working on that as well because we can really perform at the very high efficiency, for example with the HSC type of topology. And what is also interesting that the maximum voltage faced by the VRM is 12 volt. 60, which is normally the maximum voltage
in this, in the OCP, 60 divided by 5 is actually 12 volt . So is also very important for the reliability of the entire system. And now 8:1 is also a conversion, which is interesting for this system. Why? Is simply a conversion system, which is enabling vertical power flow. So, from 8 to 1 you can pretend to switch at higher frequencies for the second stage, because you have the chance to use different devices with a different figure of merit.
You have the chance also to use a topology which are offering high performance from the first stage, meaning that, your actual performance are increasing. But why move into vertical power flow? So vertical power flow is really important to increase the efficiency - not only the first and second stage, but also within the PDN between the VRM and the ASIC. So you have a certain distance between the VRM and the ASIC. So if you are able to place on the bottom sign this is clearly the shortest one, because by placing it laterally, you are limited by the actual size of this ASIC. And as stated before, these ASIC are enlarging and enlarging. So vertical power flow is the only way to enable high performance system because it is not only about the first and second stage is about the entire efficiency and the actual transient performance of the system.
Okay. Yeah performance. But what about total cost of ownership? Does it also influence that as well? Yes, that is really important because when it comes to efficiency and to power losses is a double gain because if you have a more power losses you need to cool it down.
So you are consuming actually energy to cool it down, but also you are consuming less energy. So meaning that this is one way to reduce the total cost of ownership for the end customer. Okay, sounds like a win win. Okay. So 4:1, 5:1, 8:1. What about...what does the future of these architectures look like.?
What can we expect in the coming years? Well is very difficult to predict because there are different customers, different needs, different trends, from the mechanical specification, from the electrical specification at any level before the 48 volt, within the 48 volt, and these accelerator cards keep changing. Vertical power flow is one solution. Definitely because you place on the bottom side, this is very simple from the mechanical arrangement, but of course, has a lot of challenges you need to cool it down. Something that now is not anymore on top, but is placed on the bottom. Moreover, you need to deliver a lower voltage rail to the second stage. From the first stage and normally the first stage is placed on the top.
So you have also the losses in between to consider. So there are challenges, but definitely the main bottleneck which is the PDN between the second stage and the ASIC now has been solved thanks to this implementation. So Roberto, thank you so much for sharing your knowledge on 48 volt architectures. We can see how they're really going to influence the future of AI. Thanks for being here. Thank you. Kelsey.
Thank you everyone. And thank you to our listeners for tuning in to the podcast for engineers. This is part of our We Power AI series and we want to hear more from you. If you have any feedback or any topics that interest you that you want to learn more about, please send us an email at wepowerai@infineon.com. We're looking forward to hearing from you. Stay tuned.
2024-08-06 04:17
