David Schmidt, Dell Technologies | Is Your IT Infrastructure Ready for the Age of AI?
>> Hi, buddy. Welcome back to Round Rock 2. We're here outside of Austin in the Dell Experience Lounge. My name is Dave Vellante. I'm here with Rob Strechay and David Schmidt is here. He's the senior director for product management for the Dell PowerEdge line of products and servers.
And there's so much, David, going on in the data center. Everybody's trying to transform and prepare for this new era. What are you seeing out there? >> Well, hey, thanks for chatting with me. Good to see everybody. We've been talking to customers around the world past couple of years and if there's one tone or one piece of feedback that is predominant across all areas is really how to transform that data center and prepare for the next wave of computing. That next wave has to incorporate AI workloads, it has to refresh the existing workloads that run customer's business and figure out how all of that gets incorporated together.
And customers are looking at the environments that they have today. They're asking questions about how to size out next generation. So many of the constraints come down to power and cooling, how they optimize what they have, how they make room for what's coming.
>> Yeah, that is the big constraint. Let me ask you a question, are customers... You read in the headlines, hyperscalers spending hundreds of billions of dollars and buying all these GPUs and it's overwhelming sometimes. How are enterprises thinking about it? Because they have data on-prem, data gravity, they understand that it's expensive to put everything in the cloud.
They're not going to put everything in the cloud. So what are they doing? Are they retooling their data centers? Can they get by with a hybrid air-cooled? Are they retooling for water-cooled? What are you seeing out there? >> You're definitely right about the data gravity of just they need to be processing as close to their data as possible. They certainly don't want to take that outside of their control and outside of their environment. We've heard a lot of that. And then, of course, there's a lot of just repatriation conversations going on.
But to answer your question, it is a mix of both. There is how do I take my existing data center investments and make sure I get every ounce of return out of those investments? How do I retool and think about future technologies like liquid and make the right choices as I build and expand or retool my existing environments? So it's a mix. It's a mix. >> They must be going in phases then, right? Obviously.
Because they can't just stop and say, "Okay, we need to be at a steady state before we can do anything. " So if I understand it, they're taking advantage of their existing infrastructure, maybe modernizing that, maybe freeing up some dollars to retool where they perhaps <inaudible>. >> Absolutely. And there's not one customer that is... Not everybody perfectly lines up on a single date in a single wave. Different customers, different segments are at different phases of that overall journey.
>> And I think when you look at it, it's so complex and organizations, and we see this in the data from our partner ETR around, the fact that it's right now, more than 50% of applications, not just AI, but applications in general, cloud native applications and stuff are being built on-premise and data centers, and I think a lot of that has to do with the data gravity and the actual connectivity to that. What are you seeing about, because it's not just about the servers, it's about how you connect them together and things like that. How are you seeing that evolve as well? >> There's some great examples. I mean, the fabric through which all of this data is going to move is extremely important. And obviously that starts and stops at a server to a data source at some point eventually outside of the data center.
And an example of the things we're having to build for our customers right now in the questions that we're answering is, how are you balancing that IO between what you need to put onto the fabric and move data in or out of your system overall? I'm speaking in general terms to the IO that you need to use locally for storage. And there's a lot of great capabilities out there in our servers that allow for know NVMe density upfront allows for IO capabilities, IO communication out the back. And those are obviously intertwined, especially when you're processing large amounts of data. You need to be able to move data inside and out of your system within the space of hours. We've talked to many customers who have those metrics in place. They know exactly what they need.
Those are the kind of questions and conversations that we're having. >> One of the things we see also, especially with AI, is that people haven't quite gotten to production in a lot of cases because they're worried about security. And security really starts inside. >> Absolutely. - How are you seeing that
and what are some of the questions and concerns that you're addressing for <inaudible>? >> We have had what we believe is industry- leading security within our PowerEdge servers for several generations now. The conversations with our customers have never been more centered around security than they are today. And it's not just security within the platform itself, it's security within the overall supply chain. How do we make sure that what we send out of our factory when it lands in the customer's environment, we need a way to prove that that is exactly what was shipped to them from our factory. And that's supply chain secure component verification.
That's supply chain security. It starts with what we build inside of our platforms with PowerEdge and our systems management, our iDRAC controller inside the server and our hardware route of trust. And so from there you can walk that into a variety of security options, and that's just going to continue to grow as we move forward through the rest of the year and into the next five years.
>> How do you see it playing out, David, from a workload perspective? So you've got all these on-prem apps that are working, they were moved into the cloud, very high value. Is it a case where customers are trying to bring intelligence into those apps? Are they building new apps and new workloads? What are you seeing? What patterns are emerging? >> It's a mix of both. I'd say it's probably how do I bring AI to what I have today? How do I do it with either a combination of compute- centered infrastructure? In some cases, there may be a need for large- scale AI systems that we're all talking about these days. You guys referenced it earlier, but it's really a question of what can I run on the compute infrastructure that I'm building? And so if you've got small language models, 3. 2 billion parameters, perhaps you can actually run those and serve hundreds of users on a single PowerEdge platform with two turing-based processors, for example. And we have white papers out that show how that's done.
And so you have the ability to go set that up and run that next to your current applications, next to your existing deployments today on top of our PowerEdge core compute infrastructure. >> So that would be a new-ish workload that I'm creating, essentially, because they didn't have GenAI a couple of years ago. Now I do. And so it's kind of a rag- based chatbot? That's what it would be? >> That's exactly what it is. And then, to answer your question about existing workloads or existing infrastructure, it's really a question of consolidation.
And so the consolidation is, I think we've used these numbers a couple of times, probably no surprise. You can get great consolidation going from the generations that were out five years ago, six years ago to now you're anywhere from five to one to seven to one. You look at Gen over Gen performance of some of our latest processors, our latest platforms. You have like 122% performance increase, but you also have a performance per watt increase. So you're getting 65% better performance per watt than you did in these previous generations.
And you can take that, you can use that to consolidate your workloads onto smaller footprinted servers, free up space, not only free up power, free up thermal capability in your data center as well and then you can go deploy your new AI workloads alongside your existing infrastructure. >> So it sounds like that's the right metric. Is that the one that customers are paying attention to or are you having to educate them to focus on it? >> I find that this year, more than maybe the past five years combined, everything about power and thermals, everybody has become a thermal engineer to some extent and everybody understands how that's taking place. And so it's a metric that resonates and oftentimes we're hearing about it from customers before we have a chance to tell them. Right? >> And I think when you start to look at this, like everything else, GenAI has not made everything simpler in the data center and there's this manageability aspect goes along with it. Where do you, and where does Dell technologies really focus on when you look at that manageability? >> That's a great question because we've been talking a lot about the environmentals, about what's going on inside, not only inside the rack, but how that data moves outside of the compute system and then the environmentals around it.
We're really heavily focused on providing that holistic management experience so that you can understand how air is flowing through your system. You can understand the environmentals around it. You can actually take actions based on power consumption. You can implement budgets if you need to. All of these tools have been in place within our PowerEdge portfolio for a long time. Our OpenManage software suite has had this capability.
Now more than ever, that's really coming to the forefront because customers want to know how to implement this. I didn't really have... If you asked folks a couple of years ago, how much power are you getting into your rack? Maybe you get an answer four times out of 10. Now if I'm asking a customer, how much power do you have for your rack? 10 times out of 10, they know exactly how much power, 15 kilowatts, 12 kilowatts, seven kilowatts.
They know exactly what they have. And they need the ability to measure it and everything that's influencing consumption of power, they need the ability to measure that and track it. And that's what we're focused on providing through OpenManage.
>> And you mentioned small language models before. I mean, we put out the power law a couple of years ago, which it's playing out, Rob. I mean it basically suggested that there's data on-prem, customers are going to be able to use smaller models, that open source is going to pull the torso of that power law up, which we've seen open source do that.
The DeepSeek stuff was really interesting because that lowers the cost for doing AI. And so I think there's this misperception that it's got to be this big giant thing. It doesn't necessarily have to be. People are picking their spots. Do you agree with that? >> The innovation's not going to stop, is it? It's going to keep going. There's not going to be one wave
of innovation and then we go digest it for another three years and then there'll be another wave of innovation. It's going to be iterative. We have talked for a long time. PowerEdge, typically, we talk about generations or G's and are we in a 16G, 17G? What are we? But really that concept of G's is so much more fluid because customers need to take advantage of the latest innovations. They need to do it right away to remain competitive within their environment. Our portfolio focus is how do we provide that as quickly as possible? We've been talking about our next generation AMD- based products for several, several months now, about approximately two quarters now. We're out there talking about our Intel Granite Rapids-based platforms.
We've been doing that since last year and continuously refreshing that and improving that design cycle and delivering that innovation to our customers so they have the latest, greatest platforms on which to deploy latest, greatest workload or software innovations is really key. >> Is a lot of the conversations you're having with customers is that we hit on the thermals and the power and cooling, definitely massive discussions, especially when you get into Europe and some of the other places where they're very constrained on power and- >> And there's regulatory - And the regulatory is there as well. Are you seeing that, really, they're looking at this and now there's also this inflection point not just of GenAI, but also that you have the hypervisor changing and people are looking at how do I get more consolidated on that hypervisor as well, so I have less things to manage? Is that part of what's fueling some of this? >> It definitely helps those types of customers, whether you're answering a regulatory activity or you're just out power. And by the way, it's taking place in parts of Asia as well.
You're out of power. It's helping provide solutions to that problem in a really neat or concise, neat and tidy kind of way. You say, okay, we're going to go consolidate. We have customers in Europe that are retooling their data center and obviously sustainability is top of mind for them. Power efficiency is top of mind As. We work with those kinds of customers for them to redesign and think about their future state, they're really enjoying their conversations with us because we're thinking about rack scale.
It's no surprise we've been talking about that for a while, but we're helping them think about rack scale, both in how we can deliver to them, how we can help them design, deploy, manage, and monitor. Like I said a few minutes ago, we're thinking about everything end to end and then we can help them design from a thermal perspective and a power perspective at a data center level and really meet the goals that you outlined. >> The entire stack is being re- imagined from the operating system all the way up to the applications that are getting more intelligent. You've talked about power efficiency and cooling. Silicon optionality is another component of the compute piece.
Storage and networking. Parallel file systems are the hot thing. Object. And for unstructured data, ultra low latency networks.
You've talked quite a bit about the server piece. Can you address some of the storage and networking pieces? >> Sure. I mean, the low latency and the connectivity is really where you take our storage capabilities and helping our customers deploy, whether it's our storage platforms, whether they have their own storage platforms.
The connectivity to our systems is what we're really focused on. The compute side and making sure that we have the right fabrics in place, we have the right IO capabilities out of the bag. If it's just, in really simple terms, if it's a by- sixteen PCIe slot, we going to be able to support 400 gigabit ethernet. That's the baseline unit of measurement as we design, making sure we have that type of capability inside the server.
So we've talked a lot about what we were doing across a couple of different areas within our networking lineup and how we piece all that together and build the right reference architectures for our customers and the validated designs so that they can understand how to deploy these things in the right way. We get a ton of object scale questions. I was working on something this morning, how do we design for object scale and do it at a really large scale level? And it really comes down to the server design just as much as it is the storage back end and then the type of IO capabilities we build inside the server, whether it's NVMe direct and really, really high density NVMe storage options or just basic cheap and deep type of platforms as well.
>> So the cost optimized versus, but it also seems like organizations are rethinking their data strategies for obvious reasons to get ready for AI, but also, we have data for analytics, we have data for some of our mission-critical. It seems like organizations just want to have a data state that's more broad. You talk about federated. Do you see that trend taking shape yet? >> It's starting to. I think the biggest conversations we have are happening at the design level at a large scale.
And so it is recognizing that with these AI systems comes quite a bit of data either consumed and then obviously generated on the back end. And so how they designed for that and do it in a very uniform way and know that they want to be able to dial that up as the year progresses or next year progresses. We're having some really large conversations right now with customers because they know we can deliver it and they know we can deliver at scale and we can help them with the rack seal design as well. >> And they're starting to think about, actually many are doing, creating synthetic data. So that's just more data.
>> More data- - Against data. - ... against more data, yeah. Exactly. Exactly. >> Awesome. No, I think it looks like that customers are really, I think, finally incented to actually go through and refresh. I think like you said-
>> More than ever before. >> That inflection point. What are some of the use cases that and other applications outside of GenAI that you're seeing really pushing your portfolio? >> You have a... It's not really fair to call them standard because I think they're awesome use cases and workloads, but you have a set of workloads that have been... We've been talking about it for many generations now, whether it's electronic design. There's a ton of financial workloads and applications and different subsegments of the financial industry where those customers are finally realizing what you just said, that now is the time where the return on that investment of refreshing their infrastructure, that equation is starting to look right because there's no denying to performance gains.
And so across the ones I just said, we're seeing some really interesting things. Electronic design is a really favorite of mine because I feel like it was always there, sitting behind high performance compute. Now it's sitting behind AI, HPC and HPC and EDA is right there fueling so much of what we take advantage of in our lives.
And the customers that need to perform those types of activities, they need to do it at scale as well. It's another example of the scale customer. And so as we look at some of our rack-scale type of architectures we announced last year, like the M7725, we're having them look at that platform and say, "Yeah, this makes sense to me because the investment or the return is good enough that I'm going to build a new data center. I'm going to think about building liquid from the ground up. I'm going to think about the power I can provide because that's what's going to fuel the design side of my business.
And they want to look at our rack-scale platforms and go deploy that type of capability." >> Yeah, they want to get ready for the future. And so that's where the conversation starts is we've got to rethink this infrastructure that's for the next decade, not for next couple months. >> I think what I love about our portfolio is we have solutions that can fit in air-cooled environments today.
We've done so much to improve airflow and thermal capabilities in our latest server designs that if you're in a traditional 19-inch environment, you're not ready to adopt next- generation thermal technologies. You just need to consolidate, recoup those power cooling savings and reinvest it. We can help you do that with what we have in our rack servers, our R600, R700 servers. If you are rebuilding a data center, building a new data center, and you're thinking about rack-scale, we have the ability to do it both on the compute side and we have it with our AI systems that we talked about at SuperCompute back in November.
>> I would think there's a lot of customers in the former. I mean, yeah, the big banks, of course, in the latter. And they're going to make that investment, but a lot of customers just don't have the capital to do that and it's too risky for them. David, thanks so much for spending some time with us. Really appreciate it.
>> Thanks for having me. It's good to see you. >> All right, thanks. All right, keep it right there. This is Dave Vellante for Rob Stretcher.
We're here in Round Rock. We'll be right back right after this short break.
2025-04-12 11:08
