The next five years of AI and how to prepare
Unlock AI. Unlock the potential to do more, solve more, discover more. Hewlett Packard Enterprise and NVIDIA unlock the power of generative AI combining HPE's enterprise grade mastery with Nvidia's revolutionary AI breakthroughs. We're co-developing and co-innovating to simplify the complexities of AI, making it easier and faster to deploy.
Not only are we building and creating together, our combined solutions empower teams to do the same scaling AI from individual data scientists to an enterprise wide platform for innovation. With all these things coming together as one, organizations can deliver on AI driven business outcomes faster, seeing results in weeks instead of years. Welcome to the new world of NVIDIA AI computing by HPE.
It's time to unlock AI. Please welcome HPE Executive Vice President and General Manager Compute HPC and AI Neil MacDonald. Welcome everyone to Discover 2024 in Las Vegas. I have the honor to lead you through the next hour on generative AI and show you how your enterprise can succeed today.
I hope you really enjoyed the wonderful keynote this morning. The first keynote ever to be held in Sphere. It was an amazing experience, wasn't it? And even better to see HPE and NVIDIA joining forces on AI. If you are listening carefully, you'll remember that Antonio believes that AI can be a force for good in the world, helping solve some of our most pressing problems in areas like climate or health.
And Jensen referred to the same powerful phenomenon as the new industrial revolution because of the scale of the massive changes in society and in life that are going to result from generative AI. That's heady stuff. If I were sitting in your shoes out there today, I might find it difficult to approach. I'd be asking myself, how do I take this and do something useful in my organization? How do I navigate all of that complexity to get to meaningful outcomes? I might think I don't have the same resources that hyper scalers and big model builders do. And that's what we are here to help you with, along with a host of experts, we're going to walk you through the changes that are happening in technology as a result of generative AI. We're going to walk you through what you can expect in the coming months, quarters and years and what you can do now to gain the productivity benefits of generative AI in your organizations literally now at this show.
But let's begin at the beginning. What are we talking about when we say artificial intelligence? It's not new. 34 years ago.
I got my university degree in AI. It's not a new development, it's a rich field with a long history. But in traditional terms, artificial intelligence didn't rely on massive amounts of data. Instead, it relied on handcrafted rules and expert knowledge. And it excelled in industry specific tasks were predefined rules that could be written down guided behavior. That was the error of things like expert systems.
But then, as a more modern subset of AI, we saw the emergence of machine learning and then later deep learning which allow computers to make predictions without being explicitly programmed. Those neural networking technologies that applied in machine learning and deep learning, take labeled data and train artificial intelligence in order to label new data. It's really all about learning patterns. All of that has been around for a while. But generative AI is a key shift.
Because generative AI is now about generating things, not labeling things. Generative AI is uniquely able to create entirely new content. It resembles human generated material, it might be text, it might be images, it might be code, it might be any number of things.
And because of that generative AI holds the potential to transform fundamentally human productivity. So how are gen AI models developed and made useful for organizations like yours? AI is nothing without data. So gen AI has to start with gathering all of the potential of sources of data relevant to your project, that data has to be curated and then transformed and then organized in order to begin your generative AI journey. That high quality input data is essential to better trained models and higher quality output.
So in this stage of training, large foundational models are trained on vast amounts of data. These models learn complex patterns and gain a contextual understanding that makes them capable of generating generic content. But training foundational models takes an enormous amount of data, especially if those are general purpose foundational models.
It is without doubt the most computationally intensive workload of our times, an exabyte of training data was required for chat GP 24. That's a model that has close to two trillion parameters to be trained. And that task took tens of thousands of GPUs for months and months and months.
So obviously, there are very few enterprises that are creating their own large language model, foundational models from scratch, given that immense investment in time and resources. But if you take a pretrained model and apply it to your business, that model hasn't been trained on your business, it hasn't captured the knowledge and the intelligence that you have inside your organization. So if you take that pretrained model, you're not going to get output that is high quality and relevant to your use cases in your business. So what to do? Well, enterprises can and will fine tune models or develop smaller industry, context specific models that bring that enterprise's data or industry knowledge to the model to improve its accuracy for those specific use cases. Or other organizations will augment the quality of the output by leveraging retrieval, augmented generation, which accesses existing enterprise data and information to better inform the quality of the output. And the great news is that that fine tuning and that RAG can be done with much, much more modest infrastructure in much faster time.
The key though is that the existing IT infrastructures that you run in your organizations simply cannot do this effectively. We need a new paradigm. We need new architectures and to tell you more about that, please welcome my good friend and colleague Bob Petty, the Vice president of enterprise Platforms at NVIDIA. Bob.
How you doing, man? Good to see you. Thanks for being here. So happy to be here. It's going to be hard for me to compete with this Sean Connery like, you know, base voice, but I'll, I'll do my best. We'll give it a go. So Bob in your role, you get to see all kinds of different enterprises. So you've got kind of a unique vantage point to share with our audience a perspective on what do you see successful enterprises doing today with generative AI and how they're doing it.
Yeah, there's, there's plenty of examples. We could wax poetic for hours. I think there's a combination of industry specific uses of AI that I'll get to. Some of the more popular ones are these horizontal use cases.
They're chat bot assistants, copilots if you will, big ones in customer service centers using AI to help a customer service person improve the time to resolution. Plenty of examples of use going from a time to resolution of 15 minutes down to 30 seconds. And if you as a customer were calling somebody or, or texting somebody for customer service that that 15 minutes to 30 sec is pretty, pretty impressive code assist. I think by 2028 80% of the enterprise coders, not just the scientific coders, 80% of the enterprise coders are gonna be using code assist AI agents to help them code suggesting better ways, tighter loops. There are digital avatars to be brand representatives and then there's the massive number of use cases in in manufacturing in terms of design improvement, avoidance of errors.
Those are, those are some of the bigger ones, the ones that you know, like reach into my heart are the runs around health care helping to cure cancer, helping to eliminate disease, helping to, to not put somebody through so many scans or mammograms by being able to check early on and get the things before they get too bad. So it, it covers every industry uh and in a lot of cases, they will, they will take, you know, an LLM or a subset of that LLM, find it fine tune it as you mentioned. But we're seeing a lot of industry specific ones generate because of these industry specific use cases, you know, Neil alluded to the work that was done with Chat EPT four, the number of parameters and the months of computing and the, and the tens of thousands of GPUs, that's not what you all have to, to go through. There are plenty of models being developed which will most likely work with will be a smaller model, a smaller model that's trained on industry data that understands the semantics. You will likely then fine tune that model to include your own corporations, semantics. And then you will likely use that model to inference so that you're providing your, your workers improving their productivity by fine tuning into that. And, and then infer from that model,
every time you take that step, and, and every time you take that step, latency becomes more and more important and data locality becomes more and more important. In the, in the customer service example, you know, if you're waiting for an answer of why did I blue screen? And that takes, you know, minutes. You're, you're probably gonna have some choice words right there, there'll be some whiskey Tango Foxtrot things going on. Um, that latency to give that answer in milliseconds is gonna be key, but to get that latency, it's not just about fast GPUs, it's, it's the entire system. So that data locality and the access to augment retrieval, augmented generation, the the ability to quickly augment a trained model with additional information is gonna become more and more important.
So when you describe that Bob that, that's kind of a different architecture, right? You've got a company that's generating proposals or customer service interactions or any of these use cases that you've talked about and they need to get their data local to the compute that implies other changes in the architecture too, right? Oh, absolutely. You know, if you saw the key note, Jensen talked about this shift from instruction led computing, a programmer, trying to think through all the ways to program something to query different databases to end up with a result versus programming that kind of writes itself because you're basically asking it questions. How do I improve my supply chain? How do I improve my customer satisfaction? And again, that data locality, the data gravity means you gotta think a lot more about the networking.
You gotta think a lot more about the CPU and the GPU balance and interconnects and storage is critical. Storage can't be spread out all over the place. If that last piece of information is what gets you from 15 minutes to 30 seconds that can't be in some distant location. So enterprises are doing this successful, enterprises are doing this today. PCs, we see it growing in a number of ways, but it is going to need to be a new architecture.
And that's one of the main reasons why we've been working together for the, for the last year and decades in reality. But working together to end up with what we announced today, NVIDIA AI Computing by HPE. Well, we're really excited with that announcement that Antonio and Jensen made this morning NVIDIA AI computing by HPE is a portfolio of code develop products, services and go to market enablement to help you accelerate generative AI in the enterprise. And NVIDIA has clearly been moving at an amazing rate of innovation when you think about accelerated computing, but we don't just stop there.
NVIDIA AI computing by HPE encompasses the three pillars of challenges and requirements that you're all gonna need to overcome. The people, the technology and the economics in this new gen AI world. So let's step through each of these components within NVIDIA AI computing by HPE and show you how you can accelerate your journey today. The first thing we need to talk about is a little more detail, Bob on this new architecture and how things need to change for gen AI. So maybe you can set some context for our audience here. Yeah, sure. So some of this will sound self serving. It's not meant to be.
We, we think computing is no longer about just a PC in a box CPUs are critical. We love CPUs. The accelerators that we put in the GPU accelerators that we put in are critical and we keep revving those at a faster and faster pace. The reason being, you know, there's orders of magnitude and improvement in performance. The ability to connect multiple GPUs is key with things like our Envy link fabric. I can take a 72 GPU box which you can see out here on the floor and treat it like one GPU, treat it like one GPU. So think about one unified memory for 72 GPUs acting as one neural mind.
Um thinking about the, the CPU and the GPU interconnect, we've done some work and if you look at on the floor here, um Plenty of Grace Hopper systems, Hopper, our flagship product today. Grace Hopper systems that interconnect 900 gigabytes per second that interconnect is key. Um Anytime you're slowing data between CPU the choreographer and where the work is getting done, that's that's important and then fabric. So so in that case, rather than having a CPU talking to one or more GPUs, you got a super chip, they're, they're actually in a super chip that shares the memory between the parts and you know you're, you're looking at something that's seven times, you know that super chip has seven times the speed between the CPU and the GPU than you would if you were just to stick a generic CPU and a generic GPU into that network because of the data locality and the importance of getting, getting to storage On-Prem. The fabric is, is one of the biggest components. Especially as your, as your Datacenter grows over size. You, you no longer just need to think about the north south traffic, the traffic between each of the nose in a given RAG.
You need to think about the traffic from east to west and how many RAGs you have that you may be putting to bear on a problem whether it's fine tuning a small model or, or infer So spectrum X is our AI network. We specifically designed it to be built for AI. It's 1.6 times faster than uh ??? and out of the box, we still support quantum and finaband for, you know, larger customers.
They're typically more akin to using that with supercomputing. And you've got an incredibly aggressive road map that you've announced, right? Yeah. Yes, we do.
Hopper is what we have today. We've, we've announced Blackwell at our GTC conference and you'll, you'll see that next year we're, we're trying to, we're, we're, we're, we're trying to accelerate the pace that we let's we, we at which we communicate our road map because we used to be fairly secretive about it. We would, you would find out about a new GPU when we rolled it out, but we wanna give you the confidence that this is not a one and done. And so you'll see us announcing architectures, they're not meant to give you buyers remorse. You want the best AI system in the world today, you you're gonna see them out on the floor there. As we, as we uh continue to roll out new systems, just as we've been working for the last year on what we announced with uh HPE private cloud AI Blackwell will drop right in Ruben will drop right in.
And as Jensen talked about this morning, we do a lot of work on the software side. We have more software engineers in NVIDIA than we do hardware engineers because we want to ensure that, that there's code compatibility. So you don't have to rewrite code that Cuda piece is, is essential in, in terms of ensuring compatibility. So our commitment to each other is that NVIDIA and HPE, we time to market with each new innovation that we have.
Thanks Bob. And that's a really critical part of our technology pillar in NVIDIA AI computing by HPE. But by pledging to be time to market on Blackwell and future Ruben systems that we just talked about, we ensure that you, our customers have that time to market access to the latest technology when you need it from partners you trust on day one. But now that we're in lockstep and cranking out systems together, we could talk all day about the technical collaboration between the teams. But maybe if we can give a quick focus on how do we power and cool all of this NVIDIA is generating this amazing processing efficiency gain generation over generation.
And the accelerators are getting more performant faster than they're becoming more power hungry, but they're still consuming more power, Bob. Absolutely. You know, the, the the watts per, per calculation are getting better, but the total power is increasing.
That's, that's a necessity. If, if you want 30 seconds versus 15 minutes, that's a necessity. That's where liquid cooling comes in and is is so critical. We will continue to
try to, to maximize that point on the curve with power and performance. But you will see gen to gen power increases as Jensen said, we will, each generation will improve the performance at a, at a rate that's much, much higher than the power increase. What that means is for the same power envelope, you're getting three times, five times the throughput which leads to sustainability. So, liquid cooling is gonna be the future. Um
HPE. One of the reasons we partner with HPE is um we've been, you know, they've been doing liquid cooling in a great way for 40 years. You if you go out on the show floor and you look at the boat, uh Bonna display a perfect example of liquid cooling done. Right. Well, in fact HPE has over 300 patents in liquid cooling.
And even for systems that aren't natively liquid cooled, we can help, we can do liquid air cooling surrounding air cooled systems using chilled water cooling that helps cool down the air cooling and get greater efficiency of those systems as they run. Or we can do a hybrid, we can have direct liquid cooling inside our systems assisted by fans. But where this whole industry is heading is right, where we have decades of experience from our supercomputing and leadership scale operations, which is building fanless 100% direct liquid cool systems.
And we're really excited about that and you can see it out on the show floor. But there are other ways to gain efficiency too that are really important for people to consider is the thinking about solutions, Bob. So if we think about models and software and the NVIDIA AI enterprise stack, maybe you can share a little bit of color on the efficiency that's possible to gain there. Yeah. So Jensen alluded to three essential elements that we need to be thinking about. There's the computer infrastructure which we talked in some detail about.
There's the data infrastructure that I'll, I'll get to here in a second and then the model infrastructure that the software around the data wrangling around the data ingest around arranging the data so that the models can easily interpret and, and uh infer uh is, is part of what we call our in NVIDIA AI enterprise. There's a series of libraries and toolkits and things like cudf pandas. So if you're a data scientist, you can, without changing any code, you can get 100 and 50 X improvement without changing any code. Just by adding of all of the upstream optimization. Exactly single single reference to it in your Jupiter network.
100 and 50 X on data wrangling. Um the NVIDIA M and and NVIDIA inference microservice. It's essentially a container. We, we optimize, you know, it's almost like GPT in a container. But they're typically gonna be smaller models, but we do a full, full optimization for an entire AI reference stack and we containerize these. It can, it can be models that you've developed models that industries have developed, models you've downloaded from Google Microsoft hugging face. It doesn't matter, it doesn't matter where they run either.
They, they are containers in a box. Um I mentioned the software engineers. We, we work tirelessly in our NVIDIA AI factory, our N I AI factory to make sure that those use cases. We, we have hundreds of AI agents, whether it's looking at supply chain, it's looking at sales optimizations, it's looking at product design optimizations.
We have hundreds of those AI agents that were turning in the Nims, we do that as an enterprise and we know that you all will need to do that as enterprises as well using those nims. and getting those five X 10 X performance improvement on standard models, I think uh Llama llama 3 70 billion parameter. You download one from, from the cloud, you get X performance, you use the NIM that runs in HPE private AI and you get, you know, five X the performance that additional performance translates into efficiency and sustainability. Exactly. So when we think about the collaboration though, it's more than just leveraging GPUs and super chips and software stacks, we're also selling and supporting NVIDIA Spectrum X and quantum and finaband networking to connect these infrastructures together. And we also have storage systems from HPE that are certified by NVIDIA and integrated into our full stack solutions for generative AI.
Bob, we're really thrilled about the collaboration that we have here and about finally being able to announce what we've been doing with NVIDIA AI computing by HPE. We've been working together for almost two decades, but this is really taken it to a different level as we move forward here. I can't wait to see what we're going to be able to do and what our customers are going to do together with generative AI leveraging NVIDIA AI computing by HPE. Thank you Bob so much for the really appreciate it. Get on the train now.
But we got a lot of engineers in this room. We got an engineer up here and the tech is really exciting. But at the end of the day, what's at the heart of everything that HPE and NVIDIA does is our customers. So now let's shift gears from the theoretical and the technology and give you the chance to hear some real world examples from pioneers in generative AI so that they can share with you what they've learned as they have made their accomplishments on this journey and how they got started. So, here to guide you through that conversation is my colleague Sylvia Hooks.
Thank you. Hi, everybody. Welcome. What an honor to be here today. I'm super excited to be the person who gets to lead a discussion with amazing customers. And in fact, they're all household names that you've heard of before. We're gonna get to talk to Saint Jude Children's Research Hospital, Lockheed Martin and Walgreens. And I'd really love for you guys to listen, pay attention to the nuggets that they're gonna give you.
We're designing this to be really actionable. We're gonna start with Saint Jude Children's Research Hospital and here to share more on the mission and their AI journey. I'd like to introduce Keith Perry CIO of Saint Jude Children's Research Hospital. Welcome. Well, thanks Sylvia.
It's great to have you here. Great to be here. I know that you're a mission driven culture. Why don't you tell us about the mission of Saint Jude? Absolutely.
So, since 1962 our goal and our focus has been to eradicate pediatric disease. And so that's a very active goal and it's very evolving. So that what that basically means is the knowledge that we have today is not the knowledge that we need tomorrow in order to, to solve that problem. So, and during the development of our last uh strategic plan, one of the, one of the things that became very abundantly clear is the we needed to lean in further into data science for as it relates to biological discovery. So my role and I get to work I got the best job in the world. So I get to work with some of the some fans. Fantastic. IT professionals
in order to bring the technology that we need in order to help support the mission of Saint Jude Children's Research Hospital. OK, fantastic. And what do we want to do is talk about your top use case now and not everybody is a research hospital, but some of the things we talked about are really useful in all enterprise, tell us about your top. So I, I guess to summarize um and it's very hard for me to pick one.
So uh because I'm gonna get in trouble when I go back home. So uh so um to summarize our use cases really are about how do we enhance the productivity of a very highly skilled professional people so that they can both treat uh more efficiently and effectively and further our knowledge, discovery on pediatric catastrophic disease. So with that in mind, let me just kind of walk through a few.
So not necessarily just one because I, I can't help but talk about our research and our science and they have just jumped on board this gen AI transition and the opportunity on tumor classifications. And how do we think about that tumor? Where do we where do we, I, how are we identifying that? But as I move into the clinical space, that's where we're a little bit more guarded because we have to protect that the, the thing that you try to protect within a hospital setting is that physician to patient relationship. We covet that relationship, the patients covet that that time that they spent.
But what happens is we as in health care, uh We've burdened our physician community with a lot of paperwork. So we think that AI has an opportunity. So how do we alleviate, alleviate some of that burden uh and give that time back to allow them to be, have more, spend more time with our patients. And the other thing that we have to think about is the risk within the clinical setting and jumping between panes of glass introduce and weight risk and, and, and really waste time.
And so what we're trying to do is how do we integrate that within our current work flow. So, and then there's other use cases around the administrative side, HR, legal. I even is, we're thinking and challenging ourselves around how can we take this new technology and how could it help, help us become more productive in the future? I love that. And I don't know about you guys, but I, no one really loves paperwork. Right.
My, my sister is a doctor in Maine and she spends all of her evenings sometimes during family events doing notes, right? I know that was one of the use cases that you had. I know there was a lot to go through a lot of regulation, that you wanted to be sure and mindful about when you were setting up your data. How did you get to where you are to use cases? What stood in the way? How did you get there? Yeah. So, I mean, like the rest of the planet, um, in January of 23 we, we were introduced to open AI, really December and, uh, we, we got a small group together and we started to have conversations. One of the very first things that we had to do is really increase our competency.
So how are we thinking about this technology? Really challenging ourselves to go learn? So we brought in speakers, we read, we discussed and we just had an active conversation and then really that led to really a couple of things. One, we, we realized that we needed to extend that to the institution. So that learning and discovery was something that we're we continued to do.
The second is we really needed to set some guardrails. So we spent a lot of time defining, well, what is a policy that we want to put in place for the institution that really provide the guard rails for us as we think about AI and the third was really around uh use cases. So we kind of canvass the institution thinking about asking uh where those opportunities are. We've developed over 60 plus use cases that we're at least thinking about right now.
And the, the fourth is really where we are right now is we don't really want that, that small team that I talked about earlier to be the the governing body. So we're pushing that responsibility back into our existing governance processes and governance is an important part of this. I know that you deal with a lot of sensitive data. How have you addressed like data management? Yeah, and data management. That's interesting and ongoing. So because it's evolving because I I think that the whole management of data within the health care and research space is is unique because you know, you've got that patient sitting in front of you that you, you're very protective of that data.
But yet on some cases, you need some of that data in order to inform and guide research for the future. So we've got programs in place that kind of help with that data management process. Is there anything that you wish you had known before you got started with generative AI that our audience could learn from? Wow. Um a great question. I, I guess I would, should known what a prompt engineer was before that, that really came on strong.
I, you know, II, I also wish that we would have, we would have gotten some things done because we're, our time is being consumed by this fantastic opportunity that's in front of us. But I would just encourage everybody to just, it's, it's, it's interesting, don't hide behind the risk. You can manage through it and there's opportunities out there for this technology. Thanks Keith and I, I really like way that it's just you just start small, you learn about it, you talk to each other and then I think we talked earlier about, you just have to kind of go in and try it because you won't know the use cases until you learn and get, get into it. Yeah, absolutely. Well, thank you so much for coming on and I'll talk to you soon. OK.
Thank you so much, Keith, a big hand for Keith Perry. OK, great. So, next up, we're gonna talk to Lockheed Martin and, and about how they've been developing gen AI and you might think that Lockheed Martin doesn't have use cases that are very relatable to your places of work, but in fact, they really normal and you're gonna end up wanting them just like I want them now in my place of work. So here to share more about how Lockheed has been developing. Gen AI tools is Doctor Mark Maybury, Vice president of commercialization Engineering and technology with Lockheed Martin. Welcome. Mark. You have a cheering.
Nice to see you too. Well, thanks so much for joining us. Thank you. We've been talking a lot about this, but first, let's start with what is, what is the, the give me a company overview globally of Lockheed Martin and the role of IT. Yes, sir. So Lockheed Martin 100 and 20,000 folks uh around the world
and working in a variety of public sector contributions. We uh have 350 facilities work in about 50 or more countries around the world. And in terms of the digital piece, we have a corporate initiative about a $6 billion transformation ini initiative over multiple years uh that are called one LMX. And it's of course digital transformation, but also model based system engineering across the full life cycle.
OK. Now, I know there's a lot of projects you can't talk about, but there's also a lot that you can talk about. Give me the top one or two use cases that you are using internally that maybe other enterprises could try out. So, uh so we there, there are tremendous amount of enterprise uh opportunities and the key is to look at those that have the most value.
So, for example, in our space business, we've generated used gen AI to produce over 1000 RFP submissions, really broadening our ability as a business uh to support uh the federal government and governments around the world. But similarly, just for our en engineers, our employees having chat products that allow them access uh to the to the collective knowledge of those folks all around the world is a tremendous competitive edge. I really love the idea that you have a knowledge base internally that all the employees can take advantage of. And I mean, I think everybody is responding to rfps in some capacity.
So the idea that you can uh pull all that together and, and see productivity benefits, you tell me again what those percent I might look like for productivity gains. You're, you're talking and we have some independent studies that are looking at for engineering, for example, uh we can actually produce things 30% faster. More junior folks can actually do things more effectively than more experienced folks. But it's also the case that uh we can produce things higher quality.
We can produce things sometimes 2/3 times faster. And, and we can use it actually to envision. So for example, 11 example, our engineers are able to take natural language descriptions of a future system and create formal models and and diagrams from those automatically. And then finally, if you look at the enterprise in terms of cybersecurity, we actually use, use that to detect vulnerabilities to track adversaries.
And and also we're, the system is integrated for the DARPA Semantic Forensics program, which actually is used by law enforcement intelligence to detect deep fakes. OK. Now, you said something really important, I think, which is with, with that kind of productivity gains, there might be uh in apprehension that people in your org would lose their jobs because, you know, you can do it 30% faster. You might need 30% less people. What have you actually seen happen? Actually, we see just the opposite. In fact, just this morning I went and search
and you can go and you can find literally dozens of jobs that you leverage AI and generative AI across open positions right now. We've seen this in multiple examples with the computers, with robots. Those that invest in the advanced technology actually generate more jobs. In robotics it was a 70% differential over a 10 year period of time. So uh we saw actually, yes, some skills will be eliminated just like we use computers to do things that we had humans do in the past, uh, but a significant and we, we anticipate looking at, for example McKinsey and other forecasts somewhere between, uh, a 2.5 to a $4.5 trillion
creation of, of, of, uh, of new, new opportunities. So we're gonna need more people, not fewer people. And the people that you have employed now rather than they do less or you need less people. They're actually being augmented by generative AI. Exactly.
I'm glad you said that, you know, as, as a, a fellow of the American Association for the advancement of artificial intelligence, one of the professionals who has been working now for 36 years. That was my first thesis in generative AI. So this is not a new topic for some of us. In fact, at Lockheed Martin, we've been working for many decades. And to that point,
we've actually created a secure open AI factory which is in use by the 2000 AI developers across Lockheed Martin, but is accessible the 17 large language models via the AI factory to the 120,000 employees. So we actually require our employees to be educated in generative AI. We have policies at the board that help protect us from AI we have training for all of our, our employees. And so in fact, we're, we're actually articulated as an expectation of high performance as we transition into this generative future. OK. So you mentioned another couple of things. So in two minutes.
Tell me about a quote that you gave me last week, which is we love generative AI but it's biased, brittle and baroque. How have you overcome that? So, um we as scientists and engineers know that actually gen AI is not perfect. First of all, uh it can be biased, it can over fit the data, it can actually fail in operation, uh can be brittle as we all know, it can hallucinate. It can actually fail on novel data.
And finally, uh it can be baroque, meaning it's actually not very transparent. You have 100 and 20 layers in chat GP D four. You've got uh 1.7 billion parameters uh try to find the root cause of, of an error in that. So what we do is we have a variety of if you will defenses in depth policy at the high level, including a board subcomittee that actually does uh guidance which actually follows responsible a guidance from the federal government. In addition, we have technology.
So the, the, our, our, our AI factory enables folks to work securely. So they don't have to worry about releasing, you know, private information or releasing proprietary information because it's all contained those guardrails in those guard rails in. And then finally, we have obviously some processes for talent, right? We want human understand to actually be able to leverage this this powerful technology. I I like to say, metaphorically, we are living in a promethean moment. You know, for Prometheus, if you all recall brought fire into the world, fire is a great powerful capability, it warms us.
It can protect us from dangers, but it also can burn. And so we've actually essentially managed that controlled that so that all of us can benefit from that in a safe, secure and prosperous way. OK. Mark, my plan is after this show, I'm gonna circle back. We're gonna write a book together about it and then everybody will have those same rules and regulations. Rock and roll. Ok? Thanks so
much. Thanks for coming in. Appreciate it. Thank you so much. Awesome.
See I told you Lockheed Martin a lot like your enterprise too. And last but not least I want to bring up our final guest and that is Doctor Sheida Gohari of Walgreens. Come on in Shada. Welcome. So nice to have you. Thank you. My pleasure. Thanks for having me here.
It's very exciting. It is. There are, it's a big crowd. So tell everybody a little bit about your role and about the mission of Walgreens. Yeah, sure. So I'm head of AI at Walgreens. And Walgreens is health care pharmacy retail company with nearly 9000 stores national Wide. And the mission for Walgreens is providing health care services, retail services to customers, patients.
Many of you guys, maybe you got your prescription. Walgreens. Yeah, from Walgreens. Do it continue it? And uh you, you did it or you had online or in a store shopping experience from Walgreens as head of AI and part of uh IT technology team in Walgreens or mission is providing better services to our customers and of course, make it more accessible to people. That's another thing I love that you are both a medical facility and a retail facility. So you kind of doubles your chances that people out here are gonna know what you're talking about.
Tell me the top use case on the pharmacy side. Yeah, sure. AI sorry. Sure. So we know that our pharmacies, they work very hard in the stores. Some of the stores, they've worked 24 they are 24 hours stores. So
and sometime before they didn't have enough time to focus on our patient, we know that Walgreens is using AI for many years. But we were thinking how we can adopt our service with new technologies with generative AI to provide better services to our patient customers. Not only to increase the customer satisfaction, also help or iclude as well. One of the example at Walgreens is some imagine that you go to Walgreens stores and pharmacies, they spend a lot of time to input the data manually in the system.
So how much time they have to spend instead of spending more time focusing patients. So with using generative AI, we could um create automation on some part of the operations there that's for pharmacy. And the other thing is inventory management. Also how we optimize on inventory management in a right way or safety stock optimization. And also whenever we do all these works, this optimization, we have to create a lot of reports and data aggregation part also. So we are spend a generative leverage, generative AI also to create or reports manual stuff that before people did it to create data just to feed it to our models.
We automate some of the data preparation part also with using generative AI. This is in pharmacy but in retail also. Yeah, I was gonna say this sounds like the same theme that we heard from Doctor Keith Perry about making your skilled workers more productive, right? And at the same time increasing customer and patient satisfaction. So I know for you, the the mission of gen AI was not about cost reduction. What was that? What did, what was it instead? Well, it's not not necessarily about cost reduction. but sometimes we have cost reduction as well.
But in order to increase our customer satisfaction, imagine that uh how we could improve or promise time to pick up your prescription. This is the wait time that you might have in the, in the store, right? Yeah. And then you go to the store still, your prescription is not ready, ready. So having these technologies in.
If we provide better services to our customers, we we create better life for them. And also imagine that we create, we use leverage generative AI to somehow create prescription reminder for the patients in the call centers or pharmacies or technicians instead of spending more time on taking off the notes or some of the other things. So they can actually focus more on patient answer the necessary for necessary question, focusing our patient. So there are a lot we did and there are a lot that we have many things that we can do it to help. Not only our patient, we help our employees. So there's an interesting point in there which is this is all about patient helping the patient. But the more you know about the patient, it seems like maybe the more vulnerable your data might be or there might be more and more regulations.
What are some of the things around ethics that you've had to do with, with AI and patient data? Yeah. So we are responsible to use patient data, customer data, follow all the regulations. I AI all the regulations.
This is not only about the regulations, this is about the services, also the the application, the services that we provide to help our customer, we have to make sure that we follow ethical AI like for example, we don't or or models they are not actually they are fair to generate information we don't focus on uh on some of the groups, we forget about other genders. So we have to make sure our model results are explainable and other factors. When we train our model, we, we make sure that we, we deal with bias data, we make sure that we solve all those problems to make sure we are responsible to use this technology. OK. Fantastic.
One other thing that we talked about uh last week was the where your data resides in the role of On-Prem versus in the cloud. Talk about your journey there. Sure. So today we talk about generative AI but we know that why one of the reasons generative AI model work well because they are big enough to understand more things. And also you need huge amount of data to train them even if you want to fine tune these models. So or researcher or developers also, if they want to find these type of models, they need to aggregate data from different source, millions of records. If you want to do these works on the cloud, that would be expensive and it takes a lot of time. And CPU
that's why we were thinking how we could use a Spark graph leverage or GPU on PRE GPU to aggregate this data quickly, very fast. We prepare our training data set to run generative AI models and also influencing also important. We are excited about using generative AI. We know
that this is really good, the output is good. But these are very big models imagine that the smallest one, one of the smallest one, these has 38 billion parameters. So compared with the previous transformers that they were millions, they had millions parameters. Now we are dealing with billions of parameters here.
So infer also very challenging, having these GPUs help us to somehow accelerate or response. Also, we can use them in more use cases. Fantastic. Well, Sheina, we're out of time unfortunately, but thank you so much. It's great having a, a technical woman on stage with me and I'd invite you all to join our women in tech session tomorrow at 10 a.m. Thank you so much. Bye. Thanks.
Fantastic. Well, thanks everybody for being so attentive. And now what I'd like to do is hand it back over to Neal and he's gonna walk you through a deep dive in the solutions. Thank you, everybody. Thanks so much Sylvia. So, how great was that to hear about some real world examples that your peers and industries are working on today.
We believe that every organization is either gonna become a generative AI enabled organization or is gonna become uncompetitive. So what stands in your way is an enterprise of embarking on that journey? Well, first of all, it's time to value. Enterprise teams can struggle to enable AI and IT ops to drive those AI pilots to production and prove the business impact fast enough. Data silos with your data spread all over the company can limit the amount of data that's available to you for AI work. And that leads to bias models or stifled innovation and solution environments for Gen AI are typically too complicated today, placing a lot of burden on you to manage and operate while ensuring that you have the security, the governance and the enterprise regulatory compliance that you need.
And most of all, you need flexibility because you're on a voyage of discovery, you need to be able to start small and grow and adapt as you learn on your generative AI journey. Now, to give you some amazing new perspectives on how we can help you with NVIDIA, address these challenges. I'd like to invite our CTO and my friend and colleague, Fidelma Russo to the stage, Fidelma.
Hi there. Thanks for coming. Thank you. So we begin. There's a really important message that we wanted to share and you're gonna share on our behalf.
So, first of all, it's been a phenomenal few months, believe it or not, but I personally am with Neil. would really like to thank all of the engineers, product management, marketing teams and everybody across both NVIDIA and HPE who made it possible for all of this innovation to be here today. So a great, great, thank you to everybody about across both companies. Let's give them a great big round of applause. So this morning, Antonio and Jensen unveiled an industry first tell our group more about what that is. So it's a component of NVIDIA AI computing by HPE portfolio, the HPE private cloud AI.
And so it's the industry's first turnkey solution to create an On-Prem private cloud that comes prebuilt and pre-integrated for you. And so if you think back to when cloud first arrived, business people just took out their credit cards, they swiped it, they set up their own services in the cloud and IT teams got bypassed. And so AI is a little different as we've heard, you know, so you have to have oversight for ethics, privacy, bias. You're trying to access your data everywhere. And so what we're doing is we're giving IT teams a leap forward by putting all of the pieces together, sized calibrated, integrated. So, but why would private cloud be the right model for Delma for enterprises to address generative AI? So I've read a few articles, Neil and I read a lot, you see, you see that and you know, and it says we'll make private cloud cool again, but private cloud has always been pretty cool.
And so, and now the rest of the world knows it that architecturally, you can't actually have everything in the cloud for all workloads. And, and we've been saying this for a while that actually anytime you look at a workload, you have to think about the right place for it and because all workloads need data, but especially AI. AI has to go to where your data is and for the enterprise, most of your data is in your enterprise. And so that's why you have to have your private cloud in your enterprise On-Prem and you need to build your private cloud to access the data in your enterprise. And then you want it to be easy to manage, easy to use. And then you want to optimize for AI workloads.
You don't want to take any private cloud and just use it for your AI workloads because you're gonna have a ton of work to do. So that's what we built with NVIDIA. So by doing all of that work of sizing and integrating our customers don't have to figure out how to do all of that work themselves. And it's more than a blueprint. It's an actual integrated turnkey system. Yep.
So when you would counsel our audience here on what they should be thinking about when they're trying to succeed on the AI journey, what should they be looking for from a solution and the solution provider? So, I mean, what you want to do is my personal opinion is you want your supplier, your vendor, really your vendor to be your real partner. OK? And you want them to do the systems integration for you. OK? You want them in a private cloud to give you your pre-integrated, you want to pre-integrate your NVIDIA software, your NVIDIA networking and the GPUs, like we've done, you want to pre-integrate your servers, your storage and your software.
And so what we've done is we've taken all of that. We've put it in with the GreenLake cloud, the orchestration and the experience that you're used to with our GreenLake Cloud platform. OK? And so, and the real work has gone on where you can take this and you can deploy it in three clicks.
And so, and then you're ready to run out of the box. That is tremendously powerful and not only can you deploy it in three clicks, you can have your models up and running in 28 seconds. You can see this on the floor today. And so if you think about that is a tremendous time to value and you don't have to worry about doing all of this work that usually takes people an inordinate amount of time. And the second thing is is that when something goes wrong now, you have to figure out what's wrong instead of that, you have four sizes, you pick the right size for your problem and we will help you with that and then you're ready to deploy and you're thinking about the business problem that you're trying to solve.
And not about how you put together these lego blocks from a technology event. So you're not spending all of your time curating all of these software stacks and integrating all of that hardware technology, you can go focus on your use cases and creating the value and getting the productivity gains from generative AI. Right. Isn't that what it's supposed to be about Neil? That's the idea. Right.
So if we think about how our customers and our partners can deploy private AI and get to those outcomes, how would they go about it? Well, you know, first of all, it's really about thinking about the use cases that you're trying to go about. And a lot of us, you know, all of us here, we're all going about looking at our different use cases. At HPE, we're thinking about our use cases. And a lot of it is all around productivity these days. So first of all, you're trying to take it advantage of, you know, if you've got a customer call center, how quickly can I reduce the time that I answer the call? You know, and second of all, then what other use cases do you have? And so you go about that, but then you go back to what's the simplest way that I can go about it? And that's, that's kind of what we're here for.
And if we, and if we move on and we think about everything that's involved here, it's, it's all about all of the, the set up the inventory, the IT the, the asset life cycle management of all of this all the way up the stack. So this isn't just about a reference architecture, bolting together some systems and then you have to do everything on top all of that's provided including NVIDIA AI enterprise tool chain and NIM integrated with all of the great capabilities you have on observability and life cycle management. So what we did was we took our private cloud control plane that we had and we've worked on all of, you know, the intellectual property that we put together was all around the set up the inventory, the IT asset management pieces that you said the workload provisioning. How do you get, how do you deploy a container? Everybody here who's in IT knows that deploying a container sounds easy, but it's actually not that easy. OK. How do you get that done in a click? How do you, and then once you have it up and running, how do you monitor and observe it? And so how do you do that with a simple dashboard? And then how do you do that actually with a copilot? So that in the morning you can come in and you can just say that I have a problem overnight and it'll answer you, you know, with a natural language answer and says, you know, don't worry about it. You didn't have a problem. And so that's the power of this system.
So, so that sounds like an amazing demo. Can we, are we in a position to actually share that with our audience? Well, Neil we are, but it'll be tomorrow and it'll be on the stage here tomorrow at 11:30. OK. So 11:30 tomorrow, you've got an opportunity to come see much more detail around that in use and see the interfaces, see the experiences end to end because that's really what's unique about what has been built here and in collaboration. I want to thank you and your team, the NVIDIA team again. But you know, this collaboration we couldn't have gotten here in the speed that we got it.
And really, I'd love to have everybody join us here tomorrow to see how simple, how simple and useful we've made the deployment of generative AI into the enterprise. But thank you so much. Thank you forward to seeing you tomorrow. Thank you. Thank you, everyone.
So you've heard our guest speakers talk extensively about AI governance and to do a little bit of a deeper dive here for a few minutes around privacy and governance in the enterprise. We're going to bring out Joey Zwicker, who's one of the founders of Pachyderm, which was acquired by HPE in 2023 and is now the GM and Vice President of AI Solutions Joey, I'll take a few minutes and then I'm going to walk you through some next steps that you can take here at the event. Welcome Joey.
Thank you, Neil. So let's do a little bit of that uh that deeper dive. So, as has been mentioned, many times both on this stage. And you'll hear throughout the week, uh We're all on a wave together looking to drive enterprise business value for AI.
And so it's great for us to get very excited about all of the value that we can bring together. I'm here to also talk about that. There are some very real challenges and risks that we need to work through and have our eyes wide open as we're tackling together.
One of those biggest overarching challenges is around managing privacy and compliance while we wanna push forward and innovate quickly on our initiatives. So let's break that problem down into a few key buckets. The first is we just have to know what actual model we're using for our applications.
If you're using an external service, you don't know what type of model might be there. You don't know how it's been fine tuned what type of data it has access to or more specifically what biases and use cases might be there as I'm doing implications with many of my customers. A lot of what we talk about is having to remind them that in addition to this, the prompts, you are sending those external services and models also is a vector for data leakage and privacy concerns. You don't know what information might be part of that prompt or what context about your key business objectives may be wrapped up in that and how that prompt information is being used. That's why nvidia and HPE have built together the ability to take some of these uh best in class open source models like llama and mix room and be able to patch them up as Bob talked about earlier into these nims Containers that are enterprise ready off the shelf that you can use. And you know exactly where you're gonna get as you implement that for your application.
Also, what Bob talked about is as you actually pull those models into your system to get a lot of the major value drivers, you're gonna wanna give it access to your proprietary enterprise internal data. And when you do that, you need to have fine grain controls over exactly what data that model has access to. And more importantly, even when a user goes to that model to ask it a question, what information is that user privy for the model to go back and answer and give them? Speaking of kind of models giving back answers, I think there's one kind of generative AI boogeyman that a lot of us talk about here, which is model hallucinations.
And one of the ways I wanna kind of set the record straight on that is that when it comes to model hallucinations, this is literally part of how these models are designed. When you ask a model a question, you are specifically asking it to give you the best possible guess for what it thinks the right answer should be based upon the vast swath of data that that model has, you know, been trained on and given access to. And that model is not magic.
That model is not perfect. It's not gonna give a perfect answer every single time. In addition, you can put a even finer point on it by saying, well, look, we have humans doing these jobs today and we know that humans aren't gonna give perfect answers every single time either. But as an enterprise, we have a lot of expertise around managing both through training and business processes, how these humans give answers and mitigating that and those same ideas can get applied very much to our models as well. So look, talking about these challenges is not meant to scare you around generative AI. In fact, exactly the opposite.
These are exactly the types of challenges around prompt engineering, data privacy, data management internally. These are exactly what NVIDIA AI computing by HPE and our private cloud AI are designed for. It starts with a foundation around HPE GreenLake where you've got a cloud like ease of use experience to get started off the shelf.
But with those lock tight data management controls. So you know exactly how that model is accessing data and where it's all being moved in your system. On top of that, we build cutting edge tools with NVIDIA plus open source best in class harden tools plus HPE software to give you advanced functionality built together to h
2024-06-26 08:36
