AI and the Open-Source Revolution | Fireside Chat with SVP of AI at AMD, Vamsi Boppana & Zack Kass
Hi everyone, I'm Zac Kass, global AI advisor and former head, go to market at OpenAI. And today I'm so excited to be hanging out with Vamsi Bopanna, the SVP of AI at AMD. Vamsi, it's great to be here. Thank you Zach. It's a huge pleasure to be here and I'm looking forward to chatting with you.
Likewise. And of course, somehow, in a fun twist of fate, I find myself today in India, of all places, where you did your studies. And you're back home. But, thanks in large part to technology, we're able to communicate, at a rapid pace, in real time. What we need to plan to cover today, at the very least, is hardware architecture, acceleration and open source and frontier in AI innovation.
And, of course, the big one, which is what's next for developers. And I can tell you now, having sort of stared at this for the last three years, it's never been more exciting for developers. But you know this better than anyone. Yeah.
No, no, it's it's a great set of topics. It's an amazing time for AI, right? You know, literally as we sit here, the future of computing is being reshaped. It's, it's an exciting time for hardware developers.
It's been said that it's like the golden age of, computer architecture by Hennessy and Patterson. But also for developers. Right. You are fundamentally getting access to compute that you have never seen in your life. Right. So super exciting. There are a lot of things that cross your plate every day.
And the things that you have to now consider are come so fast and furious that I can't possibly imagine how you keep up with all of it. But let's start with the basics, which is can you just talk to us right now about where compute, first of all, where it is compute today. How do you describe where it is today. And then I'll ask the harder question, which is where is this going? Yeah. Look, I grew up, as you said, like in India, right? I went to school at the I did computer science, studied architecture, and, you know, I got a PhD in computer engineering. And so I feel like, you know, I have seen the evolution of compute, at least through my, you know, career.
And, you know, for call it, I don't know, five, six, seven decades. Right. We have done, a compute in a certain way. It's, you know, the, the traditional von Neumann architectures. Right.
And you fetch something, from memory, you process it, you write it back into memory, and, you know, it's it's been that style and form of computing and of course, right. We've had phenomenal gains over all this period, you know, architecture, but also technology. Right. We've we've really, had the, amazing ride, on the technology curve.
In my own career, we have gone from, I don't know, 0.35 micron to now we are like, doing two nanometers, right? So I don't know how many ever sort of orders of magnitude that is. Yeah. However many, many orders of magnitude of improvement.
And, you know, F maxes went from I have seen chipset 1020, 30MHz to, you know, we went to hundreds of megahertz, two gigahertz. And, you know. At some point, critically and critically, we brought the cost down on all that too, right? That's right.
That's that's the other remarkable thing that's. Going on is just amazing, the things that have happened. But as much progress as we have made, you could argue that there is a fundamentally different way of doing compute that we have.
We have found and, are now starting to think about very, very differently in my mind. Right. In my humble opinion, it's rooted in a few things. The first is, you know, the world is insanely parallel, right? If you think about like when you when you process an image, right, you're not reading a pixel at a time.
You're actually, you know, looking at an image and processing all of that information in parallel, right? You have all these, capabilities, biologically in nature, that allows you to process things in parallel. Well, some of us, I admit, I struggle to even consider switch between simple tasks. But yes, your point is well taken. As a right now, you're being, incredibly humble that, but yeah, I, you know, the world is insanely parallel.
And, you know, there have been various, computer architecture innovations that have created parallel machines. And we are seeing, you know, those kinds of innovations really, really come to fruition in this time. It's also true that, you don't need to process things. Exactly. There's a lot of room for inexact or imprecise processing. You know, for example, you don't need 64 bits to process.
You know, lots of data. You can do just well with four bits or eight bits or even lower. So if you combine all of that, you have a, you have a very, very different way of thinking about compute, where you can take in a lot of parallel information, process all of that, you know, simultaneously, in an imprecise fashion, and actually get useful work done.
And, you know, what we're seeing with neural processing and, some of the innovations that have come through with compute associated with neural processing is really, really fundamentally shifting how you can do things, write algorithms that were traditionally written in, in ways where you were bottlenecked are completely possible now because you're able to reimagine them, doing it in parallel, doing it in less precision, and doing it completely differently with neural processing, getting, you know, orders of magnitude of gains. Something that just occurred to me, which was a conversation that I'd had a few times at OpenAI, which is this, the chicken or the egg conversation around the algorithms and the software that's driving the the evolution of the hardware, or is it the hardware that's actually driving the evolution of the, the software? And, you know, what's your view right now? There's this incredible scientific convergence, right where everything seems to be getting rapidly better. But, but what's your take here? Yeah, I think I view that in the general category of co-design. You know, we've always talked about co-design, right? For, for as long as I've done chips and architectures before, we've always done co-design. What's the workload? What's the, you know, architecture and what's the implementation. Right. So that's been central.
But now today more than ever co-design is so, so important. I'll give you 1 or 2 ways, why it is so right to think about it differently. If you, if you, if you did not change your fundamental data type right. Then you can think of co-design as like, hey, you know, how much prefetch do I need? You know, do I detect branches in a certain way? Right. You know, model the workload and you're getting, you know, 5% more, you know, ipc your throughput through your machine. Right.
And that's important. Super important. But can you imagine where okay, I have a choice now whether I can implement this thing in six bits or four bits or eight bits.
And you know, what does the machine look like? You could suddenly now have the, choices have, implications that are, factors different, right? Not percentages different. And when you have such big differences, it's so, so, so important to really cool optimize and core design the algorithm workloads in the machine together. Speaking of choices, let's talk about one of the things that you and I spoke about recently, the criticality of choices, especially for, software developers and the the front and center question now, which is open source versus closed source. And I think, you know, spoiler alert, I think you and I agree on on where the world is going and should go, but let's talk about that.
What is what is the future of of AMD and open source look like? Yeah, we we are we are extremely passionate about our commitment towards, openness, and open source building open ecosystems. And it actually permeates through everything we do from having open choices and enabling open ecosystems on the hardware side to open choices and, software openness through our open source and ecosystem efforts, but also just in how we interact with our customers. Right. We are we are a company that, really, really, passionate about delivering exceptional customer service and being open enables us in unique, unique ways. So let me give you maybe 1 or 2 examples, on the hardware and the and the platform side. Right. You know, can you imagine going to a customer and a customer is being told that you've got to use this exact network choice, when they really want something else? You know, there are customers that absolutely love the fact that we will go to them and say, like with every we have an open architecture.
You able to plug in your choice of networking. If you look at software again, right. Everything that we do, from libraries to, our, ecosystem components that interface with frameworks, for example, they're all out in the open and it's, it's a, it's a choice that customers then now have to be able to innovate on top of the open components that we provide. And then, if you look at how it manifests, for example, in customer interactions is many times we actually work together with customers on open repositories where we collaborate in the open, where customer innovation and driven innovation and ecosystem innovation all coming together to fuel innovation at an extraordinary pace.
That is not possible if it was just being developed in silos. There are a lot of straw men, and there are some oak steel men that support the closed source and and the closed architecture environments. The arguments that you hear, what do you think? What do you think are the strongest and and what do you think are sort of the most misplaced? So, look, I think there's always a, you know, a place in the stack, call it right where you may have the, the need to be closed. Right. There could be, you know, significant proprietary innovation. There could be, requirements from an applications perspective, let's say security that prevents you to, you know, from, from disclosing certain things into the open.
So those are those are areas that obviously we care for as well. And, you know, we make those choices. But, anything above those considerations, we tend to lean quite heavily into the open. I would agree.
I think what's fascinating to me right now is whether or not you can make an argument that any given company needs to open source. The fact is, the world needs open source in order to move the science forward. And that building in the open, I think we can now sort of confidently say is one of the great ways that we deliver better, faster, cheaper goods and services to more people at lower cost. And this is one of these incredible truths that's starting to play out in real time and is evident sort of in part by things like, you know, we've seen, in from from Deep Seek and Meta and others who, who are moving not just, behind, but actually at the, at the state of the art.
That's correct. And that's fueling a tremendous amount of innovation alongside. Right. If you look at llama, you don't think of llama is just like, hey, here's an open source model, right? You think of a whole ecosystem that rides on top of that, whether it's, you know, frameworks that support llama, whether it's optimizations and, you know, whether it's even like, you know, hardware silicon providers that now are able to look at these models and create the next wave of hardware innovation that works well for these models. Let's talk about the, open source frontier and, and, what what are you seeing and what are you what are you encouraging from your team? What are you encouraging from your partners? And, and sort of what do you think is, is in the near term for the open source frontier? Yeah.
As I said, you know, and the, things of open, development, open ecosystems and open source, fairly expansively. Right. We look at it from a hardware perspective, from a software perspective. And also, you know, in terms of how we think of open interaction with our customers, on the hardware and the platform side.
One of the areas that we have been, driving a significant amount of open innovation is in, the networking space, with respect to the standards, that we, we want, the industry to adopt, we are donating things out of internal development into the open, but also then looking for the industry to partner with us. We are collaborating with the industry on things like rack standards, you know, things like, data types. That we want the industry to, to, to collaborate with us on. So there's a, there's a number of factors on the hardware and the platform side. And then on the software side, which is where I think quite frankly, a lot of, a lot of our, our, development, has been focused on and a lot of my personal passion also lies is, is, we have been, tremendously, supportive of significant initiatives that have helped the AI developers, like, we really strongly, support the PyTorch ecosystem with SES that we support the development that we support around that ecosystem.
But also other frameworks, like Jax, for example, we support model hubs, like hugging face that, you know, we collaborate with also inference frameworks like via the Lem Lang. We are very, very close collaborations with these emerging open source communities that, that are thriving in the, in the space. So a number of these, initiatives is what I would point to. So given that we're seeing improvements in the software, the hardware, the research itself, and given that even, you know, you and I couldn't we could spend all day, every day trying to keep up, but we wouldn't, with everything. Then maybe you can, but but I couldn't.
What's your advice to developers for keeping up with the pace? For those who are trying to build, you know, in the open ecosystem, what do you have to say to them? Yeah. Look, again, I think at the highest level, I feel like it's such an exciting time for developers. Right. There's just so much more that can be possible with, today's compute. I'll give you an example.
Right. You know, many, many years back, right. As I was playing around with the first neural networks, it was I, a simple convolutional model that I was trying to train. And it was it was not easy for me, to actually, you know, create my own sort of backbone and train a network, right? Because where could I go? Right? Where could I go to get the compute? But now if you if you were faced with a similar situation, a developer that wants to do something and of, you know, build a, a model architecture that you would want to try out, you have so much compute that actually allows you to do is think about, even in the last seven, eight years, how much improvement has happened in the amount of compute rate, it's, orders of magnitude more compute that developers have access to. So it's an exciting time, right? You can you can experiment, you can unleash innovation.
You can you can do that. I would also say fundamentally right, developers can think about their work differently. In the past, it used to be like mastering the two and mastering some things, you know, that may be, you know, limited in, in certain ways, but now it's like you have a whole ecosystem to collaborate with. Right?
You can say, hey, you know, I want to put together a project and you can say, okay, I'll take various pieces of stack that are readily available in the open community, right? I can take if it was on AMD, for example, I'll take it's all in the open. I'll take, an inference library. Like real. It's all in the open.
I'll take a set of models like Lama. That's all in the open. And then you can stitch all of this together very, very, very quickly.
Like you can pick one of our dockers that goes ships every two weeks. And you can be up and running serving and load them in literally minutes. Right. It's not hours. And you can do that on an on a cloud with public access to extremely capable compute instances. So it's, you know, taking advantage of the ecosystem and really being sort of, agile.
Understanding where the ecosystem is moving is, is one, one piece of it, the other one that I also I'm personally very, very excited by is I think it's going to be, an age where I will assist developers, significantly. Right. If you look at the fresh, college graduates they're recording these days, all have assistants sitting by their side, right? Even as they type the next set of, you know, code is getting generated, but something. And not and not just the new software, I mean, I the reports are that lots of, you know, very seasoned software engineers are starting to adopt these tools as well.
That's right, that's right. And we see that internally. Right. We have projects that are coding assistance and, you know, able to solve sort of deep problems. So it frees up developers from from called quote unquote some of the more mundane. And then it allows you to really focus yourself on creativity. Right?
I actually, can recall a super interesting conversation I had with, you know, a professor at Stanford. He runs a venture fund. He has 30 companies in his portfolio. When somebody comes and pitches an idea to him these days, he was, he would tell them that, hey, don't pick just one idea. Pitch six ideas.
Right. And why? Because you can actually prototype all of these relatively quickly. One of these can actually make it right.
So why are you limiting it? So what's your take on GPU and CPU is there. And you know you and I talk about this, but my my my sense has always been that there's a, there's a reasonable chance that actually we're, we're discounting or underestimating how much how much role the CPU could play in this, or, or is there another computing unit that we're not yet familiar with that that could play a part in actually serving a lot of the AI? Yeah, that's that's a great question. And I think it's also clear that, AI is going to get more and more efficient as you, as you innovate. Right. So, today's largest model capabilities will be will be subsumed by tomorrow's smaller model capabilities. And that trend is continuing on.
And so as models shrink, the energy requirements come down and you get efficiencies that will make a much more pervasive. So we believe in that. We see that as a trend. And then, you know, coming back to your question on like how do we see architectures? We, I add and actually subscribe to this philosophy that, that the, AI is going to be everywhere, right? It's going to be there in the highest performance GPUs.
It's going to be there in CPUs with specific instructions that accelerate AI. We were also the first to drive a dedicated AI accelerator onto an x86 platform. And those are tiny, tiny accelerators, extremely high energy efficiencies. And we have, embedded platforms where we have, you know, from cars to communication networks to NBA games.
You know, AI is is getting infused. So, so we see a world where AI is pervasively infused everywhere. However, I would say that for the highest performance, capabilities and where the, the, the largest amount of innovation needs to happen and is happening in the near future, you do require the general purpose capabilities of a GPU that remains the primary workhorse. And, you know, people forget, right, that, you know, GPU is a graphics processing engine, right? You know, it's at its heart, but today's compute GPUs are hardly graphics engines.
They are actually general purpose compute engines that are that are incredibly valuable for doing intensely parallel neural network processing, with the rate of change and algorithmic innovation, that is, you need the general purpose capability of a GPU. And at least in the near to come and call it the next 3 to 5 years. That is expected to be the workhorse. Speaking of the next 3 to 5 years, let's let's look forward a little. So obviously you recently released, Rock, 6.4. Congrats on the release.
Thank you. What does it mean for today and what does it tell us about where AMD is is going? Yeah, we've been, on this, on this, you know, big push to ensure that we deliver the, best in class experience for developers to our tooling and our software and, you know, our journey has been also to partner with the open community. So, so we're continuing on that. If you looked at where we have come from, we've come a long ways in the last, you know, 12 to 18 months. Right. When, when when we first sort of announced our AI products, in the GPU space and, you know, you were able to pull down a rock, that runs on these and said, hey, you know, what might my experience be? I don't think we could have said that. Hey, any model
you run, your experience would be this. Right? Because we didn't necessarily have all of that. But in the last 12 months, based on the work we've done with partners like Huggingface or PyTorch, I can say extremely confidently that, you know, any model, you pretty much run out of the box. So that is a reasonable, statement we can make because we've now tested, right. That there are 60,000 backbones, for example, that we test nightly with hugging face that guarantees that a million models that are hosted there are going to be able to run.
So we've come a long ways right, to be able to take workloads that are just any workloads and running on our platform. What is our journey now? Moving forward is to ensure that the latest innovation, with respect to performance, with respect to the new algorithms, is coming as fast as possible on to the AMD platforms so I can kick start for, you know, practicing a number of features. You know, at multiple levels, whether it's libraries, computer communication or, you know, basic things like driver support for new products that we are putting out. So it packs in a whole set of capabilities.
And then as we look forward, we always support our next generation hardware. So we've announced that our, our next set of products, MIPs 55, which will be coming at the middle of this year, you know, we'll have software support for that. It's again introducing new capabilities like for bit precision. So new algorithms, new libraries to support those, increased focus on communication as we are expanding our footprint of use cases, that require distributed inference and, training, support.
So lots of exciting things to look forward to with problem. So one of the reasons I'm in India is to visit and meet with a bunch of clients here. And the question, you know, the question I've always gotten is how do we, you know, let me stay ahead. And now the question is, have, you know, the people are asking us, how do we just not fall behind? Right. This is this is the question that that enterprises are asking as they start to feel like the startups, the startup ecosystem, the folks that are adopting your infrastructure, your platforms are moving much faster than theirs.
What is your advice to businesses for future proofing their tech stack? For, for, for, you know, large scale AI integration. But let's just let's just call it business evolution. Let's just call it what it is.
I mean, it's it's the new normal. Yeah. I grew up in India, so I can definitely relate to, to some of the thinking and thought processes there.
Right. So, I'll give you maybe an, a, a, you know, interesting way to think about it. When I, when I grew up, this was in the mid, you know, late 70s, mid 80s. Right. It was extremely hard for India, to provide telephone connections.
Right. This was, you know, landlines that were getting deployed. And so it was it was quite the joy when we first got our landline at home. I remember that right.
And it was a long wait. You had to wait. You had a pie and it was all by allocations. And even, you know, we were amongst the more fortunate to be able to get that the vast majority of video never had landlines. Right. But then what happened was, mobile came along and, you know, you can see every Indian has a mobile phone now, right? And what I what I think when I think of that, what happens is it's a, it's a generational leapfrog, right? You basically skip a bunch of technology in the middle and just leapfrog to what is what is, the latest and greatest.
I think that kind of an opportunity exists with the AI. You don't need to tread all the path that has been read. You can catch up. Right? And in, in a geeky way, I can think. Right, like, okay, you don't need to understand, you know, applications that were traditionally solved by ordnance and CNNs when the best transformer architecture basically solves it.
Or you can just pick it up, right. You can download the latest model, which is a Transformers model, and it works fine. So that sort of opportunity exists. I would also say it's a it's a very unique time because a lot of people grew up in the AI programing. It's very close to matter, right? Writing, you know, painful assembly and, you know, detailed, you know, C kind of code.
But then the way programing environments and abstractions are being, evolved right now is it's giving rise to higher levels of abstraction. And, you know, you can program in Python or you can program in block level programing abstractions that are way, way more easier and more efficient. And you're able to get to innovation much faster. And then from a pure business point of view, I think the a lot of focus in the, in the past few years has been, you know, on creating the models and creating the innovation, creating the intelligence.
And we are now starting to see the shift towards, leveraging that innovation to serve and to build applications. I spend a lot of time now trying to tell people that we are approaching a future of unmetered intelligence, this idea that intelligence will be free, and we need to start thinking about differentiating collectively and individually on the basis of how smart we are. And I, you know, have to make some some pretty bold assumptions about the speed at which I think are somewhat safe. But but are still assumptions about the speed at which the inference will improve on both, performance basis, but also on a cost basis. What are the trends that you see? First of all, what do you think? And, what are the trends that you see informing our path to unmetered intelligence? So I am I am, with you in, in, in the in the way you see it. Right? We are on a extremely steep slope of driving down the cost of inference and, you know, making intelligence more pervasively available.
I was talking to one of the foremost AI researchers in the world. You know, they are creating the next set of AI innovation that they believe will be in the next 10 to 20 years. Amazing amount of new drugs will be created, right? I was talking to another professor friend of mine at Sanford. He said analytics are amazing for the long tail problem, right? So it's just like there are many, many disciplines where the long tail problem is extremely hard to solve. And if you can clean that up, you can transform factories, you can transform automation, you can transform so many industries. So, I think as we drive down the, the, costs and as we make intelligence more pervasively available from the cloud to the edge to the endpoint, I think the opportunities to create disruptive products and innovation just is on a different curve that keeps rising.
Yeah, I agree, my, my father's an oncological researcher and I'm always reminded that there are only 10,000 oncological researchers in the world. Right. It's a very big problem. But it turned out when we're born, only a certain number of those people qualify for the job.
And so we're all sort of sitting around saying, hey, when are you when you are going to figure this out? That's right. And we suddenly have a chance now to say, hey, well, actually we have we can supercharge you, right? What if you know, what if it's 10,000 researchers behaving like it's a million? That's right. I mean, if you if you look at just to just to catch that thread, if you look at a lot of the new drugs that are being created. Right, it's the discovery of the magical sequence of, you know, proteins or DNA sequences. Right. And can you imagine if all of this data is being processed by, you know, the right model? It's it's almost you can see it.
It's going to happen. Right? The new class of drugs is going to emerge. In closing, what do you what what can you leave us with? I haven't felt this excited in my entire career, that we are living in an amazing time. I feel that way, both from a technology intellectual fulfillment perspective, but also the possibilities, to improve, you know, human life at large, right? That come with it? It's just such an exciting time.
And I find it like the privilege of flights, to be able to be here at this time, do the work and make an impact. I, I second that I, I remind myself and anyone who will listen to me constantly that today is the best day ever to be born, you know, and tomorrow probably will be two. And that trend so is no signs of slowing.
And despite all of the things that we perceive as being wrong with the world, irrevocably, in some cases, in fact, they aren't. And in fact, our modern history has proven that the world does get better, thanks in large parts of technology, thanks in large part to the work that you all are doing to help, good actors do more good at a faster rate. And it certainly seems to suggest that we are on the precipice of some incredible step functional improvements in the human experience. And, you know, if you and I and our optimism are right, then it could, you know, could be pretty magical. But anything else you want to leave us with? Yeah. And finally, for our developer friends, come check us out at our advancing AI event in June and stay tuned for regular updates at the AI at Amdocs handle.
Absolutely. Thank you so much. We will do this again. And, in the meantime, I look forward to seeing all of the work that, that you and your team are going to, are going to produce. Yeah. Thank you so much, Zach. Really enjoyable conversation.
And I'm looking forward to the next time.
2025-05-20 13:35
