welcome to Tech unheard the podcast takes you behind the scenes of the most exciting developments in technology I'm Renee hos CEO of arm at arm we're shaping the future of computing with the industry's most powerful and energy efficient compute platform designed to unlock the full potential of AI our technology is at the core of innovation for leading companies across the globe in this podcast I'll be sitting down with some of the brightest Minds in the industry to share you insights stories and Visions for what lies ahead today I have the privilege of speaking with Jensen Wong the CEO of Nvidia a true Visionary my former boss and a personal mentor of mine we're going to dive into his journey the future of AI and how nvidia's unique culture of Relentless Innovation and ambition continues to push the boundaries of Technology we sat down and met at invidious headquarters in Santa Clara to talk [Music] ready to go I was ready the moment I walked in it's great to be back well thank you yeah it's great to be back it's great to see you it's great to be back here at Nvidia uh this building did not exist when I worked here many many years ago uh how many how many years ago now 20 uh uh I started in 2006 I left in 2013 yeah see 20 years ago yeah 20 years ago these buildings did not exist it's a it's a nice feeling to be back though familiar thanks for uh thanks for spending the time thanks for having so now that you've grown so large one of the things I've always been curious about Jensen with with Nvidia is hiring uh the culture is one of a kind uh company does things in one a kind way how do you identify folks who are going to be successful inside Nvidia we're not always successful in doing that look how you turned out it's it's always a shot in the dark um I think that the interview process is is not an excellent way to judge whether somebody's a good fit I mean obviously everybody could pretend to have a very constructive conversation you could learn a lot from just watching YouTube on how to interview and so you know the technical questions of course people even share what Nvidia technical questions are and we try to be as rigorous and difficult as possible but but it's hard I I think that my method is is always I go back to reference checks you know and I asked them the questions that I was going to ask the candidate and the reason for that is you you could always make make for a great moment but it's hard for you to run away from your past and so I I I think those are good I like asking one in-depth question and and just thinking about how they reason through it and but I think in in the final analysis Nvidia has been successful for a lot of people our attrition rates very low as you know and and it's a really diverse environment and with with a lot of really interesting people in background and we have people from from just about every great company in the world and somehow we've made them successful here and so I think that that on the one hand building a great company is about getting great people on the other hand building a great company is really about creating the conditions by which those people could do even better than they thought they could and you know a lot of that has to do with being transparent about explaining what nvidia's vision and strategy and and and what makes us work as you know I spent a lot of time doing that and uh our company has always been known for its transparency about explaining what what challenges we have what opportunities we have what strategies we're executing and information is Flowing uh fairly readily inside the company with respect to you know what what is it the company strategies are I always find that it's strange when companies have too many silos and you know need to know basis I think obviously you know people don't need to know what they don't need to know but the more that they know the more they're empowered to be able to make good decisions on our behalf and so I I tried to air on the side of transp Arcy I try to air on the side of empowering people and as a result you know the company is one of the I think we're the smallest large company in the world for sure you know I think that's just that comes with the the incredible productivity of the people and we got 30 30,000 people or so maybe a little bit more than that now and they're making hundreds of decisions a day and if all 30,000 of them are you know statistically moving in the direction making decisions that are in ambiguous decisions often times but they're making it in in the direction of of what is in the company's best interest long term it adds up really fast one of the things that always amazed me is that um you know back to that point and again I don't know whether it was hiring the right people or self- selection but by having senior leaders who are extremely comfortable with the ambiguity and the fact that you would reach down into different layers of the organization I.E the project is what's most important uh I just wondered how how did that happen it did it's just something that as you grew the company and had senior leaders who were aligned with your vision that it grew up that way because it it's just amazing that so many of the senior leaders here when I worked with Nvidia or worked at Nvidia they were completely fine with the fact that you would just reach around and get the right people in the room to solve a problem well first I didn't ask them as you recall I recall and the reason for that is because you shouldn't have to ask permission for something that is that obvious you know and so the reason why we said it that way is that Nvidia was designed to be a full stack Computing company was we were designed to be a company that would build gpus and CPUs and networking chips and switches and um we would do architecture and design of chips and develop system software and create algorithms and even you know create solvers and so how would you organize such a thing where everything has to work together on the one hand but you have to build it in Parts on the other hand yeah and so the way we solved that problem was instead of having organizational silos we thought of the organizations as a place where the leaders can groom people create conditions for them to succeed be of service to them to help them remove obstacles and such but the mission the mission is the boss and it cuts across the whole company so it can cut across systems and chips and networking chips and software and algorithms and it can cut across all kinds of domains and by organizing that way we also created transparency you know the all these silos became porest and when organizations are porest it tends to be better you know because you have a lot more people who are able to help you criticize it you have a lot more people who help you improve it and so I I love this the paracity of our company if you will I I just love that everything is transparent and everybody's helping me make it better and you know it's not like everything is in some kind of you know Dark Side you you almost acquired us which would have been would have been fun but you acquired melano I know you you're still sad about it I'm still sad um every day I I cry a little bit but uh but I'm here thank you but you guys have done so well but you did acquire melanox which has been not only an amazing acquisition in terms of your strategy but it also just seems like seamlessly to your point of a porous organization where the mission trumps everything it's it from the outside in it looks incredibly seamless in terms of execution how did how did that happen I mean how how did you make that so I mean m&a is so tough it is it is cult very tough yeah it is tough well first of all uh there are 10 people I think maybe more 10 10 12 people on the melanox management team the Nvidia Israel management team that sits on easta that's great we have architecture we have research we have software systems the chips we have Nicks and switches we have mvlink switches now we used to have uh Justin finan product line but now we have a whole ethernet product line in the short time that we've been together the product portfolio of melanox well quadruple and they're integrated into every aspect of everything we do uh if you look at the transformation and you recall the acquisition our vision was that the unit of computing was no longer going to be for example a GPU which is really a peripheral arm helped us in fact quite importantly to transition into a company that was building an S so now remember what an S so is an S so is basically a whole computer y whereas a discrete GPU is is the last thing that comes up in the computer the CPUs come up the boot ROMs come up the operating systems come up and eventually the GPU comes up in the case of an S so you have to bring the whole thing up yourself and so it costed Nvidia to to evolve from being a algorithm company which is really what a GP company is to a Computing company that was our first entry and the so wasn't easy for us in the beginning we built some amazing ones now and then the next Evolution for us was Building Systems and djx1 was our first in fact I'm still quite fond of of Shield which is our Android TV computer and I'm very fond of it because it it was really nvidia's first full system that we we created in the learnings on Shield must have been amazing not looking back I remember when we started that yeah it is still the the most popular Android TV box that people back in the day it was a Playstation Xbox controller with a display and we were just thinking to ourselves how do we do this yeah it is still my favorite thing that Nvidia has ever made I yeah I completely forgot about that yeah that learned L were a good old days that was a system yeah I learned a lot yeah yeah I learned a lot and to this day we're still maintaining the software yeah it was utterly unobvious that there was a fit in the marketplace for this and I remember folks inside the group suddenly having to Source a whole set of components and and a bomb that we had no idea exactly it was my excuse to turn Nvidia into a systems company yeah and people will ask me you know the djx1 which is the the computer that changed everything right right well you know how how did that come about well djx1 is just a very large Shield it's a very large Shield yeah yeah exactly and and so to to me the fact that Shield was made out of plastic and djx1 weighs 600 lb you know that transition wasn't a big deal the the big deal was that we were now able to build systems and and then when when we bought melanox the big idea was that the computer was no longer going to be that node but the computer is going to be the entire data center that the data center is going to be the unit of computing right and if you don't if you don't design GPU the CPU the Nick the switches all of the transceivers and connect everything together and be able to boot that system up you know from nothing and get everything all wired up get everything all running and distribute workloads across it if you don't do that you're really not going to understand what it means to build these AI super clusters and that transition that that Vision was so clear that it it was necessary for galvanizing the two teams you know in order to Galvanize teams you have to have a very clear vision and we had a very clear vision and that that Vision was also very tangible because you could see it right you know sitting right in front of you there's super cluster and got all the gear from both companies and and so the the the vision was clear and inspiring it's tangible or we have to make it tangible CEOs you have to make abstract things tangible um and and we went off and built it and and so anyways I also think that their culture is great yeah and that Clarity really helps but going back to kind of the vision thing for a second and this's another thing I when I tell stories about the company Shield's a good example Cuda in the early days chasing oil and gas is is a good example where it's completely unobvious yeah people didn't realize in fact that was our first that was the first completely unobvious what the um the real end quote killer app or or endstate is yet you have an incredible resiliency to experiment with ideas early and test them even though the market doesn't either appear ready and or there even a definition for it were you chocked that up to is that incredible intuition is that seeing around I mean what you we we've had good intuition you know 10 times in the company as you know and the benefit that that Nvidia has is we're surrounded by extraordinary people I mean you know yeah these are these are the finest computer scientists to finance strategists and business people in the world and they're egoless and they want to do great things and and so I I think that that one uh we start with that I think the second part is we're good at intuition I think we're we have a good intuition about what problems need to be solved and how to get us from where we are today to becoming the company we want to be and so I think our intuition is good about what the what the the various stepping stones are and you know each one of the things that we did a lot of I was asked you know why are we building Shield I mean what a waste of time and and I said we're going to be a systems company someday and and all these systems are going to be connected to cloud services why go break our pick on on the largest systems why don't go do this one first and if we can't do this one we're not going to do the large one and so to to create the conditions where the company could go go learn some new skill skill fail but not not damage yourself you know and so can that only happen at companies where the leader is or was a Founder because again very very few companies do what you just described both in terms of being having Clarity the vision but also resiliency to continue to understand where to go is that there's been a lot written recently about founder mode versus manager mode and obviously your founder leading a company 30 years later it goes without saying the amount of success you've seen but can this only be done what you descri described by the founder leading the company I I don't think so I think you're doing great at arm um you know when I watch you do your work I'm very proud of it many well I learn I learned I learned from you which is not which is being truthful and I love watching you do your work and and it makes me happy get brings me great joy and pride I I don't think so I think that it is true you have to have uh great resilience and you have to have perseverance and and I describe it as pain and suffering as you know and you know teaching moments yeah pain pain and suffering is how you feel I felt it yeah and in a lot of ways you have to get used to it you have to you have to get used to the idea that there's pain and suffering involved and and you know that that the journey to success is not about one achievement led by another achievement and another achievement it's not like that you know there are there are big setbacks sometimes there embarrassing moments you know when you're a CEO and and um you you haven't you haven't enjoyed any of that yet but but uh it'll happen well I hope it happens because it'll be good for you yeah absolutely um but you know all those moments are are um I don't know what I learned from it but it made me stronger you know and and I know I could survive it uh I know I didn't like it at the time but when I look back on it there those are the moments where you you that's right you're the most proud of yourself you're most proud of your company and that you survived in and so so I I think the company our company is strong because we have lots of stories like that yeah you know in the halls of this company are are just are just filled with extraordinary stories of one setback after another setback after another setback and with many leaders who who went through it yeah most of them are kind of like oh this is a nearly as bad as when that happened right you know every time something happened it's like ah it's not this is nothing so the the ability to to be able to go directly to remember when that happened this is nothing and yet this is incredibly painful yeah it's it helps the company move through these challenging times so you and I have been around this industry about the same amount of time and some of the stuff that's going on with AI I know I feel this way were things that I just thought I would never see that the future generation would be able to experience the kind of transformation that seems to be taking place it feels like to me not to sound star treky but this is the final frontier in terms of I can't imagine what is next beyond what we're seeing with artificial intelligence broadly how do you feel about it are we accelerating so greatly the transformation of an industry that we've never seen before is there anything next after this it's just unbelievable what we're saying I guess I've always expected that computers would demonstrate intelligent behavior that we would be able to write software so well and thought we would write it that algorithms would eventually solve problems in a way that seemingly the computer is intelligent I never thought that it would result in Industrial Revolution and what I mean by that is and you've heard me say this that for the very first time the computer industry has now transcended beyond the the traditional computer industry that for the very first time we're now no longer a tool an instrument but we're now a manufacturing industry and so what I mean by that is you know right now while we're talking our phones are in our pocket it's not being used and when I'm not using it you know not using this tool it's not doing anything for me and most computers are that way my laptops in my office is doing that most people's computers are that way if you need that tool you go use that tool however this new industry of AI factories which is what we're building now they're running all the time whether you're using or not they're producing tokens they're ingesting data they're producing tokens they're generating intelligence intelligence is being manufactured at a very large scale and the idea that this computer used to be an instrument a tool is now a factory a manufacturing thing and that is producing incredibly valuable things at very large volumes and so this is a new time for our industry as this has never happened before and the idea that computers are now the manufacturing instruments Machinery behind this incredible thing called tokens you know intelligence tokens is just an extraordinary idea and so we're at the beginning of a new Industrial Revolution is it racing faster than you thought it would you've been closer to it probably than anyone with with Alex net and djx1 and have seen the pace of innovation from where I sit and we've been looking at it inside arm quite deeply since I took over it has gone far faster than I would have imagined two and a half years ago far faster imagined even a year ago you're involved in everything around it is it moving even faster than you imagined no we're trying to make it go faster we've gone to a one-year cycle and the reason for that is because the technology has the opportunity to move fast for sure and because we are now not just building chips and we know that the rate of progress of chips anymore if you're lucky with a new process node to get a few percent that's incredible and so how do we get X factors of performance with each generation well the way we solve it is we designed six or seven new Chips per system and then we use code designed to reinvent the entire system and invent you know new things like mvlink switches and new system racks that allows to drive copper across the entire back spine of a system to connect all of the gpus together and very large packages and 3D packages and such we're using all kinds of techniques to to do that as a result we could deliver two to three times more performance at the same amount of energy and cost every year and that's another way of essentially reducing the cost of AI by two or three times per year that is way faster than Mo's law and so you compound that over right five six 10 years we're able to drive Incredible cost reduction for intelligence and and the reason why we're doing that is because we think that this is at a time when we all realize the value of this if we can drive down the cost tremendously one we could do things at inference time like reasoning you know today when you use chat GPT which is an amazing service I use it every day I used it this morning you hit enter and your prompt is loaded and it generates the the output but in the future it's going to iteratively reason about the answer it come up with a tree search maybe and maybe it does some kind of iteration and reflect on its own answers and eventually it produces an output it might have gone through a hundred a thousand inferences but the quality of answer is so much better yeah we want to drive the cost down so that we could deliver this new type of reasoning inference with the same level of cost and responsiveness as the past I have seen a demo of the open AI model that does reasoning and it was it was shocking to your point it it went through a logic tree you could see the tradeoffs it was making simply the way a human would yet at a pace completely unlike the way a human would but then as you fast forward and this is what's so fascinating to me about this what's going on now is it exact to your point you're introducing systems full data set or infrastructure at a pace the industry has never ingested at before CPUs bought every two or three years ultimately depreciated Now You're Building Systems on an annual beat yeah people want to pay for those systems and deploy them as fast as possible right now where we're talking it's so easy to say but you know we're delivering new computers that are this room size each year yeah all the cabling all the networking all the switching all the software it's yeah it's really quite insane do you see it and I I'm not ask you to to forward forecast but this is more just a technology ingestion question can it continue at the current Pace yeah I think so but it has to be done in a systematic way in the sense that everything that we do we do in an architectural way and what that means is that the software that you develop for yesterday's clusters like Hoppers and that software is going to run on Blackwell and that software will run on Reuben and the software that you create for rubben is going to run on Hoppers well this architectural compatibility is really quite vital because the investment of the industry on software is a thousand times larger than the hardware not to mention no software ever dies and so if you develop software you release software you've got to maintain the software as long as you shall live and so the architecture compatibility that the idea of Cuda is not that you know there are millions of people programming to it the idea of Cuda is that there are millions of gpus several hundred million gpus that are compatible with it yeah software doesn't die software doesn't die yeah and so whatever Investments that you make on one GPU you can carry forward to all the other gpus and all the software you write today will get better tomorrow all the software you write in the future run in the install base and so number one we have to be architectural and really disciplined about that second even at the system level we're super architectural now we'll change pieces of the technology to advance system design without you having to leave everything that you did yesterday behind and so for example you know when we first came into the data center business a hyperscale data center had power distribution that was like 12 kilowatts per rack well black Wells 120 kilowatts per rack it's 10 times 10 times the density now of course it's 10 times the density and it reduces millions and millions of dollars of servers and compressed it into one rack and so the amount of savings Energy savings and you know space savings it's just incredible that is very similar to our story you the arm architecture has been around for 30 years and we have software that's been written for it for decades exactly and that is what people sometimes don't always everything we develop on every single arm ship we carry forward we just showed something the other day somebody did some benchmarking and Grace was four times the performance per energy per watt than the best CPU in the world and Bravo energy efficiency is vital yeah it's everything do you see anything architecturally starting to break when you go from 500 megawatt data centers to 5 gwatt data centers just relative to the network latency things of that nature without getting into proprietary stuff do you kind of at a high level physics standpoint start to see some things that start to break everything breaks physics is obeyed which is the problem but everything breaks first of course we're moving up the power density curve very very quickly exponential and so from 12 kilow to 40 Kow to 120 200 and it's going to go beyond that and so we're trying to compress densify Computing as much as we can when we do that of course liquid cooling becomes more efficient when we do that we can use copper for longer copper using electricity for as long as you can is good so that you don't hop across electrical to Optical will ultimately have to go Optical but we we'll stay with electrical as long as we can and so as much of the data center as we can it's it's more cost effective it's more energy efficient it's more reliable and so that causes us to densify the other benefit of densification is that all of the gpus that are in a particular Rack or in adjacent racks can behave as one giant GPU it's really quite amazing amazing yeah one of the things I've always been curious about Jensen is Keynotes that you did at computex I remember watching the one you did I it was on a Sunday night and the sheer volume of content that you go through is not only unbelievable but as someone else who does Keynotes that are not nearly as long or as in depth I just marvel at how you pull that off do you do massive amounts of rehearsal for I back in the day when we worked together I remember changing them at times the evening before and you still pulled them off but now the level of depth that you go into particularly given back you're talking about the data center architecture you've expanded it out how how do you prepare for it well we're preparing for it every day you know that's kind of the nice thing about our job is We're Not actors and so so is a kind of our job we kind of live in it and so we're number one we're preparing every day but a lot of what you and I do frankly is teaching yeah in order to shape an industry in order to shape the market and to introduce new ideas like what we're trying to do a lot of it is teaching know it's not advertising it's not you know and we're a platform company meaning we can't really do what we do without other people doing it you know with us and so we're about teaching inspiring showing maybe demonstrating and hopefully step by step by step you know we get more and more Believers in Cuda in the beginning Nvidia accelerated Computing today and joining us in our journey in Ai and then now the next big thing that we're working on is physical Ai and how do we teach AIS that obey the laws of physics on the one hand and then also understand the physical laws on the other hand so I think the journey is fairly long and so you know GTC and computex these are opportunities for us to do that to celebrate our ecosystem and the work that they've done teach him you know or sort Inspire them about the next quite similar we'll do qbus I'll do presentations and I Chief of Staff will say gosh the slides are easy that's kind of what you say all day and I'm thinking well how could it be different it's still hard though to be honest because we don't practice so you know it's it's not because we choose not to practice by the time we get all the stuff put together it's there is no time to practice and so you know we just grip it and rip it Jensen thanks much great appreciate great to see you good job of everything you're doing proud of you thank you I'm Renee hos CEO of arm and it's been a pleasure to have you with us today and thanks again to Jensen Wong for joining us for our very first episode we'll be back next month with more exclusive conversations and insights from the world of technology make sure you follow Tech unheard wherever you listen to your podcasts thanks for [Music] listening armtech unheard is a custom podcast series from arm and national public media media executive producers Erica oser and Shannon burner project manager Colin Harden creative lead producer Isabelle Robertson editors Andrew Merryweather and Kelly Drake composer Aaron levenson arm production contributors include am badani Claudia Brandon Simon Jared Jonathan Armstrong Ben webell Sophia McKenzie Kristen Ray and soml Shaw Tech unheard is hosted by armed chief executive officer Renee hus [Music]
2024-10-17