foreign welcome to this Cube conversation exclusive I'm John Furrier host of the cube we have Matt Garman SVP of sales and marketing for AWS Matt formerly ran for a long time the ec2 team which we know is the compute in the cloud which is it's really changed the game is a core product we want to continue to get more compute big part of it of course Matt LED that team now he runs sales and marketing and Global Services for ages of course he's a cube alumni met he's here to talk about the surge in genitive Ai and Amazon's role in bringing that value of the Enterprise and the ISB Matt thanks for joining me today and joining our podcast great John nice to be here thanks for inviting me back great to see you I know you guys got a lot going on I reported the other day that you guys announced the AI initiative and a lot of people were jumping on the Amazon's catching up kind of bandwagon you guys have been doing AI for a long time many have saw that and even Jim Cramer from CNBC publicly walked back his comments acknowledging Amazon's deep work and ML and AI let's start off by clearing the air on Adam's position you know it's history Andy jassy always said there's no compression album for experience you guys have that briefly explain the history the trajectory and the experience AWS has with mlai yeah happy to do it so so at Amazon and AWS we've been super focused on AI and ML and and have long felt for frankly 20 years we've been working on the space and and have known that this is has been and will continue to transform how companies do business and so um like you said we've we've had an expertise in this for a really long time and at AWS in particular our focus is always on how do we help our customers get the most out of new technologies that come up and I think uh recently lots of folks us included are incredibly excited about the potential the generative AI has to fully transform lots and lots of Industries and businesses and when customers think about that you know we we wanted to make sure that we don't just toss a brand new technology out there but we want to really be sure that customers can leverage it in a safe effective way that makes sense for their business and um and that's how we really think about this space it's an area that we've been deeply investing in and an area that we feel passionate about will help our AWS customers and our customers all over the world really transform their business and we think the approach that we're taking at AWS uh is ultimately how most customers are going to want to consume and build generative AI into the applications that they run yeah I definitely want to get into those approaches and some of the experiences you've had but before we dive into some of the questions around the product opportunities I have to get your reaction to all these conversations on Banning Banning AI from the Enterprise Goldman Sachs to Apple who even just last week banned employees from using chat GPT even regulation is rearing its ugly head what's going on here why are people freaking out about AI I mean it's too early to ban in my opinion or even regulate what is your position on this how do you see this playing out what's what's going on yeah I think it's um you know I think ban is a is a fun word that people like to use but but that's not really it I think what you read is is that when chat GPT came out it really uh inspired and caused a lot a broad swath of people to really understand what the power of AI was and that's that's where a lot of us have been working uh on this for a long time but it did a great job of really kind of bringing into the public consciousness of what's possible and so I think he saw a lot of people get really excited and want to jump in quickly and I think when you look at what some of the banks are doing or what some of the companies are doing they're not so much Banning the idea of generative AI they're encouraged I mean they're they're putting the brakes on their own teams to be careful about putting their own IP into those systems part of how those systems learn like chat GPT and others is that when you enter questions in when you put data into that system it takes that system integrates it into what what it knows and then it builds a broader Corpus of knowledge that it can go answer questions from so a lot of companies are you know Banning they're they're putting the brakes and so that they have the right controls and Security in place so that their own IP doesn't leak into those models and I think that's appropriate it's in fact when I talk to customers and Enterprises one of the things that they're most uh worried about is that they understand that in the future their own IP and their data is actually what's what's going to be one of the most valuable and differentiating things that they have going forward and so what they're they're putting in place is controls to ensure that they have that right set of controls over their IPs so that their employees don't inadvertently share it into one of these models and it gets kind of uploaded and then available for everybody and they kind of lose that IP they don't realize what you're saying is they don't realize that they're actually contributing to the revised corpus with their IB which then comes into all kinds of issues around IP rights and releases it essentially that's right that's right exactly it and so you know I think if you're if you're bank number one you want to make sure that your data doesn't get uh loaded up into the model and so that bank number two can learn from what you're doing a lot of great possibilities I think one of the things that I've observed over the past 13 years covering you guys and and 10 years at re invent looking forward to this year is the makeup of your customers right you've had a mix of customers from early startups then Enterprise quick adoption and then massive growth more higher level Services you guys serve essentially every kind of customer at this point in every industry and those customers want different things and if you look at the Enterprises today versus say isvs even of yesteryear and today Enterprises emerging with this kind of like super cloud mindset or isvs just want to do SAS they all want different things what are some of the key differences in how Enterprises want to consume generative AI versus say how an isv wants to consume it generate AI yeah I think well the the first thing that you you mentioned is is spot on I think everyone is going to want to use generative Ai and appropriately so it is a a powerful technology that has a potential to help us be more efficient more effective and really change customer experiences I think when you think about those differences and how a startup thinks about things or how a large Enterprise thinks about things or a SAS provider thinks about things um you know a lot of them are not totally different um as you might think they're they're stages of adoption may be different I think if you're a startup you're trying to figure out how can you get out there fast how can you iterate quickly how can you get access to some of these technologies that may only normally be accessible to really large companies and that's one of the things that cloud and AWS enable and so you see startups like hugging face like stability like Runway like I can go on and on anthropic building on top of AWS because they can get large-scale capacity quickly they can iterate quickly they can learn and they can grow so that's that's where a lot of startups love to use the cloud out and and that was as you know that's where we kind of grew up from the very beginning as as the value proposition and generative AI is no different there I was talking about this one okay that's right interrupt go ahead when you go look at a larger scaled isps it's really not that different of a story I think one of the things that they love is the ability to scale the ability to test new capabilities I think if you look at um you know large isvs like Adobe just launched uh last week new gender of AI capabilities inside of their Creative Cloud really cool stuff that these larger established isvs are doing and rolling out really Innovative new technologies and capabilities all based on generative AI sorry go ahead yeah it's just once you've just I'm going down the same road I was thinking which is Enterprises have a little bit different needs than say a developer or a startup that's growing rapidly Enterprises might want sas-like experiences like code Whisperer right for example or developers wanting like say Bedrock for the building blocks how do you mix that together what's your take on that do you see that same thing more SAS for the consumption side developers want to build with bedrock isn't that kind of where the action is I mean can you where is the I guess my question is do you believe that to be true and where's the action yeah I'll tell you like uh you know John I think really our take is there is no such thing as a homogeneous customer customers all have different ways that they want to consume this technology some are going to want to consume it at a package layer some are going to want to consume it all the way at the infrastructure layer and I think that's where AWS really shines and how our product strategy is is that we want to have capabilities for everyone we for people that want to build their own models we build our own silicon and I think increasingly that is going to be a competitive Advantage for us to have choice we have we we have and for a long time have been the best place to run GPU infrastructure and so customers love uh running large-scale GPU clusters in AWS but we also build our own infrastructure that we think has costs and performance advantages uh in the sense of trainium for large training training clusters and inferentia for running large inference clusters if you think about sagemaker it's the development platform of choice of almost every single ml developer out there to do things like make sure that you're doing safe by AI make sure that you're testing various different models to see what actually works well with your application and then Bedrock is providing an easy to use API so that the variety of models whether you're using uh you know we think that they're over time there's going to be a large number of these foundational models that folks are going to want to be able to use for a different set of use cases and they may even want to combine different ones and so Bedrock provides a really easy to use API so the customers can combine those um now now the one thing that I will say is consistent across almost every single customer that wants to use generative AI is that they want to make sure that they do it in a secure safe environment where they know that their IP is safe where they can have explainability where they have as much information as possible on how the model was created and really that's what our focus is is how can we give Enterprises that assurance that they have the highest performing infrastructure but also the best and most secure platform in order to go build that generative AI so that they know that their data and their IP doesn't leak out um uh to places where they don't control it on the security thing how do you ensure that what's the key value proposition there it sounds good back that up be specific what's what's the security compliance is it is it more um regional thing is that you guys have with your architecture what's the security um I guess how can you seal of approval from aw how do you ensure that you know I think there's there's a range of things for our for our first party models we have our own models which we refer to as our Titan models and those we're very careful from a copyright perspective of which data has been used to build that model so and we're very clear about that so customers know that they can be assured that uh the data that went to build that model is is something that we have the rights to use to go build it we we provide things like open source models inside of JumpStart art and when you're running on some of those open source models many of which are becoming really really powerful and in many cases cases are actually outperforming some of the proprietary models today customers are able to run those entirely inside of their own proprietary VPC or networking and so they can run that model there is they can isolate that from any sort of external connectivity and know that anything that they use in that model stays inside of that model stays inside of their VPC the same with bedrock where people anyone who uses any sort of tuning to tune Bedrock models which is one of the key features that we'll have inside of our Titan models um we ensure that that data doesn't leak back into the core foundational model and stays inside of the customer's VPC so many of the controls that they use for the rest of their Enterprise data work just the same for their gender of AI capabilities and we think when we've talked to a lot of customers they've come to trust AWS and our security models they trust their data inside of AWS and now when they run their generative models on top of some of that data we can provide some of those same controls to help them understand how their data stays inside of their environment I think that's a really great point in fact we've been talking about this whole prompt engineering wave where it's essentially a call I mean as a prompt is a call in prompt tuning that's operational and then obviously out autonomous is just software you mentioned Choice earlier I think that's a fundamental comment I want to just double down on that you guys have been known as a company even Amazon from the early days of you know selling books choice now you guys got a broad selection of Jenna you mentioned a few first party models that's your model um and open AI has theirs it's not on AWS and we'll come back to that in a second third party models via Bedrock which you guys announced which is getting a lot of traction and then the recent wave of Open Source Innovation just in the past like month and a half you saw a huge surge you guys got hugging face out there some of these individual models are clearly very more prominent get that and important looking ahead when will customers want to use the prominent models that you guys have and when will they want to use some of these long tail Bedrock like products and open source how will you balance those yeah I mean our our goal is is to give customers both the choice to be able to run what's best for their application because the model that's optimized for a financial services customer may not be the one that's optimized for genomics data may not be the one that performs best for e-commerce or images or any of those other things and that's why you know stability is a stability AI is a great model for images right now but um not for text and by the way they'll change over time and they'll add some of those and so we want customers to be able to pick and choose what the best image that they are the best model that they want to use for the best use case um and that's part of where sagemaker plays a big role and we we make it really easy for customers to a B test things and in a cloud you can do that you don't have to spend billions of dollars to go build your own model you can leverage some of these others and test if Model A performed better than model B um or if some combination of models is actually the optimal one for you and I think over time that's largely where people will land is they'll tune and and kind of build on top of some of these foundational models and they'll have their own model that they that they tune and then condense from those then that's the thing that they'll actually use in production and we want to make it super easy for them to do that process but then also cost effective and secure in order to actually use that and scale that out because cost is one of the long things people are looking at in the future and they're worrying about the cost of generative AI is going to be and so we focus on all areas of that to try to make sure that that we can meet all of those concerns and and have the best option for them whether it's first party Amazon models or or models open source or other proprietary ones our goal is over time to support every single model out there that's awesome and this whole conversation reminds me of early days of AWS when you had the same do I build a Data Center and provision all this stuff or do I put it in the cloud and get instant value variable elasticity I mean same kind of thing happening Dynamic here with with the cloud and and foundational models it feels the same you can stand up your own if you want good luck with that or mix and match and code your own that's right and look and over time you'll see us leveraging generative AI more and more in some of the applications that we make available to customers as well I think you mentioned earlier code whisper um is a great example of that where it's a coding companion but still with that Enterprise in mind right we're still we have automated reasoning built in to make sure that you're building secure code we have the ability to highlight if we're showing you code samples that come from open source what is the licenses and to ensure that you want to use the code sample that comes from open source these are it starts from that fundamental uh starting with the customer that we do and working backwards and we we really like to think what are the things that customers are really going to care about when we roll these out and our Focus you know which is a little different than others which is we are laser focused in AWS on how can we have generative AI make our customers successful and um you know and a little bit less you know we're not distracted by productivity Suites or search or any of those other things we are laser focused on how can we make sure that our AWS customers can take best advantage of these Technologies and we start with those use cases and then work from there so Andy Jesse shareholder letter he was very optimistic and bullish on this basically saying this is a transformative Adam salewski's comments as well and what kind of what you're getting it reminds me of the old days of AWS early days you know Andy's would say undifferentiated heavy we automate away the undiffrageous heavy lifting well kind of AI can do the same thing for differentiated heavy lifting what's your reaction to that because now it can do both right you got the cloud for undifferentiate all the the the toil provisioning and all that stuff now you're seeing AI take on more tasks Shifting the human augmenting the human capability so differentiating seeing a lot of conversations today around how AI can actually automate and differentiate for companies this is a big part of the refactoring on the business side what's your reaction to that comment yeah I mean I think look I I think generative AI is an incredibly powerful capability that has a has a chance to make us much more efficient much more effective you know it's not going to replace people uh any anytime soon you know I think that's that's that's a long way off a lot of people are worried about that but you know every time a new tool or capability comes out that's kind of transformational on what you can do I think people worry a little bit about that but if you think about code whisper code Whisperer is not going to make it so that you don't need developers anymore it's going to make it so developers don't have to write bespoke code it's going to make it so that developers can write more secure code but they can focus on some of that that piece that's like what is the Innovative customer experience that I can go deliver for my business and for customers and not have to worry about you know the blocking and tackling of necessarily writing code I think you know a future coding language is probably going to be English and that's okay you know voice yeah exactly but it's going to be saying it in English and then and then the tools will will translate that into code and so you know the expertise may not be understanding the nuances of Java or C plus or anything like that but that's okay I don't think um it doesn't make it just changes some of that skill pieces now you got to think about the parts of your application that you want to go build as opposed to how you build it so yeah humans plus AI is better than AI by itself 100 yep and that's going to be like that for a really really long time and probably forever I think that's the big thing that we want to get out there is people shouldn't be afraid of it it's an opportunity I think it's one of the biggest ways we've seen combined all the other ones and we're going to report it heavily you mentioned gpus I want to jump on that real quick so supply seems to be a bottleneck um Nvidia stock is all high and I think they're kind of like hoarding all the gpus my opinion but I won't get into that um there's demand for training and inference what do you see as the core constraint in the industry and what does the industry have to do to have a line of sight to relieve the pressure there's more demand you mentioned you've got the GPU service how how long do you expect it to take to clear this up and get get uh get more freed up you know I think um that's that's a good question I think there's a number of constraints here I think one of the things that that um uh but that's key is that it takes a lot of compute power um to go build some of these uh foundational models it takes billions of dollars of gpus but not just gpus servers networking data center power electricity um all of those pieces right and um we've been building a lot of those things for a long time we have the largest GPU clusters anywhere in the cloud uh we have the best performing GPU clusters in the cloud and long long term I think that power is actually one of those things that's um that you have to really think about because they these clusters have the potential to use you know hundreds of megawatts to gigawatts of power now you know by 2025 we'll be running all of our global data centers on renewable energy which helps a lot because there's a risk that there's um that some of that power causes environmental issues and so I'm super happy that we made that investment and commitments uh you know 10 plus years ago to do that and that's that's great but we're also gonna you know we're going to want to think about how do we scale those in a bunch of different ways and I think that's part of where our custom silicon comes in gpus are awesome Nvidia does a fantastic job of building a really good product and they're going to be super important in this space for a really long time but we also think there's space for custom designed silicon and we think that things like products like trainium have the real potential to help customers lower cost over time reduce the power Footprints and improve performance and you know there's a lot of work to get there and there's a lot of innovation that's going to happen in the industry because of so much focus in this space but um but we feel like we're at the Forefront of that and can have a a competitive Advantage for our customers and for our business by having that low-cost option for customers that actually in some cases cannot outperform what gpus can do yeah I think you're being a little bit humble there Matt you guys have that I'll give you a props the Silicon work and the physical layer you guys have been squeezing every ounce of physics out of it at AWS for years I've reported many stories on that with James Hamilton Peter DeSantis a lot of great work there but that brings up a good point you know AI is all about chat GPT which shows a ubiquity of it people always writing a paper for me a blog posts and tweets but the the action of AI is up and down the stack it's physical layer reminds me the old OSI model back in the day you mentioned physical this AI up and down the stack and it's going to be startups are going to leverage this not just for the application layer but there's work to be done can you just share your thoughts on the kind of generative AI that's happening up and down the stack I I think that there's just going to be Innovation across the board I think every every single industry there's going to be Innovation at networking there's going to be innovation of the compute layer there's going to be innovation of the tool layers there's going to be innovation in in supporting services like vector databases and other things like that there's new startups that are popping up every day focusing on on different parts of that tool chain I think all of those things are really interesting and and as we've talked about all the way up to the application stack where there's all sorts of new technologies so I think it's a technology that can be applied almost anywhere and um and that's part of what makes it super exciting and uh it's an incredibly fast moving space and you know frankly whatever we talk about this month uh you know may be totally different six months from now there's a lot of folks out there innovating and that's part of why AWS is great we give people a platform to go innovate and you know I'm I'm not the one to to guess what all those folks are going to go build with the capabilities we give them other than I know that they'll they'll build some stuff we don't expect and that's that's part of the fun of it well our just our surprise on the siliconangle in the cube is we stored all of our transcripts for 35 000 interviews in the cloud on Amazon and we've been using transcribe and other services we have an index turns out it's a large language model hey great we're turning on Cube AI right now so that kind of well never would have been available had we not been leaning in and this is something that I want to ask you because what I'm seeing in my reporting is that there's two types of customers right now on the AI side there's ones that have been into the cloud and ones that aren't are not they've done a list and shift but not truly in the cloud the pandemic showed us if you were leaning into the cloud you had a Tailwind if not you had a headwind with AI there's a feeling that if I don't lean in I might be caught flat-footed like the folks that didn't get into the cloud with the pandemic what's your reaction to that do you hear that and what do you tell customers because it's not like just jump in because you have to like there's a benefit for getting in there where you're I'm hearing that from customers saying I'll put the toe in the water I'll jump in I'll play around I'll explore discover but I don't want to be flat-footed like the pandemic where I didn't have leverage that's right I think that that you're you're exactly right I think getting all of your data and your workloads in the cloud enables you to adjust to to changing um Trends and Technologies and and I do think generative AI is one of those that every single customer and company has to really think about how they're going to integrate into everything that they do and it's harder if your data is not in the cloud and so almost like a A Step Zero is to make sure your data is in the in AWS that it's available in a data Lake that you can look at be your compute and workloads are there that you have your structure around it and so many of the customers who've already jumped into that cloud Journey are in a good place to move fast and others are hustling because they realize that this is capability they're just not going to be able to do in their own data centers there's just there's just no way of doing it the scale is just not possible the speed the technology is moving it's just not possible to do in your own data centers and so but I think this is further evidence and an impetus for people to move to the cloud quickly but I also encourage every single customer to be thinking about how generative AI is going to change their business um over the next you know many years yeah or if you've got a stack of gpus keep those and you know sell as a service opportunity there only can last couple questions just to end things I really appreciate your time I know you're super busy running running operations over there at Amazon field and Global Services fun questions thought exercises you can answer any way you want and again open AI which is not available on AWS anthropic which is available on AWS I've been talking to insiders and VC firms in some of the top Enterprises and they all want open they want Choice okay many complain privately they would like to see open AI run out with run with bedrock would you ever offer Sam Altman lots of customers for open AI via bedrock sure there's no there I would I will I would love to show I I really like I think all customers I want to have choice and so I would love to have every model that customers are interested and excited about running in in Bedrock and AWS all right there it is open customers and developers also want open source so you're starting to see that there's been a Big Sur just in the past month and a half a lot of fruits of the labor from the open source Community really jumping on uh AI big time we reported explosion of developer and entrepreneurial Innovation and value creation this is the next area where the startups and the unicorns are going the next Dropbox is going to be coming out of these communities all took advantage of AWS in the early days you guys got the big models the long tail of Open Source models are emerging what's your view on the mix of the models the big prominence to the open source long tail how do you see the mix playing out do you have an opinion do you have a visibility it's kind of how that might shake out in terms of mix yeah I it's hard to say I think some of we've seen awesome results from some of the distilled open source models recently Facebook's llama model was awesome I think uh uh there's a new model that just came out this week that's light on I think which is an even smaller model that's outperforming llama now on the open source uh World totally trained on AWS um you know I think there's there's a there's a lot of this interesting uh uh a lot of interesting Innovation that's going out there I think there's also always going to be need for these really large core models too that help distill some of these open source models and specialized models but um but you know I think it's it's such a fast moving space it's hard to say that that's why I think that the choice is so important it's you know there's anybody's guess as to exactly which horse to bet on um we we'd prefer to make it really easy for customers to switch horses if they find the one that they like better later Matt thanks so much for your time final question as people ride this wave it's a big one if they're not on the wave riding it properly they're going to be Driftwood as we've been saying on the cube what's your advice there's a lot of change you mentioned just some of these really good examples of you know carbon footprint which by the way Stanford did come out with a study saying it's worse than anything else even crypto so that's a good play for the cloud for you congratulations and but but for startups out there what's what's your advice because the entrepreneurial track is not the same as it was during gen one you got to get customers but scale is a huge thing we've been saying how do you see the this next gen wave hitting what's your advice to startups and for companies there's a lot of change how do you keep on top of it what would you advise right part of what we think about is is that we think AWS is a great place for startups and and all sorts of customers to to to actually as a channel to get to customers they're you know the vast majority of Enterprises and companies out there are running their businesses in AWS but we're not going to go build the broad swath of innovative new technologies we'll we'll deliver a lot of stuff but there's a lot that we won't build and um and partners are key to what everything that we do in AWS and so we have a lot of programs from Marketplace all the way through to some of our Channel programs and um and certifications to ensure that our partners are available for our customers to use in a really easy to to use really easy to integrate way and so I'd encourage all of them to look at some of those programs that we have in the in the partner ecosystem um and in Marketplace as we're seeing that that's one of the ways that a lot of Enterprises want to bring these tools together to be able to use broad Swap and things awesome Matt Garman senior vice president sales and marketing and Global Services for AWS continuing this next level it's legit next gen ai's here uh super exciting it is transformative we we love it and we love the the change and accelerated towards business transformation Matt thanks for coming on thecube exclusive conversation all right thanks John happy to be here thanks bye okay this is thecube I'm John Furrier exclusive country with all the thought leaders and and Executives and the business making things happen ah generator of AI is the hottest changing Trend happening right now obviously Dave is at the heart of it and Cloud scale um John Furrier with thecube thanks for watching [Music] foreign
2023-08-27