Scale your tech with a dynamic duo — AI and hybrid cloud
in a world where generative AI can enhance any business function folks want it everywhere and why shouldn't they have it well sometimes you can't just click by using generative AI effectively also requires managing your existing data infrastructure in a way that both makes sense and doesn't break the bank today we're going to talk about how your deployment options can enhance or hinder your ability to use generative AI we're talking about hybrid clouds y'all on AI and action in this series we're going to explore what generative AI can and can't do how it actually gets built responsible way to put it into practice and the real world business problems and solutions will encounter along the way so welcome to AI in action brought to you by IBM I'm Albert Lawrence and today we're going to get into how hybrid cloud and AI need each other it's more than just cost in Roi if your infrastructure is built right it can allow you to take your Innovation to the Next Level so today I'm joined by guests Hillary Hunter and Ashman hus hey Hillary is CTO infrastructure NGM Innovation at IBM and an IBM fellow she's an expert in both cloud and AI Computing what's up Hillary hey there thanks for having me glad that you could be here Ash is a leader in IBM's Innovation studio and an expert in machine learning and generative AI he has extensive experience in building complex multi Cloud systems welcome Ash thank you for having me Albert so I know you both are wondering why did I Choose You two for today's episode well like I said today I want to explore what sort of tech Foundation you need to support Ai and how to build it because it seems like people are so focused on generative AI they stopped talking about hybrid clouds and my suspicion is that that's actually the key to success I'm seeing some nodding heads over here so I think I'm on the right lane Hillary maybe you can help me out with this first question why aren't people talking about how important an intentionally designed hybrid cloud is with respect to implementing generative AI you know Albert I think a lot of times we get really swept up in the latest technology terms and in our desire to try to learn everything about it and adopt it as well as possible all of these Hot Topics really have something to do with AI and I think it's just sort of our enthusiasm and excitement around the latest technology that sometimes we as we stopped talking about some of those prior ones but as I'm sure we'll unpack more in this discussion here hybrid Cloud absolutely is key to a successful set of AI deployments where you meet the latency the cost the consumer experience that you want out of your AI Solutions and having both conversations at the same time will result in much more successful business outcomes that have a lot more value to the customer into the Enterprise okay well ask do you agree yeah I I do I do agree and I I think that um just as human beings we we've been swept up by this magical new technology called generative Ai and it's one thing to consume generative Ai and do what we was known as inferencing which is you're prompting a model and getting a response back it's a completely different thing when you've got to take data you've got to cleanse it you've got to format it you've got to get it into a form that you can then use and consume with AI and that's all the non sexy parts and I think people don't like to think about the non-sexy hard parts so much and you know that that's kind of where a lot of the input goes into creating these magnificent models so why don't we start off with just getting a good understanding of exactly what is a hybrid Cloud hybrid Cloud we think of as the capabilities that span from Enterprise Computing through private Cloud deployments meaning use of it in the very agile way and use of kubernetes and other modern Technologies on premisis and into public cloud and I would say also out to the edge right so all the places that I takeen we operated with high degrees of efficiency with as of service capabilities with consistency of operations consistency of visibility control that's kind of the span of hybrid cloud and when you think of there where does it run into AI based on what Ash said I'll I'll throw another hot technology topic that we used to all talk about all the time Big Data in that big data era we were talking about where is the data how do we analy ize it how do we process it how do we get insights out of it but when we used to talk about big data we were very conscious of where that data was and therefore if the data is spread across that hybrid Cloud landscape all the way from traditional Enterprise it into private public clouds and out to the edge then you're going to want to have the AI conversation in those same terms because AI really is about getting insights from those data bringing uh new capabilities to your clients and it's both where the data is across that ful landcape in hybrid cloud as well as where your clients and your customers are and those things then feed into where do you want to create Ai and then where do you want to deploy AI so hybrid cloud and generative AI what makes these two such a dynamic duo I think a a Mis Noma that comes across as people seem to think that AI is just a single API and there's this magical huge model and you're just going to be going to this magical huge model now in reality for an Enterprise that's not the case for very ious reasons whether it's compliance regulatory reasons latency where your organization is physically located you're going to end up in a situation where you need to train multiple models and there'll be different models in different places doing different things and what that translates to is you're going to have models running on premise you're going to have models running in a particular geography you're going to have models running in the cloud with the cloud provider for example it may be something that's related to e-commerce or something that's public facing to just general consumers and so having Ai and hybrid Cloud as a dynamic to you it was the only real way that this would work why is being able to run AI where your data and where your customers are so important there's so many reasons since number one is latency I mean for example if the latency for you to go and do data ingestion to something is considerably long you'll have during the time of doing training and and and so forth and even before that labeling and annotating data lag and that lag can translate into lots of manh hours and then that doesn't become very cost effective I work with a lot of clients in financial services for example and if you think of banking insurance we as consumers are always interested in topics like fraud right we want to ensure safety and all those kind of activities um that we're doing in that sector and therefore the companies in that sector are constantly concerned about fraud they have very sophisticated algorithms and things like that and one of the best ways to explain latency in this whole kind of hybrid cloud and AI context is to talk about you know you really don't want to lag in detection of credit card fraud for example you want that fraud detection to be instantaneous because we want to know that our bank has the best possible fraud algorithms running that are going to be able to detect um fraud as it's potentially happening if our you know card is lost or compromised and the organizations that are you know moving aggressively with the AI are really looking at latency because they want to process and look at every transaction that's flowing through a system and that requires really powerful computers mainframes in many cases but being able to do AI right there on the Mainframe can enable an organization to in an unsampled way meaning each and every transaction that's flowing through the system do a more sophisticated fraud detection and as consumers you know we have better protection they have a better product Etc and so let's let's take a moment now to zoom on in a bit on the folks that are building these systems right so now we've established and I'm getting understanding more now why these systems really do matter and how every second does count but how does a hybrid Cloud environment support a better experience for the programmers that are using AI a typical life cycle that a a programmer or a you know an engineer who who build software will they will go through where they'll have sort of a development environment normally when that development is taking place a lot of that's generally taking place close to where the developer is and it's in their own local integrated development environment whether that's running on a computer that they have physically with them or in a data center very close to them so that you know they get a very fast feedback and fast experience so being able to work with data and use that data to train a model being able to do that in a fluid and flexible way makes a night and day difference to to actually developing software and and and that doesn't change because now that developers an AI engineer or or or a data science plus programmer and in in this new world those constraints still apply take a very very simple example A lot of these kind of things are done using notebooks okay notebooks running in in Python if you uh have got your data very very disconnected and very far away from where your notebooks are actually running it's having a real frustrating impact on being able to to code and review and get that feedback cycle going that someone who's on the ground who needs to build these systems needs to contend with on a day-to-day basis okay well that person on the ground who's need to contend with things on a day-to-day basis I'm trying to jump into the mindset there and trying to understand the benefits of flexibility but then also the necessity of security at the same time so I'm curious how does someone balance those two things when you're thinking about designing an architecture for AI one of the most satisfying things for a developer Albert is to see their capabilities come to Market come to fruition reach customers actually have a difference right I mean that's we all come to work every day wanting to to make a difference through the stuff that we're creating and at that backend side of having created something through the process Ash was describing oftentimes there are a lot of checks and balances related to your point Albert related to security related to compliance and the general topic is that of AI governance AI model governance are we confident in this technology do we know how and where it's being deployed is it exhibiting any drift monitoring it all of those kinds of things and I think the hybrid Cloud conversation has a lot to do with this because we like to think about I like to think about AI is not only the model but really A a platform conversation that enables that endtoend developer life cycle from what we talked about at the beginning and pulling together data and curating data to actually testing and building and modifying a model and testing its use but then also governing it when it's put out there into the wild so to say when it's put into its context wherever it is across that hybrid Cloud landscape picking a vendor with whom you can have an AI governance framework that makes the deployment of AI be a yes from all those that are in Risk in security and compliance because they know the safety of which that you know application was constructed they know how the AI is going to be monitored moving forward and they know they can do that monitoring no matter where that AI is being deployed across the hybrid Cloud landscape I think that's a really critical aspect of the overall AI considerations of an Enterprise as well because you want to ensure that the developer has a consistent set of capabilities all the way from the data prepping cleansing to the model building testing the application evaluation and then the end governance and that really makes AI a yes not AI a no in many cases where developers get really excited they've created something and if there's you know concerns about Providence if there's concerns about online monitoring and their work might not come to fruition as quickly as if AI is viewed by the Enterprise as an overall platform discussion that really follows that entire AI life cycle and I I want to add to Hillary's point I I want to talk about two paradigms here we'll call one classical software development and the the second Paradigm AI development in classical software development you go through a software development life cycle where you write a set of instructions for a computer to follow and you have got clear business requirements which translate into sort of inputs for that application or algorithm that's being written and you expect a certain outcome and you write your application code in such a way that you handle things that are outside bounds and so you have a input a set of instructions and an output and the output is going to be what you expected to get or you're going to get some sort of error message when you're building something in the other Paradigm which is using AI you're taking an input and you're using a function to come out with a probabilistic output and what that does is it creates a dynamic situation where if you're the person who's creating uh the models and you're doing the training those inputs that you give to that model may not necessarily always be the same as Hillary was saying having that govern and having that observability in place is so important because in a traditional software development life cycle you can kind of go yeah we finished the project yeah we know what this does with AI you need to observe it and check that it's not drifting that it's not changing over time because of the inputs varying because the business landscape has changed and so it's kind of like you you don't necessarily just need to think about building a model and deploying it you need to build a model and keep your eye on it and what and iterate on it all the time and again the hybrid Cloud comes into play here because you want to be able to do all those things really quickly and very very easily because as a team you're going to need to work more cohesively than you ever have so Ash you and Hillary you both are placing a lot of weight on the shoulders of data infrastructure with these examples that you're giving I just got to know what happens if your input goes wrong there's this phrase that one of our IBM leaders uses which is that there's no AI without IIA there's no artificial intelligence without information architecture and the point there is really that this conversation about being as effective as possible with AI is really one also about making sure that you have a grip on your data where is it is it in a platform or an environment where to Ash's Point your developers can really get in there and analyze and play with data and find the most effective Ai and the most effective solution solution and then are you using AI again kind of in this platform approach in a way that you are prepared to do the governance necessary to make sure that you know what's continuing to happen with your data as you deploy Ai and that you're compliant with local rules and regulations make sure to protect personal privacy and personal information and things like that and these are all really good aims but they become to some extent guardrails that you have to keep uh in line as you deploy your AI and so I think there is a lot of work on the data landscape fortunately I think because of that earlier Big Data era some organizations have gotten a lot of their data collected well and cataloged and indexed and have appropriate metadata others are finding that this is now the impetus and I think that's actually maybe not a bad thing you're realizing that your data architecture doesn't hold up to everything you wish it was because this is the greatest opportunity to fix it and a lot of organizations historically didn't really get in and rework their information architecture because they couldn't find a high enough business value to doing so now is the time because there's such high business value to be had from the current capabilities of generative Ai and the Next Generation AI technologies that are out there that it is a a rallying point and I've seen some really amazing structures put together organizationally as well where Chief data officers Chief AI leaders uh Chief technology officers cios security officers risk everyone is coming together to the table to say now is the time that we tackle this information architecture in AI we're going to do it together this is going to be a shared Mission a common goal because we can Define that there's a 2X 3x 4X 10x value to our business if we do this right one big word that you just said right there that's sticking out for me right now Hillary is opportunity okay but when you talk about the massive opportunity that there is here there's another o word that comes to mind for me which is overwhelmed because it seems like it's so massive of an opportunity that people can feel kind of overwhelmed so let's help to break that down some how do you design an infrastructure for AI in a way that doesn't break the bank the answer is hybrid Cloud right you want to be able to start small kick around some ideas look for some use cases figure out what those benchmarks are to check that the models are behaving the way they should behave as I said in this Paradigm of software development you've got to be able to measure the benchmarks of what the model that you're going to create is going to do and what various inputs you're going to throw at it and the outcome and so you want to do that in a really fast Nimble agile way and the way to do that is to pick some use cases start small but think about using a hybrid Cloud architecture so that when you do start to get some traction and you start to make some success things like compliance governance Observer ability scalability you're already answered because of the fact that you're taking advantage of a of a hybrid Cloud architecture can you give me like a concrete example or two of of starting small and then broadening out yeah yeah absolutely so to do AI really well you need people from the business to be involved you need data scientists to be involved you need devops Engineers to be involved you need to bring all of these people together and so bringing all these people together and then having some small use case that you can kind of go you know what we think that this is going to give value to the business being able to have everybody work together and build like a small model put it into an environment where you can test it and then that team can collectively share that information with each other is this working are we getting what we thought we were going to get out of it okay that feedback cycle between the people involved in creating these things okay is what's going to lead to creating sort of like a flywheel effect and the momentum to be able to scale so now I'm thinking about what does the best case scenario look like when it comes to you know current generative AI solutions that are effectively using hybrid Cloud I think that uh a really good example and one that is very easy in the world of generative AI to to translate into sort of business outcome and for people to understand especially sort of your CFO is customer care right dealing with lots of inbound customer care inquiries okay being able to have a generative AI model to be able to interact with customers and to sort of decipher what it is that they actually want what they want to get out of it but customizing that tone and that that way of like communicating using a large language model to each customer individually I think is a really really powerful and and one where we're seeing a lot of traction awesome so customer care right there all right Hillary what about you a second thought about customer care we're certainly seeing a lot of productivity around that topic I'll share just one of the most fun parts of my job is I get to work on a bunch of internal AI products we refer to that as being customer zero of IBM's AI so what we're using it uh when it's fresh off the presses and often you know straight out of research into our hands and we have a lot of exciting momentum there in really doing what what I refer to is giving people superpowers in their day-to-day and that means you know removing TD parts of tasks you know comparisons of contracts or complex you know literature about electrical specifications or all these other kind of things and throughout many of the Enterprise clients that we work with you know a lot of people don't want to be spending time on these parts of their job and it gives them superpowers because they are able to complete these tasks much more quickly and they're able to spend more time on the more complex insights in their role and more time contributing back into contractual processes and Technical in engineering processes rather than waiting through documents and I think that there's an enormous amount of productivity that we're seeing on the teams that I work with and very similar things coming from the customers that we're working on very similar implementations with wonderful well it seems like both of you are very committed to making this idea of the hybrid Cloud a lot less cloudy for anyone who's very curious about it so thank you so much for that I'm going to give you a few of my own personal takeaways from this and at the end I'm going to ask you to let me know did I get this right okay so first off to do generative AI correctly you need multiple models running multiple places so it's not possible without hybrid clout for intelligent real-time decision-making every millisecond counts so latency is a huge factor and start with the small use case and curate a multi-disciplinary team in order to support your build that sound right you nailed it outut awesome oh my gosh awesome well Hillary and Ash it means a great deal to me that you came today for this conversation this was fantastic friends thank you so much for tuning on into this episode we really appreciate your time and I want you to stay tuned to this feed for even more great AI insights see you soon [Music]
2024-07-03 14:34
