AWS re:Invent 2024 - How generative AI is transforming legal tech with AWS (NTA311)

yeah generative AI That's what is it's so impressive and exciting and that's what we are here to talk about today so welcome everyone to NTA 311 transforming how generative AI is transforming legal Tech with AWS my name is palavi nargund I'm a principal Solutions architect at AWS and I'm also a chief technologist for the legal Tech Community within AWS joined with me today is John MO CTO of net documents and Ulta alushi who is also a co-founder of legal Tech Community within AWS Ulta over to you thank you palavi hi everyone um I'm uh excited to be here with all of you for the next hour we'll be talking about the legal industry the landscape and the technology adoption within legal industry and then John here will walk us through the Innovative Journey with generative and AWS within that documents and then palavi will close us with use cases in legal Tech and reference architectures imagine a future where the the legal technology and legal industry is revolutionized by the generative AI picture all these tedious tasks being automated allowing legal professionals to focus on the most important aspects of their profession think of legal research being not just faster but smarter and more comprehensive than ever before data in the 21st century is the equivalent of gold in the 19th century and through artificial intelligence we have been able to open up new data sources that were not reachable before imagine legal profession undergoing a transformation as revolutionary as other Industries have seen like Finance or Healthcare where through AI assisted research we have been able to expand the global markets introduce instantaneous payments improve customer experience and even extend our lifespans now imagine the same Innovative Spirit applied to law but what if we could amplify the capabilities shortening the contract drafting that it takes lawyers or legal professionals to do today or the way that it like how much time it takes them to predict cases up to 44% of legal tasks today based on Research can be automated which is a a figure that outstrips many other Industries so this can actually be done the real magic though of generative AI is in freeing legal professionals to focus on the most human aspects of their profession and that is to exercise empathy strategy and advocacy but let's first Define legal Tech legal Tech is a combination of digital tools software and applications that get applied to Legal companies and that includes law firms legal councils within large corporations to boost the productivity to enhance the um the value of their legal services and to reduce cost all through the automation of legal workflow when it comes to applying technology within the industry we first want to understand its challenges technology setup today should be able to support processing large documents of unstructured tech of any size and form and that setup has to be enabling the end users not creating a bottleneck the second is balancing skill and value we have seen a growing concern with devaluation of legal skills as we know as an example Junior lawyers spend a lot of time doing research contract Drafting and other legal tasks and the question becomes how does the next generation of lawyers for instance gain that practical experience if automation is in the loop and how do law firms in fact justify work uh like billing for work that now gets um implemented by machines the third concern that we see is with data privacy and security and we know that legal companies handle sensitive information that goes from corporate data to client data and as this data gets digitized and moves to the cloud there is a growing perception that risk of data breaches also grows exponentially but in fact the data shows that organizations that have reported data breaches within the cloud environment they are attributing those breaches to human error and misconfiguration of cloud environments and the last one is the challenge with productivity that is a complex one to execute but what it means in itself is reducing the time it takes for you to do these repeatable mundane tasks and if you don't select the right systems the right architecture and the technology to complement that legal workflow productivity may actually decline at AWS we believe that technology should enhance not replace the human elements of of the legal practice in legal tech there is this concept of a flywheel which is this idea that the small initial effort leads to significant positive outcome over time after each Revolution we see positive Roi and positive outcome for customers that Implement their process their data and their people against this flywheel concept let's illustrate that concept through a Document Handling approach we start by mapping document repositories organizing and centralizing this data so that it is searchable by everyone in the organization then we process this with natural language processing to understand its content and then we extract the metadata from it to better organize it then third we add the AI and machine learning capabilities to to analyze patterns and to learn to create systems that actually learn contextually what that data actually represents and so at the end of the day there is a system that gets deployed for legal professionals to now run very complex queries against this data set and that produces this precise context aware responses in a matter of minutes and as we said after each Revolution the ROI improves over time and the customers that have worked with AWS and AWS Technologies is they tell us that they see contract generation completion within hours versus weeks and productivity Improvement by up to 15% but your numbers here could be different your use cases could be different let let's talk now for a little bit about why legal customers choose to work with AWS for 18 years AWS has been delivering cloud services to millions of customers around the world in a variety of use cases and workloads the AWS region and availability model is recognized by gner as the recommended approach to run Enterprise applications with high availability and the 108 availability zones 34 regions that are available today give you the opportunity to run Enterprise applications across the globe especially when you have to abide to Country data resid laws AWS is the most secure with 300 plus security services around compliance and governance and that includes features as well as supporting 143 security standards and certifications AWS cloud is Sovereign by Design and we will talk more about this in a little bit but what that means is that our pledge to digital sovereignty allows us to build services and and features that meet always the customers's regulatory requirements and we are at the Forefront of generative AI with a 100,000 plus customers already using Ai and ml Services now when it comes to generative AI our mission is to democratize generative AI by offering flexibility and choice for our customers that want to run workloads in production and we look at generative AI in three layers of a stack and we'll go deeper on this in a little bit as well when it comes to deploying generative a in production at the bottom of the stack we see um models where we provide silicon AWS build silicon chips so that you can train models from the ground up but in a more costeff effective way that is applicable to training and inference type workloads as well in the middle of the stack we have we provide you secure way to build and scale applications leveraging leading AI generative AI or large Foundation models so that you can switch between these large language models made available through our Amazon Bedrock service which you may have heard multiple times throughout the week and that is our fully managed service which in very short terms gives you the ability to switch between models you can customize these models for better performance and even allow it to do automated tasks without writing any code and at the top of the stack we have applications that use these large language models but that help you to get up and running pretty quickly so all you have to do is point your data sources to the application and then you can ask it questions create content or even add additional uh actions that you wanted uh to take all while leveraging the intelligence that exists in your data uh in your organization's data repositories now we talked about security privacy and responsible responsible AI that is at the top of our um priorities when it comes to working with legal Tech customers and we place this um in the Forefront of every technology that we build any architecture and any service that you place on AWS inherits the same security standards that we build we provide that Enterprise grade security meaning that anything like we said is built on AWS inherits that and we give you full control of the data full control of your data while also giving you options to enable encryption keys to manage encryption keys and in addition apply data protection policies across your data's Journey uh in your organization our identity management system also allows you you to enable and manage user access policies so that you can um enable tight controls over who accesses which data and when when you customize and this is important a foundational model your data does not leave your virtual private Cloud on AWS and it does not get used to update that base foundational model and we'll talk more about this in a little it now at the beginning of the session we talked about reimagining Document Handling and we also talked about why customers choose to work with AWS and they say that they need security privacy the right performance the right cost structure and the right solutions that are relevant to their business having said that I would like to introduce John MZ on stage CTO of net documents thanks Ulta so I'm going to I'm going to start um with a little bit of context about net documents if you don't know net documents um we've been around 20 plus or so years I've I've been here as CTO for about two years um we're all over the world we're Global six six regions um doing business in and many many countries around those regions about 5,000 firms although we're adding it seems like several a day uh frankly um so we're growing pretty rapidly right now uh there was a big push uh into cloud services uh through the pandemic with legal it saw a very big burst relative to some industries that had already shifted to Cloud quite a bit before then 185,000 users globally and that's legal professionals uh assistance things like that over 8 billion documents and that's greater than 20 billion files and so we were adding millions and millions of files every day with our customer base into the platform 21 pedabytes of data although um again this changes almost every single day uh I think most importantly and and probably most impressive for me is we're doing roughly about 650 million transactions a day and so when you think about the platform as a legal professional you get a document you get a new case you get a new matter um the first thing you're doing is ingesting that document into your document management system so every single day you're interacting once it's in the system every piece of your workflow is is running through our platform whether you're checking it in checking it out editing it changing it sharing it all runs through the net documents platform and then we are ingesting quite a bit and so when you think about the process and you'll see some of this uh coming up in the slides it's not just about storing a document we do a lot of of processing a lot of metadata extraction and creation on that document and we're doing about 25 million new documents a day uh through that process which is which is pretty incredible frankly um as you think about that the legal space started with this foundation and it's been pretty much the same for 15 or 20 years within document management and for the first time in a long time we're seeing that change and so first and foremost lexical search so if you're if you're a legal professional and you're looking for something in a document system and think think about this a firm a large firm could have literally hundreds of millions of documents in their document management system you have to be able to find it and lexical search is how we've done it for years I can tell you in talking to our customers the holy grail for them has been semantic search uh it's just really been Out Of Reach and it's been Out Of Reach for reasons tied to cost uh the ability to even actually process and manage it well um and then it is a change and sometimes change can be challenging I think the generative AI push that has happened in the last year is changing the expectations for professionals and particularly legal professionals they want to find items as they would ask for them naturally and that's what we're seeing and so a pretty significant change moving to semantic search and it's deep within our our road map and what we're already delivering today beyond that is this concept of check in and check out and if you are a legal professional you know what you know what this is if you're not a legal professional this is essentially putting a document in the system and taking a document out in the legal world it's a lot more complex though because there's controls very tight controls a big part of our platform is being able to have permissions rights authorizations not just for people working within the firm or the corporate legal but also uh customers as well as Council outside that you have to share documents with so checking in and checking out is a very very important part of the architecture as well as the platform what we're seeing though is that is getting to a point where we want co-authoring we want to be able to work in a document at the same time uh we don't want to have to check things in and out and and uh multiple copies and some documents in a document management system could have thousands of versions depending on how long the case or the matter ran uh we want to move to co-authoring this is again an expectation our customers have and then security uh security uh we we've already al already talked about this critical uh for AWS frankly it's one of one of the primary reasons we've we've chosen AWS for our Cloud strategy uh for US security is Paramount these are the most sensitive documents uh that you can find in the world frankly uh you have to make sure they're secure uh I think the thing uh that is really interesting right now from a security perspective is it's evolving to include Ai and so you're going to see here in a few minutes how we're leveraging AI within our document management workflow what's getting really really interesting is now ai is doing things like generating content uh for legal professionals it's analyzing documents for legal professionals uh we need to understand what the AI is is actually analyzing we need to understand uh what the AI is authoring we need to understand when changes are being made by Ai and ensure we apply the same security we've applied to users for for the last two decades to AI within the platform so critical part for us so when you think about that and that documents we're shifting our our thinking uh document management with workflow is what we've been doing again for two decades plus where we're evolving to is this concept of an intelligent and simple platform and when you think about that a lot lot of customers and a lot of companies frankly and if you literally walk through reinvent you'll see I'm guessing a couple hundred companies are solving AI for a very Niche problem we think about AI across the entire platform and so we we're building the platform to enable AI in every aspect of the workflow we want this to be a workflow engine driven by Ai and data uh that delivers a very very intelligent and smart experience and so what you'll see you'll see a little bit of of us solving Niche problems uh as we started this call it late last year where it's moving to very rapidly with AWS is this underlying data platform that drives an intelligent experience and an AI experience through every single workflow through every single workflow so y AWS um I I've got to touch on this this is a relatively new Journey for us um we're 12ish months into our our push into AWS um and it was a it was a pretty big decision for us so so imagine a company 20 years plus uh running in in a private uh environment for many many years since day one uh proprietary uh encryption entropic encryption that we built 20 20 so years ago uh big hsms everywhere securing all of our data and content uh so a pretty big change for us um when you think about this and and Ulta touched on this as well uh I want to touch on a couple things mainly my data is a little wrong with 99 it's apparently 108 it's probably 112 now um but the level of cultural matching that we had with AWS and our team internally was astounding and when you look at these metrics uh they're all impressive you the regions the security the ecosystem for us it was AWS really thinks like us they they think about innovating quickly the fact that we're here every year and I'm I literally was typing emails yesterday after the keynote to my team we got to do this we're going to do that let's pivot over here you have to be able to do that and and if you want to stay viable and you you want to meet the customers's needs which is what what we're here to do as a company uh we needed that kind of a partner so a lot of a lot of great capabilities drove us to AWS I think if I had to pick one uh I would lean deeply into security um when you think about the security model that AWS is using uh and the ability to secure at the Silicon level the ability to isolate security down to single data elements and objects is really really important for us uh and we we felt by a long long margin the future State and the and the kind of the Forward Thinking from a security perspective uh was critical absolutely critical so what does this mean how how is this changing how we're building where where are we applying this uh over the last year uh to to build this platform this intelligent platform and I think of three things I think of data driven experiences I think of right tools for the job and then I think of nurturing a culture of innovation and when you think about data driven experiences for me it's about flexibility and I and I just talked about sending the email to the team yesterday uh we have to have a platform that can adapt to Technologies and be able to move quickly and so we're we're building an architecture with AWS that will allow us to switch to their their mesh framework when it comes out uh because we we asked for it about seven months ago um and and we love that and we we want to be able to take advantage of it and not take 9 months to do that or or even a year or more and so the architecture is built for that we need to be flexible we need to be responsive and not as in a UI and and stretching across screens we need to someone got that we we need to be able to respond to the customers needs as we understand how they're using the platform right because we're introducing new experiences we're introducing AI into our workflows we're learning a ton right now we're understanding how customers are using the platform we've got to be able to respond to that behavior and change how the data is driving the experience and then scale and scale again I'm not talking about ec2s and fargate I'm talking about being able to scale the organization uh and the technology because there's too many new things that are coming at us we've got to be able to scale and move fast and so for us this is organizational structure how how we organized our teams and we we shifted over the last two years years from uh teams of about 10 or so and some some cases even a little larger most of them tightly coupled with dependencies uh and we're Shifting the models where our teams are as small as four to five um we have um incredible adoption of AI generated code that's being embedded in our workflows with our development teams to get more scale from smaller teams and more efficiencies critical part for us being successful over the next few years right tools for the job driven architecture um and I I'm a I'm not saying we don't believe in apis anymore we definitely believe in apis but we are leveraging Amazon to build an event driven architecture because we're we're moving away from the response request model in our our platform and and our applications we want to be able to take those events understand what our customers and users have done in the system and Trigger actions and jobs and you'll see some of those here in a few minutes without any intervention from the customer or the user that's incredibly important when you think about leveraging generative AI die in the workflow you've got to be able to be proactive and reacting to those events versus waiting for someone to take action Ai and automation again another another big part for us our company as a whole um outside of of what we're doing with our customers uh we are looking at Ai and automation across every segment of our business uh we think it's that important and we're leveraging that to to work faster we're leveraging that to deliver better better metrics to our customers better metrics to our operations team so we can manage the environments better with AWS very very important part of what we're doing here um it's interesting when you think about uh AI automation you have to be comfortable picking up new and and different tools that you haven't used before to run your business uh and sometimes that can be a little hard and frankly uh you have to account for that cost model because you're going to have that overlap but I can tell you our experience is once you make that call and you get into that new world it becomes so much better you get much more efficient and you you get past that overlap cost very quickly and I think the thing that's for me that's that's really been insightful with our teams is they're actually happier employees they're they're able to do their jobs better with with higher quality and higher efficiency the last one here not very important at all it's just the cloud service Partners like AWS we couldn't do this without AWS there's no there's no way we could and and our decision to move into AWS uh was really simple we we couldn't keep up we couldn't keep a a private environment anywhere close to the level of innovation we're going to get from from AWS and again you don't have to look any further than this this event this week to to see that um nurturing a culture of innovation so it's interesting are I I built this slide um and it it said adopting a culture of innovation and uh my CEO corrected me and he said well we've been Innovative for 20 years and so we made a a tune there but it really is the trth I mean we we came up with entropic encryption 20 years ago which is pretty darn Innovative uh we were doing Cloud document management when most people were doing on premise and so it is true we are definitely an Innovative company an Innovative group um it's really important to understand we do that through engaging with customers and and I think a lot of people lose sight of how important that is our engineering teams spend time with our customers our engineering teams go through the same product training and onboarding training uh that our customers and partners go through we're organized for agility we're we're a company of empowered engineering and operations uh and design uh we want our our teams to go figure out the problems uh and then literally build three or four Concepts and see which one works see which one doesn't work and make decisions that way uh we move very very fast for an organization our size um and you know I I think generally we're doing daily releases on most of our applications um and then we push our teams to challenge the status quo and this is going to sound a lot like Amazon and it's it's it's not a surprise like we we have adopted a lot of the same core values um I want to hear the challenges I want to hear why the solution that we created 15 years ago that was incredible at the time isn't good enough today and so we're not just building new applications we're Reinventing and reimagining the problems we solved 10 or 20 years ago based on the technology that's available uh today a critical part of of how we're building and how we're innovating all right so what does that what does that leave us with um it's it's pretty straightforward maximum performance maximum security maximum Innovation and maximum value and and really that's that is the the number one objective for us in that partnership with Amazon very straightforward goals very straightforward targets which I also think is is really critical for getting your team centered in in moving forward so the last slide um around Amazon and and how we're doing this if you will I wanted to talk about why Amazon and this is a screenshot of our um our kind of analysis evaluation and I always I always joke with with our Amazon rep um I don't think they ever thought the deal was going to close um we went we went through uh the ringer on the analysis um to the point where we were building our infrastructure during the evaluation uh this wasn't a PowerPoint slide exercise for us we we built we built the new data objects we built bu uh all all the new uh ec2 instances we ran them at scale uh as well um we had a a lot of sizing done so um I'm again we're a year into this we got to a 90% accuracy on our cost to run and so far uh holding true to that frankly it's come down a little bit because of some of the graviton releases that have happened since we made the decision um we we also really um looked at a cloud partner and for me taking an organization that has never never run in public Cloud uh you can't you can't just turn an organization over these are incredible incredibly talented people um but there's also a learning curve and so we we were very very uh explicit in saying we're going to Resource this with a with a critical partner for about 50% of our resource allocation uh and then train up and and create a bridge for our existing team really really cool to see if you if you're looking at the data that I look at uh as of now 95% of our existing team within CTO which is operations engineering security uh have taken and have been certified with AWS in the first year which is incredible frankly the fastest I've ever seen it happen I've I've done this more than a couple times and so uh we give them the platform and the ability to learn uh and grow along the journey not expecting them to to be AWS experts uh on day one um and then the last thing I would say is the the architecture um um we we really wanted to understand what the complete architecture was going to look like before we started doing the work and it was really important for us because it allowed us to make very thoughtful decisions around what we're going to do first and a lot of a lot of folks go after the easy stuff first um we did the exact opposite we went after the four hardest things that we knew we had to get after because we knew we were going to Pivot we knew we were going to find find changes we want to make uh we knew uh we were going to struggle frankly and so uh we figured if we can knock off the four hardest aspects of our architecture and our platform in year one the second year would be pretty easy uh and sure enough that's where we are uh we're we're fighting through some of that stuff right now um but we're also doing it the right way and we're not sacrificing the architecture for time uh we're very focused on building it the right way and leveraging our our well architecture reviews and our Amazon Partners to do that which is which is pretty incredible okay so I'm going to jump in uh to Ai and action so as you think about the cloud Journey Keep in mind um we didn't just shut down a road map like we we can't do that we're a business we have customer needs uh we have a we have a space with Gen that is exploding right now and so we had to we had to get after a road map at the same time so we're literally doing our Cloud transformation with our existing platform and building new capabilities in parallel and so when you think about AI which for us um is one of the biggest markets we've seen open up in the in the last 20 years um real quick award here um we have an application patn Builder Max um and this is pretty cool this is not a solicited uh media this is just someone that went went to a conference saw Scott Kelly who leads our AI uh group demoing pattern Builder Max uh and was just blown away by it uh and and this is a t if you if you're inlegal uh you know this is a tough space it's got to be right it's got to work it's got to make the job easier or else no one's ever going to adopt it uh and so just some really cool accolades best artificial intelligence uh for 2024 which is which is a pretty big deal I think if you went to a legal Tech conference this year there's probably 600 legal Tech popups that have happened and so to win that award was was pretty awesome when you think about pattern milder Max and some of the value I'm going to click through this real quick couple things we can do um we're plugged into an llm um I love the comment yesterday uh with Matt uh how important choice was and so we're building models and we're building applications but we're also allowing our customers to bring their own models and applications uh as well and so if you think about pattern Builder Max it allows you as a user as as a no code solution to build custom applications and so if you're if you're a immigration firm and you have 35 documents that you have to do for every single person that's coming in uh for a Visa or an IM immigration application uh you can build an application with pattern mid that will automate that form creation Auto automate that form uh completion for you and literally take what what normally be several hours if not a couple of days down to 20 or 30 minutes um you can actually Auto profile data now with us and so when you think about the auto profiling uh you have you have a document and in the current day um if you go backwards documents had very limited metadata most of it was human generated and so the quality was generally lacking or poor very very at least it was inconsistent across the firm and so we're now leveraging llms as well as uh some of our vectorized data with with Titan and semantic to autor profile documents at scale uh think about dropping a million documents in the document management system and you have a consistent taxonomy that can literally update hundreds of metadata fields on every single document within literally a few minutes and you have that consistency that frankly legal professionals have been looking for for decades incredible value and incredible Time Savings doing this in the background as well and so up and up until about a month ago um you could run an application you could select documents we can now run an application against your entire Corpus of data and so the days of having to select 10 or 20 or 100 documents are are moving away the customers our customers expectations are changing we need to be able to run this at scale across their entire corporate of documents and again some of those customers that's 100 million documents or more I'm going to give you an example here Al I glossed over redlining which is probably the coolest thing we're doing right now so I shouldn't do that um we are now getting to a place where we are redlining documents uh with generative Ai and so when you think about that think of think of an attorney uh they have certain ways of of writing uh contracts or writing summaries or briefs uh the AI can understand that it will actually Auto Redline the documents something a legal assistant would typically do send it back to the partner to review accept changes uh and streamline that process you're talking about in some cases um two or three steps of of editing and redlining that are being done now with generative AI which is pretty incredible so I wanted to give you an example here just a a quick screenshot normally uh we we demo this but I uh made a rule 20 years ago to never do live demos um so what you have here is a typical old way manual limited some somewhat inaccurate in the new way you can see here some of the the examples of the metadata and this is a this is a leasing contract and think about this in a in a firm you have these management companies that will have thousands potentially tens of thousands of contracts they have rules they have compliance they have they have parameters like what's my out clause uh a lot of these companies are acquiring contracts through through acquisition this simple Auto profiling capability will allow someone to run those contracts in bulk it'll flag exceptions within the contracts that they need to have an attorney look at so out of 100,000 contracts these 15 have a really really bad exit clause you need to go look at this and potentially repaper just a simple example where this would take weeks for a firm to dig through that's that's happening in just a few minutes and then it's streamlining risk it's it's actually reducing the risk within the firm dramatically by flagging and autom in that process so at the end of the day the next big thing for us pattern Builder Max has been amazing we've got a lot of customers using it where we see the the the kind of the sphere tipping over a little bit is the AI assistant this agent to agent conversation and so and and P is going to talk to the details and and which we need to get to um for us it's it's a simple front door um there is a learning curve uh with AI and what we found with our adoption is the the fast adopters are leaned in but there's a large segment of people that are still a little nervous about Ai and so we need that door to be wide open it needs to be a very simple Wide Open Door conversational threads are what people like it's what they do they're used to doing that now it's it's a pretty common practice I joked two years ago um my wife would have never used Siri and now she doesn't send a text that's not Siri based and the the world is getting much more comfortable with that conversational based interaction uh and then safe and secure we have a saying in that documents we're going to going to bring AI to your documents we're not going to push your documents to AI so keeping the AI services within our platform and contained within our walls that we've built literally the most secure walls you're going to see for a document management system we want the AI inside not outside so it's a critical part of our strategy okay last thing I want to give you a couple a couple examples of of the legal AI assistant and we we have this in beta right now it's just getting getting out the door um ask a question are there key the key man Provisions in the PSA what are the remedies on breach what are the non-compete and non-solicit Provisions in the agreement can you give me a timeline of the facts presented in the documents we're doing this now at scale and and the results are incredible and the reason the results are incredible we're not just connecting to an llm because you saw today like the the llms Chang every single week they're getting better it's it's like a foot race we did something very unique the and pavi is going to walk you through the technology here um we're leveraging vectorized data with Titan with semantic data with elastic with the llm services as a kind of a trifecta to analyze the documents and provide responses back very unique right now uh and the results and the quality of the results are through the roof it's allowing us to get to a place where we can do redlining with accuracy that lawyers will trust last thing to say here October we launched we launched this in October of last year we have 6,000 apps that have been created Creed by our customers we're averaging 30 new apps a week within our customers and we're doing 6,000 sessions a week on this platform so we're seeing rapid adoption and if there's any question of our customers going to adopt generative Ai and legal I think we've answered the question in Spades they're adopting and they're ad adopting very very quickly all right pavi I went long I apologize so I hope I give you enough time thank you very much John actually that was a fantastic journey of um your Cloud Journey migrating to AWS as well as Innovations in the document management space um so Ulta and John covered a lot about challenges in the legal industry and how the uh especially within the legal industry the documents or unstructured data is the core object and we saw the challenges that these are in massive amount of scale I was actually zaap by the scale it's billions of documents in a year that we are talking about so imagine searching through such document scale and bringing that as a Forefront to the user so that they can converse naturally with the gener with those documents as well as um honoring the security permissions because that's going to be super important so this is where we are seeing a lot of use cases that are being asked by our customers and we broadly categorize those into three categories the first one is summarization and and um shortly I me John talked about the legal assistant and there was uh you can summarize contracts where you need to understand what are the agreements and where where are the risk areas that are associated with it or generating case summary is very critical in the legal research field so summarizing the document having generative AI summarized that document for you and make it accessible to your users uh saves a lot of amount of time for the legal professional so the key ke here is optimizing your legal processes and then improving legal professionals capability we talked about legal AI assistant a slide ago where idea is you have these massive amount of documents you make them searchable and when you are at that scale just writing semantic search and making those accessible is a huge amount of task and that is where we are using the generative AI advancements and it's not just uh searching through the document ments but it is converging in a natural like flow so the idea here is to turn matter management into Knowledge Management you need to be able to get good uh handle on the amount of unstructured data or matters that you have and makes that categorized well organized and searchable the third uh use case broad category that we see is the text generation and you would have seen this is mainly about if you have standardized templates you can draft standardized contracts or even our R fi processes the goal here is to remove that undifferentiated heavy lifting and prepare or automate the workflows in such a way that you are able to create the draft versions of documents now remember one thing we will be very cautious here or one thing we follow the principle for legal professionals is your documents need to be a first of all honor all the permissions it needs to follow the permissions that you have you are allowed to within the data the context accuracy is extremely important they are very context aware documents and again we are giving you the draft documents so that the legal professionals can use their expertise review the document and uh provide the final human in the loop picture so we did talk a lot about generative Ai and um llms and Foundation models but let's put that into perspective just quickly yet so generative AI again is a subset of deep learning model Where You Are are using large pre-trained models which are Foundation models to create or uh to create uh new content or ideas and large language models llms that we keep kept referring to are um subset of uh or the specific Foundation models that cater towards text generation or text related task now a quick poll to the audience how many of you are building generative AI applications today okay I see quite a few hand raised how many of you are training your users to write better prompts okay all right so we all know that we cannot use the foundation models at the large language models as is they need to be customized and this is where um you're familiar with these four different steps I'm just going to quickly walk through them so you start always with a uh prompt engineering where you uh specify specific instructions on how tailor the output of the foundation model for your needs the second one is the retrieval augmented generation where the key here is to identify the right or relevant context from large Corpus of data and it's not just one data source you may have multiple data sources so you want to get the right context from those large Corpus of data and then augment your prompt with the relevant information how however sometimes what happens is rag architecture is not enough sometimes you don't get the model performance that you're looking for either it could suffer it may not meet the accuracy needs of your use case or it could be related to latency or both in those cases we have seen our customers take small amount of data and adjust the foundation models weights in such a way that you are creating a specialized uh uh a model with specialized task but again retaining the underlying uh generative capabilities so that's where you would find tune or pre-train the underlying um Foundation model there are certain cases where that is also not enough it's either the foundation model has biases that doesn't work for your use case or you have large amount of private data that is not being seen by these models when they were being trained and in those cases you would go for training the model from scratch now remember as you are going through all these steps the time cost and um complexity increases but it also increases the accuracy so we encourage you to look at what your use case is evaluate the different steps and then use the right uh methodology to customize the foundation model so uh let's take a look at uh the generative AI and ala talked about it uh at length that we do have the generative AI stack that's divided into three uh macro layers and there is infrastructure layer where Ulta went into details about trinium and inferentia and you saw the announcements that came in yesterday with Matt and Amazon sagemaker so Amazon Sage maker is actually in the infrastructure layer here because just having the right chipset is good thing but you need to have additional capabilities such as distributed training automated model tuning Etc so this is where Amazon sagemaker which is our managed service to build train and deploy models at scale has these features such as distributed training where it has can handle large amount of data and then give you options to uh flexibly deploy these models as well as responsible AI features and um mlops so we did talk about if uh but majority of our customers do want to build with with use uh existing Foundation models and that's where Amazon Bedrock comes into picture now bed Amazon Bedrock uh ala went into detail about this it's our managed service where you do get a choice of large Foundation um selection of foundation models to choose from and these are Foundation models from leading mod model providers as well as there are features that that can help you build your uh build your generative AI application and then uh on top is the applications that Leverage The llms or FMS which means these applications are completely built for you sometimes it may happen that you may not have the specialized skills to build the generative AI uh uh applications or it could be that you just want to explore the data set real quickly there is a new data source you want to see what the data source looks like instead of building a Genera application you can use what is already built in so this is where Amazon Q comes into picture it can uh help you get that access to the Enterprise data again with the right level of permission and that is where Amazon Q security comes into picture so Amazon Q security the way it is built is if user doesn't have access to data outside of Q the user will not have access to the data inside of Q and that's already built in so you integrate with your data sources and get started with Amazon Q uh again Amazon Q helps you uh interact with your Enterprise data as well as generate code uh create generative dashboards so there are a lot of features that um Amazon Q offers so let's take a look at uh our architectures I mean we going to talk about few architecture patterns but before that I want to quickly highlight the Bedrock features that we will be using to um to go through those architectures so one of the one of the core principles that we believe in is to give our customers uh our core value is mainly to give you model Choice data privacy and customization so from model Choice perspective John touched upon this a little bit earlier that it's it's a it's a race there's new models that are always coming into picture and no one model fits all use cases so we want our customers to have a choice so using Amazon Bedrock you have a choice to select from the leading uh Foundation model providers such as en topic we have meta we have coare and this slide is already outdated because we have announced our own model yesterday that NAA Nova came into picture so this is where I mean it is like it's just a day old and my slide is already outdated so this is what the fast uh Innovation that is happening in this space so we want you to have the uh access to the right model that meets your cost security and latency requirement this is going to be the key when you choose a when you want want to choose a foundation model that works for you think about the accuracy requirement think about how much it's going to cost you and think about the security practices having a choice of model is great but what about security and that's the common question that's the first question we always get asked so first thing is first none of your data will be used by any of this Foundation model to train uh to better their model so that's the first thing Bedrock is built on on the security principle other thing that of course the data that just like any other services the data is encrypted in transit and at rest as well as it will it does have the permissions the right set of permissions that you have uh with other a other AWS Services also so there's a specific control that you can Define on what features are accessible responsible AI we cannot move forward without talking about responsible AI so definitely want to talk talk about Amazon guard rils Bedrock Gils now Bedrock guard rils uh provide you additional safeguards that are built on top of the foundation model providers so using short natural description um you it helps you define certain topics that are avoid that that has to be that should be avoided by your generative AI application and then you can also the guardrails also help you detect and block user inputs and Foundation model responses that are um uh that are that fall into those specific topics the other thing I want to highlight I mean you can also Define um you can also control thresholds where you are controlling uh where you are protecting against the harmful content jailbreak or the prompt ingestion attack the other thing I want to highlight is the Gils you define them one and once and then they are model agnostic that means you can really configure it with any model which is offered within bedrock and uh again uh Bedrock offers number of features and those are the godil integration is native to those features so we're going to go through reference architectures for a couple of use cases and how we build using uh using Bedrock features but the first important point that I want to talk about is rag in action and how many of you are uh building applications with rag architecture okay so let's uh jump through a little bit of details so rag typically Ty Ally has two workflows a lot of times I hear complaints from my customers that well the rag is our documents are not coming out well there there's some Accura accuracy issues ETC so I highly recommend the customers going uh looking at their data injection workflow closely why let's take a look at that so we talked about earlier there are large number of documents and the goal here is to make those uh documents available via semantic search in those cases when you want to make search documents available via semantic search there are certain processes or decision points that are involved the first thing you would do is you want to uh put these documents in a vectorized database now you cannot put that entire document which is 500 pages sometimes it cannot go in the entire in the vectorized database as is so it has to be chunked which means there is specific paragraphs a specific text that is going to be uh that is going to be chunked and that's how the it is going to be then embedded to into a vector representation and goes into the vector store so there are document chunking choosing the right embedding model and choosing the right Vector store for your use case are going to be the key decision points that you that you have to make and this is where I have seen majority of the times the document accuracy suffers when you are building the rag architectures in the legal space the documents that are going to come in they are going to come in various formats they are going to be PDFs there are going to be words there are going to be text files the PDF scan PDFs they're not laid out for so this is where understanding how this documents needs to be chunked and vectorized is going to be a key activity and let's say I'm a user who wants to search through specific AR uh contract and um understand whereby if I have a uh if I have an NDA with organization a when that query comes in from a user and this is where the text generation flow happens this is where the embedding model is going to translate that query into Vector representation which then searches through the vector database and generates context out of it and that context is provided along with the prompt that you will write to get the uh will get passed on to a large language model and then you get your output now sounds very cumbersome right there a lot of things that you have to worry about chunking and moving the documents Etc well that is where Amazon Bedrock uh knowledge bases come into picture knowledge bases give you gives you that it removes that undifferentiated heavy lifting and gives you the choice to make choose a right embedding model choose a right chunking strategy it is natively integrated with a choice of vector stores and then uh and then you can spin up your manage rag architecture to understand your document workflows the important aspect about why knowledge basis is the improved accuracy and this is core this is a core architecture and core point to anytime you want to build uh semantic search related capabilities or you want to understand analyze the large documents understanding how well your documents are chunk is important and that is why knowledge basis offers you different uh uh it offers you a control over your um chunking strategy such as hierarchical chunking if a document has parent child relationship it will automatically group The chunk the document together in such a way that that hierarchy is maintained so when you are searching through that document the document accuracy improves because you have that uh hierarchy maintained similar to semantic search especially in the legal documents semantic um uh chunking capability is extremely important because the document has related text scattered throughout the document so so the idea here would be grouping that text and then um storing that into in the specific chunk now there are a lot of uh features that uh that knowledge bases offer in terms of improve improving our accuracy which I cannot go in the next 3 minutes but there is a session that I will have it listed that tomorrow is happening which goes uh through the Deep dive into knowledge basis and how to improve accuracy I would highly recommend you attend that so let's take a look at couple of architectures real quick so our architecture with respect to a contract in Tech process is something that pretty much all organizations have whether it's a illegal or whether you are an Enterprise who has a legal department so let's put what are what we learned which is uh put creating I mean when you have matters turning the matter management or contract management into a Knowledge Management so let's put those uh uh learnings into uh into a practice where we have have taken the executed contracts and then we are not only doing the executed contracts but we are extracting the metadata and attaching that metadata to the vector store this will help when you have millions of documents and you need to understand that my when I retrieve my documents from semantic search perspective I need I have it filtered based on what I'm asking so this becomes a key these are the features that the knowledge base will support so I have taken the list of executed contracts I have added them extracted the metadata add attached that metadata to Vector databases and uh created a knowledge base so when a user like me who comes in and I ask uh I need NDA with or a let's say there is a conversational user interface now you just heard Dr Swami suban announce agentic interfaces um there are multip mple the there there is uh prompt there is a model routing prompt caching Etc a lot of beautiful features have come in so using that you can build a conversational interface which is going to take that query understand what the ask is extract the specific metadata that this is NDA and a parties organization a is going to search through the knowledge base build the right results and send the final response now in my case I'm saying that uh well there is no NDA with ar or a do you want to create one and this is where you can use a conversational assistant where we haven't uh created a rules Spas system there is no if else Happening Here We the knowledge base or the agentic approach understands from the existing executed contracts and it learns from the standard template and also it's in own inherent inherent knowledge about how contracts work or different contracts is going to ask specific questions and and get those um uh draft contract created for a legal professional to review similarly text generation RFI process I hear there are multiple uh weeks that are gone in and this is where again you create an um execute historical um RFI contracts search create a knowledge base and search through them and this process actually the uh screenshot that you're seeing we have implemented this RFI process it does save from weeks to days of creating generating a new RFI from previously executed rfis um uh improving the productivity of employees now similarly if you have a quick way the similarly if you have let's say we talked about a lot about documents but let's say you have videos and images in those cases you can also make the make those accessible for example you have a lot of videos where there's testimony in those cases using um uh our AI service which is Amazon transcribe you can transcribe the uh text put that into S3 and simply put it to Amazon Q for business which is going to then make that accessible and then you can quickly start looking through asking questions such as who testified and what was there uh can you summarize this testimony uh again maintaining who has access to what that is core and a very important conversation so having said that in closing just quick uh takeaways um so we learned from uh net documents about their journey and um how they're bringing uh user efficiency and what their road map is to uh build more so I want you to think about the top three use cases and situations that um uh that can improve uh your business process or workflows and what brings the most value uh in generative AI use cases what's the ROI associate ated with those use cases so as a final ask I want you to Leverage The AWS community and start working on those you have it uh access to maybe account Tes you're working with the partners or if you're not clear about the mechanisms please uh reach out to us we can help you put in uh the right direction but this space as John earlier said it is expanding it is for you to remain viable and competitive this is going to be an important task that you have to do is you have to adopt uh you want to adopt generative AI improve efficiencies optimize your uh workflows build automated workflows and take advantage of it um that's it for now from me here are the upcoming sessions if you want to take a picture of it um aim 305 is where uh you can go uh learn more a

2024-12-11 06:13

Other news