The Future of Enterprise: A Deep Dive Into Big Data and Natural Language Applications - Majid Hasan

The Future of Enterprise: A Deep Dive Into Big Data and Natural Language Applications - Majid Hasan

Show Video

welcome back everyone towards our next session and for this hour i had the great pleasure to introduce in my opinion a  very interesting session where we're gonna   take a deep dive into big data and natural  language applications it's my great pleasure   today to introduce to you Majid Hasan who will  be presenting from Singapore today Majid is the   chief technology officer at ESGenie he's a data  scientist with an economic event and Majid is   passionate about leveraging Market expectations  and with alternative data and AI modeling   he's currently working on an analytics startup  that promises significant significant gains in   the accuracy of economic forecasts at a friction  of the cost of currently fundamental research and   due diligence practices by reverse engineering the  Market's Collective forward-looking expectations   using Leading Edge asset pricing models Majid  holds a PhD in finance from edhcc business   school with over 10 years of experience in  building Big Data products for small medium   and large organizations he's currently  the CTO at ESGenie where he's leading the   development of an industry-leading product for  unstructured ESG data processing and Analysis   Majid it's my great pleasure to welcome  you for the first time to Big data days and   it's a pleasure to have you today and I  really look forward to your presentation   floor is yours all right thank you Jan thank you  for the great introduction it's my pleasure to be   in this great conference and I'm looking  forward to the session and if uh for the   audience if anyone has any questions uh during  in the meantime please feel free to ping me on   the chat uh I'll be I'll try my best to answer  all the questions as best as I can alright so   yeah so let's get started with the talk so the  overall I think I'm going to talk mostly about um   the natural language applications for Enterprise  use cases as the top title suggests um so let's   start with sort of the broader AI trends that are  happening in the world right now and mostly in the   context of a natural language processing so as I  guess many of you would have heard about chatGPT   um and open AI which you know I've got a lot of  attention recently so that's one of the examples   of a natural language processing but it's not just  limited to chatgpt or open air there's actually   a lot of advancement that has happened in the  actual language processing I would say over the   last five to seven years and broadly speaking you  could categorize these uh advances under natural   language understanding which is a short term  for describing all the different types of AI   models that can understand language in different  ways so this could be vary for example so these   natural language understanding models could be  for example very general purpose models like the   chat GPT which you know you can ask it to find  answers kind of based information on information   retrieval questions like you can ask it about  who the presidents of the United States were   you know over some period of time you can also  ask it to write poems stories Etc but you also   have more you could also have more specialized  models that could do one type one or other type   of language tasks for example summarizing text  or classifying text into some predefined set   of labels or question answering mode and all of  these models essentially fall within the broad   umbrella of natural language understanding  and some of these models are also what we call   generative language models so for example GPT is a  basically classic example of tax generation model   and then it's not all and then also other types  of generative models for example there is video   generating models there are image generating  models 3D shapes generating models Etc and on top   of this there's also multi-model generative models  which basically use all the different modalities   like that Vision um and images all together to  be able to generate different types of things um and then on top of this board also these  generative models beat language models Vision   models we're seeing these new and new uh  these newer applications and interfaces   that are powering the modern web now so from  for example chair GPT to also other types of   uh so basically all of the web as we currently  see it it is Shifting towards a more intuitive   chat based interface where people can rather than  having to learn the user interface complicated   user interfaces people can now interact with  applications using simple natural language   descriptions of what they want to do and they'll  app can understand what they're trying to ask and   uh produce their desired results so that's kind of  the context or the overall theme of the the talk   and then for the rest of the talk I think I'll  go deeper into uh how to think about integrating   natural language processing or natural language  understanding uh to for Enterprise use cases so as I'm saying so basically let's start  with kind of defining the natural language   understanding so it basically it means in  the context of AI it's being able to perform   different types of tasks that require some sort of  an understanding of the language so it's examples   of these tasks could include semantic search  which is for example if you're uh if you have some   Enterprise documents or even external documents  and you're trying to search for some information   historically majority of the search engines like  Google they used to be based on keyword matching   so they would almost literally match the keywords  that you typed in your search and see whether   those keywords appear in the documents uh and then  find the documents that contain those keywords but   with these natural language models because they  have more of a more complex and more I would say   nuanced understanding of language that no longer  require explicit keyword matching if you ask if   you're searching for something animals they  can recognize that zoos are also related to   animals birds are also related to animals Etc  so they can find these documents that do may   not strictly contain the same specific word but  that are still closely related to that product   another example is text classification so  which is for example if you have some set of   labels for example let's say you have different  types of documents like your uh you have your   annual reports your financial statements  your sustainability reports you could use   a text classification model to classify your  documents into these different categories this   is a also a very common use case in Enterprises  for example you could classify customer queries   or customer complaints whether it is related  to some credit card related issue whether it   is related to opening a new account or whether it  is related to something else and that then helps   you direct that query to the Right audience or the  right Department within your organization another   example is of natural language understanding is  text generation which could be quite useful for uh   care use cases where you want to generate perhaps  for example legal contracts so some of these uh   letter language models could be used to generate  a specific types of documents these could be for   example and will report sustainability reports  uh your uh illegal contracts or even responses   to your customer queries Etc uh and then another  yet another category is text inference uh which   is more kind of a like a reasoning task where you  have some tests so for example question answering   and you're trying to deduce whether something  is the case so this is also a use case that is   applicable to Enterprises for example in a setting  where if you're a bank and you're trying to uh and   you make you have to decide whether to approve a  loan application or not so tax inference is quite   useful in those situations because what you need  to do is you you need to be able to read through   the text and decide decide whether it does meet  those specific requirements that you have set in   your in your tax policy to be able to approve the  law so that's where you could use a text inference   model to make those specific inferences on  whether or not this application is not approving   uh it's a similar example which is quite close to  tax inferences question answering which you could   use to avoid specific answers to some questions  for example questions about uh where's the let's   stick to this law and application example who  is who submitted this application example uh of   what what is the amount of loan that they are  asking for what is the duration of the loan   Etc so you could extract these specific questions  without having to read through the entire topic um so this is sort of the broad category  of different uh kind of the the things   that you could you do with natural  language models uh that currently exist   and one of the key questions that comes up once  you've kind of start thinking about natural   language what else and implementing them into uh  into an Enterprise use case is what type of models   because there is it is quite a diversity of models  and how to basically go about implementing Which   models should we use what are the considerations  involved in deciding are all the models the same   if not how to decide the pros and cons of  different types of choices that we have and   while there's a lot of diversity in terms of the  natural language models that currently exist both   in terms of their architectures their technical  capabilities uh their performance in terms of   their accuracy Etc from my experience I would say  one of the key things that matters the most is   um the better the model is open source or it's  available or it's like a commercial closed Source   word of uh and the reason that kind of matters  quite a bit is because with open source is the   considerations in terms of implementation are  quite different so with commercial models for   example chat GPT is a classic is a you know  well-known example of a commercial model and   errors um it's implementation in a way is much  easier because it's a commercial commercially   available uh commercially available API all you  have to do is basically get the API key get a   subscription to the API key and then you're  pretty much good to use it there's not a lot   of complicated uh integration or development  involved uh in using the model while with open   source models typically you have to have some sort  of an internal data science team even if it's just   one or two data scientists who can uh use the  model and deploy it internally on your own cloud   however the advantage with open source models is  that you have much more flexibility and control   over the model you can retrain or fine tune the  model over your internal data and you can make it   much more accurate or much more useful for your  own specific use cases while for open well for a   commercial apis typically they're all trained by  the party that is maintaining the API so there   is no necessarily a guarantee that there will be  a perfectly good fit for your specific use case so it's because of the implementation  considerations that I think this is one   of the key choices that an Enterprise  has to think about when they're looking   into incorporating uh natural language  understanding into their Enterprise use cases   uh so let's get a little bit more into the these  choices between open source and close uh closed   API models so open source models that's the  key consideration the key benefits would be   that they can be hosted or deployed within  the organization and that is also especially   useful in relevant in situations where the data  privacy is a big issue which as you know is the   case for many organizations today especially for  example in banking financial institutions it is   sometimes by law that certain types of data sets  cannot leave uh the organizational governance so   in which case that consideration alone largely  rules out using you know this kind of commercial   apis because there the data must necessarily leave  the organization boundaries while with open source   model that is not the case you can deploy the  model internally within your own cloud and the   data will always stay within your organization  so that's one big advantage of the open sources   as it's good for privacy security reasons and  then another um another key benefit in terms   of the performance is the ability to train  and fine-tune the model on your own internal   Enterprise data which could make it much better  than commercially available choices uh after the   fine tuning the model could perform much better  on the specific use cases that you have internally   the key disadvantage however is obviously that  it's more complex to uh it's more complex to   deploy open source models because you  can have to do that work internally   even without the fine tuning and training you have  to think about the kind of infrastructure the size   of the models as the models become larger you need  to uh you need to uh you need their considerations   regarding the size of the GPU and how many  clusters do you need to be able to deploy the   model so that you can run it at uh at a larger  scale especially if you're a large Enterprise and in contrast the commercial models or apis  uh they're maintained by the specialized API   companies so for example open air and then you  can access the model through their API so it's   much less work in terms of deploying the model you  actually have to do no bar for deploying the model   and in some cases because these are commercial  apis they're already being um they've already   been trained on lots of data they're already  kind of ready to use so the quality out of   the box can be better than an open source model  so for an open source model in order to make it   uh better you might have to train it some more  on your internal data or fine tune it but the   commercial apis out of the box their performance  could be better but in the disadvantage is that   you have less flexibility in terms of further  making any further improvements on that uh   quality of the model because it's typically  harder to train you have less control over   how you can how much uh to vertically or in what  direction you can train or fine tune the model   and then um the other disadvantage obviously  as I was saying is that uh that around data   previously so if the data or if you want to use a  commercial API the data must uh like the chances   are the data could have to leave your organization  uh so for uh so yeah so this is kind of the key   um so these are kind of the two key considerations  I would say one is the data privacy and the other   being do you need the flexibility to be  able to fine-tune uh or train the data   uh and maybe I'll explain a bit a little bit more  on that and since the situation so I think the   data privacy issue is relatively clear but the  uh but how do you decide whether or not do you   need to retrain or fine-tune the models or more  I think that typically depends on how uh unique   or how Niche your data is if it is something that  is very specific to the industry for example if   you're looking at some technical documents in  manufacturing industry or healthcare industry   the chances are then off-the-shelf model is  not going to know that jargon the technical   jargon and nuances abbreviations so in those  situations typically the benefits of fine-tuning   or retraining the model on the specific data set  are quite sizable uh so so that's one way easy way   to decide whether you should go with a commercial  API you know of the sales model or whether you   should invest in retraining or fine-tuning the  model for those specific data sets if but if your   use case is something which is fairly generic  data in the sense that it is like uh it is it's   not particularly specifically organization or  any specific industry if it is something like   uh like a novel or if you just for example it's  like a customer open kind of generic customer   questions the chances are an off-the-shelf  model is as good as as a point to you so now let's uh have a little uh so now let me  again a little bit into how do you leverage a   natural language understanding for Enterprises  especially in terms of how do we think about the   Enterprise use cases um so I would think  so basically I think as a starting point   um the starting point should always be centered  around some use case a clear use case in order   to decide whether there is a need or whether  there is a good fit for a natural language   model uh to be employed uh for the benefit  of the Enterprise so so start with this use   case that kind of has something that kind of  involves some kind of language understanding   or uh a reading and then break it out into  smaller or smaller language related tasks   and then uh so before we jump so I guess what I'm  trying to say here is that before we jump into   incorporating any natural language model the first  thing to do is to basically really think through   whether the natural language model what is the  use case that for which we're trying to use it   and how specifically is a natural language model  going to add value is it even does it even make   sense to add a natural language uh to add some  kind of AI capability into that use this and that   depends on YouTube being kind of very specific  and disciplined about what are the different steps   that are involved in the use case which of those  steps can be automated using a natural language   model effectively and what would be the and what  would be the benefit would it be the cost savings   would it be the additional Revenue potential  or would it be uh something else so quantify   those kind of specific benefits that you could  come from uh incorporating the different types   of language models or different types of language  understanding capabilities into your use case and   that allows you to kind of uh uh at least get a  rough sense of the possible Roi that you could get   by using natural language understanding for a uh  for that specific use case and that that gives you   a sense of whether it is worth going worth trying  to uh invest in time and effort into incorporating   natural language understanding into your use  cases because from personal experience what   I've seen is what happens many times is like uh  or sometimes organizations or people see a trend   happening and they think oh this is great and you  know we should integrate it but then the once this   and they start implementing it before doing all of  this kind of Roi calculation and then what happens   is later on you realize that actually integrating  a technology especially AI or Cutting Edge AI is   actually quite a quite a costly and time consuming  process and if you haven't fully thought through   what the costs are what the benefits are what  happens uh quite often is that later on you   put in a lot of effort and time and energy into  uh deploying this AI models internally and only   to realize that actually we cannot sustain it  it's we don't have enough benefit to be able to   justify it and then the project doesn't really  go anywhere so that ends up being part of waste   um so I'll discuss some case studies to basically  make it more kind of concrete about what are the   some of the good examples where you know you  could think about it including natural language   understanding so for example one example would be  uh so these are examples that I've seen from my   personal experience so a climate Point building  an internal search engine to track internal data   so this use case uh is kind of related to  semantic search capabilities that you could   do with language models so what happens in this  case for example in case of a climate fund um   did they receive a lot of Grant applications from  these climate friendly project around support and   the climate fund needs to decide whether or not  to approve uh each of those funding applications   and which funding applications are kind of more so  basically rank those funding applications and then   make a decision one and what that process entails  so if you follow the previous step-by-step guide   what you can see in this use case is that there is  clearly a lot of language related classic involved   so in the usual workflow what would happen is that  um the the fund would receive all this application   and then some analysts would manually read through  those documents identify categorize different   types of funding proposals into different  categories that this is a renewable this   is a solar form project it's the wind farm project  this is some Hydro project this is some other type   of renewable energy project Etc and then there  would be a specialized analyst that could read   each category of those documents so for example  all the solar power plant related applications and   then rank them by some set of concentrations you  know for example how much investment is needed for   this project how much solar energy it is going to  produce what is the cost of energy production per   unit of investment what is the amount of CO2  emissions that will be saved by each project   etc etc uh so the first task is which is you  know ranking and classifying as you can imagine   that can be automated using uh for example a text  classification model so that is something that can   be done using natural language understanding  so that is support so we have identified one   step of the process that can be uh used um  that can be automated using their Energy   natural language understanding can be used to  utilized the Second Step which is classification   um of which is basically identifying extracting  the specific information or the details about   each project that sort of thing could  also be handled using for example the   question answering order you could write down  a predefined set of questions that you run on   each one of those uh Grant applications  to extract the specific informations   uh so that's and yet another step that can be  automated and the third step would be kind of   ranking those projects once you've identified  the key extracted the key information you could   kind of do it's kind of like a text inference  class you you could ask a model you could use   the model to decide which which project or which  Grant application promises the more CO2 emissions   first unit of investment so that also can be  uh so there you can do some kind of protection   currency so there are at least three steps in  this workflow where you could use the natural   language understanding to basically automate  your process which could you know save you a   considerable uh time in terms of the manual post  of processing or reading through those documents um and yet another example again from financial  industry would be a financial advisory company   that is using language models to summarize  industry Trends risks and opportunities so   this is uh so in this case if you think about  it without using llms what would happen is some   analysts or consultants in the financial advisory  company would have to you know manually search the   web uh look through the industry Trends or look  through the news talking about industry risk and   opportunities and then write short summaries or  uh a create short presentations to be discussed   and shared internally that then they can use to  basically create advisory for companies in that   industry or sector you know what are the kind of  latest opportunities for them what are the risks   that they need to uh they need to be concerned  about what are the regulatory emerging regulatory   uh regulatory trends that they need to be cautious  about Etc so so in this case again if we use the   step-by-step approach we can see that there is at  least this Tech summarization task which we could   perhaps automate using natural language uh models  once we have searched the web we could pass the   data to a language model and ask you to write  short summaries about you know different types   of Industry Trends risks opportunities Etc another  thing uh we could also use natural language models   to for example uh do semantic search on on the  relevant information so once let's say if we have   a news API that is sourcing uh that is allows  us to Source a lot of different news about the   relevant Industries we could use it language  models to search the most relevant news um if   related to the different types of Industry Trends  or risks that we are interested in that we can   then summarize uh yet another uh use case in this  for language models would be to for example uh   um um yeah so broadly speaking we could be  a semantics to identify the relevant news   and another to summarize social element  news into um into a neat presentations   uh yet another example could be an agritech  company that uses AI to assess its suppliers   so this is a lot of like uh as you can see the  final step of this process is uh some kind of   a text inference uh text inference uh problem  so what you could do is so basically in these   use cases uh the way it would work is if you're  getting a lot of documents from your suppliers   and you're trying to screen your suppliers for  example uh in terms of their in terms of their   commitment to various sustainability initiatives  in terms of the alignment to your commitments   you could use a language model to extract the  relevant information from their documents and   make inferences so your language model takes  in uh your considerations and then looks looks   through your supplier information to identify  where there is a match where there is a mismatch um and a similar use case could be applied for  the supply chain company that wants to classify   incoming regulatory updates and classify them  into different types of regulatory changes so   that the relevant team can be updated um about  these upcoming regulatory changes so that they   can prepare themselves so this use case you can  see it's it's a fairly clear-cut uh it has a   fairly clear-cut focus on text classification  so you could use the tech classification model   to streamline this uh sort of use case and  finally as I was talking about earlier you   could utilize a text classification model or a  text generation model to for example automate   customer support uh so it could you could be  done in two ways as you could have a language   model that classifies a customer queries into  different types of categories and then you pass it   to a specialized customer service representative  for that uh topic and that answers the query or   you could just train a language model to be  able to answer the user's question directly   based on your internal data so you beat the model  about different types of problems and how to best   answer them and then the model directly interacts  with the user's query and gives them an answer so now let me I I guess I'll get you a bit more in  depth into one of these use cases so I I I'll go   through the the climate fund example in this case  the climate fund receives funding applications   from green projects and needs to evaluate these  applications to assess the most power promising   projects and as I described um uh just now  the first thing we need to do is to basically   break it down break down the overall use case  into specific language related tasks so we can   identify where there is scope for incorporating a  language model and what are the possible benefits   so in this case the relevant language task  would be semantic search to find uh the relevant   documents or and to find the relevant information  related to for example emissions avoided benefits   to women Etc another possible um a use case  another possible use of language models would   be to summarize the text so for example if the  same project talks about emissions avoided in   multiple different places or across multiple  different documents we could use a language   model to synthesize and summarize that information  into one a neat paragraph uh yet another example   would be to make uh to use a language inference  model to make specific assessments for example   is it the case that the emissions avoided by this  project are more than thousand times or is it the   case that this project is creating more than 10  000 new jobs Etc and then finally once we have   made this assessment we could use a language  model to write a short summary about perhaps   ranking the different projects comparing the  strengths and weaknesses of different projects and uh the benefits of these of replacing  the current workflow with the use of language   models we depend critically on what the current  pain points are so in this case for example the   current pain point would be for example that it  takes too long to read through the documents uh   because and human labor is costly and the reading  to the document is not the most valuable use of   the analyst type so that would be the uh the  kind of the core pain point or problem that   the use of language models puts uh solve and  then I think another important thing uh is that   um it's also um there's also another kind of  side benefit in using technology or language   models which is that if your employee leaves  a whole bunch of the expertise typically leads   with them well if you're using a language model  whatever training or whatever learning the model   has done over the years it will stay within  the organization it can become better over   time and it can allow you to transfer Knowledge  from one employee to the new incoming employees   because they can then use the language model  as a bit confined to educate themselves about   what has historically been done what about the  projects funded but what the rationales for them   Etc so it speeds up the onboarding  or uh training for the new employees   uh So based on these considerations what we could  now we can kind of begin to think about what the   ROI would be you know we have uh identified  what are the key areas where we can use the   language models we have identified the current  uh plain points or problem statements and so we   have uh so all we need to do is identify if we  remove these pain points how many of these pain   points we can remove using the language models  and what would be kind of the ROI from that   so the way to think about that would be to you  know we know that uh reading through the documents   to identify relevant information takes time so  if we replace that with an AI model the immediate   benefit is going to be the time saved and the  cost saving from the analyst uh the time for   the analyst time and that cost saving is going to  be roughly the time to read one document times the   number of documents there are today so that allows  you to kind of put some sort of a dollar figure on   what is the value add what is the possible value  add of adding uh a language model into that part   of the workflow and then the second part would be  to writing kind of uh synthesized summaries or to   to basically make assessments on the projects and  that again would be proportional with the manual   um in the current workflow that um the cost of  it would be proportional to time to generate   one document or One summary times the number  of these uh summaries to write and then and then one other benefit that perhaps is  somewhat underappreciated is sometimes is   incorporating these newer Technologies they  also can save you uh cost and effort in terms   of maintaining Legacy databases or our document  stores which are not always as easy to you or   as effective so when you replace uh so when  you utilize a language model for example the   language model can store a lot of the information  internally so you no longer necessarily need uh to   maintain all the um all the Legacy databases and  so there could be like a tech benefits in terms   of moving to a more uh in terms of moving  to more efficient uh ways of doing things   and yet another benefit as I was saying would  be in terms of training of new employees or   retaining the knowledge that has been learned  over the you know over the years without and   not having to worry about losing that uh knowledge  if a few of your analysts leave the organization so now that we have a sense of uh our um   our use case our problems and possible value add  we can kind of decide the United States where   we can decide you know what is the how do we go  about implementing so let's say we have done the   previous exercise and we realized based on you  know the the course to reading these documents   it makes sense for us to incorporate language  models you know that will save us significant   amount of time effort and the benefits basically  uh there is an Roi so once we know that there is   an Roi then the question is okay how do we  go about uh implementing uh or what is the   course so the previous steps I guess we've only  talked about the benefits from the the course or   the benefits from incorporating the model the  next thing would be to understand what are the   costs of incorporating these different types of  models that are going to replace this manual work   uh so in this case um you know the cost again  will depend on your choice primarily whether   you're using the commercial API or you're using  an open source model if you're using a commercial   API the course would be you know uh so the case  of commercial apis obviously it would depend on   what specifically is the business model of the API  they're producing but let's say uh for the sake of   argument that is going to cost us thirty thousand  dollars per year to be able to replace all of that   manual reading and you know summarizing of that  data using the API and then we can say and then   if you use like let's say an open source model for  open source model we know that let's say the data   is somewhere specialized and we know that we will  need some kind of internal training before the   model open source model is capable of handling uh  that uh handling those tasks sufficiently well so   for internal open source models it's generally uh  the case that you will need at least one or two uh   data scientists and let's say a bit currently do  not have any so then you need to have it at least   uh cover the salary that these data assignments  and that would maybe cost you at least 100K USD   and then on top of it the model needs to be  deployed uh on some GPU and that would let's   say post you another ten thousand dollars per year  and then on top of it the water training for any   additional training you will need to prepare  some training data set and then you will need   to run the training which would be time consuming  depending on the size of the model and that could   let's say cost you another 30k in in one time uh  in one time so overall in this case as you can see   uh after it's uh these kind of considerations that  commercial using a commercial API makes much more   sense because uh we only pay 30 000 per year while  for open source models The Upfront portions of   load fire so at least uh at the beginning it seems  reasonable so let's start with a commercial API   see how it works if it works well and we see that  there is uh and if you see how the performance is   and then if we have some gaps in the performance  that we feel the need to improve the models   we could always switch to a open source model  that we can fine tune or train on our own data So based on these considerations both the costs  and benefits of nlu uh using uh natural language   models in our workflow we can now finally kind  of quantify our Roi uh you know we have our the   course of reading the course of manually reading  the documents and posts of uh using the API we   can say that it would roughly you know lead  to so these are kind of made up members that   it would uh but they're not completely made up  numbers I think they are actually informed from   an action case study um they are you know what  we've seen is like you could get up to three x   reduction in cost of manual reading uh you  could get up to 5x Improvement in the speed   of processing of information so that's another  benefit so uh even if it costs the same if you   can get that information processed relatively  quickly you can make more decisions you can   process more documents you can process more  funding proposals you can basically do more   business uh the same amount of time so that has  a impact on your Revenue and then it could also   get you up to two times reduction in the cost of  data management system because a lot of your data   is now handled lot of your data management is now  handled by the model uh that you have uh deployed um and then on top of it so I  would also like to mention that   um so while these quantifiable rois are important  in the sense that they're important for you for   uh for an organization to be really clear that  yes there is a use case and to get that project   approved and have it uh sustain and have  the budget allocated and have it you know   uh moving through the the different approval  layers between the organization they're also   non-tangible benefits that may be difficult to  quantify but could be quite important so for   example one benefit that we generally see with  uh sort of integrating natural language models   is that it improves the usability of that area  because with the traditional Data Systems the   users have to learn SQL or some other complicated  programming languages to be able to just interact   and manipulate the data and that limits the a  lot of the internal users or internal employees   ability to actually do anything meaningful  with the data if they're not familiar with if   they're not super you know experts in Excel  or SQL or some other programming language   well the natural language model it becomes much  easier or much more convenient for any employ   any user to engage with the data so so there is a  so there's that benefit in terms of the organelle   it improves the data driven culture within the  organization and to employees engagement with   the data which could have a whole bunch of  whole host of you know uh aside benefits in   terms of the employee engagement or uh employee  efficiency productivity Etc so these also ideally   the organization should have uh think about and  kind of try to incorporate them into the ROI and uh so sister as I'm saying so this is  informed uh this case study was informed by   an actual case study so and then obviously it's  there's always a good side but then there's also   uh with any new technology they're also uh the  risks and challenges so the key challenges in   this kind of a case study uh that we've noticed  for uh that the AI models have uh this tendency   to what we call hallucinate what that means  is that sometimes for example if you're using   a model to do question answers sometimes the  model will generate answers that were not uh   actually in the text so if you're for example  asking a model uh so you could also see that   in for example chat CPT if you for example ask  it uh sometimes a question about some author or   some religion active States it might give you an  answer that has nothing to do with that person   uh so that's quite a natural feature of the  way that the language models are currently uh   trained and implemented so that is something  you always have to if your use case requires   accurate information that that's something you  always have to uh to be cautious of and typically   have some sort of a layer of manual inspector to  make sure that your model is either sufficiently   trained to be able to produce accurate answers  or if the model is not sufficiently trained then   you have an analyst just go through it and then  you know flagged any after that seeing the city   and so yet another issue uh typically tends to be  for a technical or companies or industry specific   jargon that off-the-shelf models do not understand  so this could be for example examples of   um acronyms or abbreviations so companies use  abbreviations in order to describe different   things which models do not understand or they  confuse it with something else another example is   sometimes uh things that have a specific technical  meaning within the organization for example   um what does it mean for the uh for the  let's say in the case of this climate   Point what does it mean for a project to have  uh to be uh to have a sufficient commitment   to uh renewable energy that would have a  specific definition uh within the context   of the organization that the model by default  uh so either you need to train the model or   it would make errors if asked a question  that depends on that sort of understanding   and yet another issue with foreigntegrating these  uh language models with existing a data stack so   that's also kind of an important point I would  like to highlight because what happens is like   when you think about AI models it's not just about  how good the model is it's also about how is your   current data stack is how is the model going  to communicate or interact with your existing   data if your existing data is stored in a way  that is difficult for the model or the API to   interact with it then you're not going to get  any real benefit out of the model so you also   have to think not just about the technology that  you need to bring in but also the technology that   you currently have and how compatible the two  are and you might actually find that the best   bit forward is not so much to jump on the new uh  bringing the new AI models But first you think   about your existing data stack and of whether uh  and and to change the existing data stack so that   it becomes easier for you to try it out different  different types of the airport because the ROI of   that investment in the your existing technology  stack or database stack would be much higher than   basically uh investing in making an AI model work  with your legacy databases or technology stack uh so to sum up this kind of a use case of the  key benefits that we've found from this case study   about that they're both first of all tangible  and intangible benefits the tangible benefits   in case of natural language understanding and  especially for this use case this sort of use   case typically revolve around cost cutting  so language models taking you know remove   a lot of the manual work in terms of reading  summarizing uh annotating or classifying the   text and that's a clear uh that offers a clear Roi  as somewhat indirect benefits can sometimes come   from Employee Engagement with the data as well  as from improvements uh or having to do less of a   maintenance of your legacy databases as you more  of your data gets moved to the language models   key challenges tend to revolve around the  models hallucination which is important for   uh the use cases that you've had a high degree  of factual accuracy uh and the the model the   inability to understand some key terms uh key  company specific or industry specific terms um yep so thank you for your time so  that's all I have for my presentation   I'm very happy to have uh have  take any questions if you have thank you Mike for this wonderful presentation   um I personally have to say that I learned  a lot in the process I did know a lot around   this and I think this is a great example of how  you could use this on a very practical basis   uh I think I'm joined by a lot of others because  I saw quite a number of questions coming in   I'm starting with a question that came in  quite early in the in the chat already and   there was a question from Jean-Paul karongwa if  I'm pronouncing it correctly and this question is   for commercial apis what's your advice to ensure  interoperability and avoid vendor login issues   yeah so that is a great question so  that uh typically always is an issue   um and um so um my broad answer would be that uh  the best way to avoid a lock-in is to uh is to not   rely overly uh is to have a think about your Tech  stack from the beginning so you so my advice would   be under no circumstances should you um devise  your roadmap in such a way that you're overly   dependent on one commercial API you should always  uh have a uh you should always set up things in   such a way that you can change uh that you can  if you try different apis from the get-go uh to   be able to do the same task and in general of if  you use the model of the Shelf it is the case that   they are not directly interchangeable but with  some um sort of additional work around it you   can always uh so for example if if you'll already  use a bit more technical um language so what you   could always do is you could always write your own  wrapper functions in your programming languages   let's say you're using python that basically  add another layer of processing on top of the   response that you get from the commercial API  so what you then do is you can have different   apis in the background but then on top of it  you have your own wrappers so that at the end   of the day the responses that you get is always  standardized so it doesn't matter you can always   switch from one API to another without affecting  your Downstream tasks and I think that is really   important because otherwise the models AI models  are not like your regular software you know even   if if one model is currently the best it's working  better than all the other choices there is no   guarantee that will continue to do so the models  get retrained and the retraining changes their   behavior and even if it improves their behavior  overall there's no guarantee that if you improve   their behavior for the specific use case that  you are using it for and if it then turns out to   become a a sub-optimal model for that use case you  might very well have to switch to another model   spirit I think that's some great advice in  terms of looking at it more from a holistic   perspective in just uh in instead of just a  specific point solution great we still have   time for some other questions let me project one  on this on the screen then it's from Wayne when   you're saying this seems like a useful case study  that I can propose to my organization but in terms   of the natural language understanding that don't  understand internal company jargon what would be   the challenge in training the model to understand  a better fit for my organizational purpose   um so for that question I think this is also a  great question um I think for the perspective   of training models for on your internal data the  uh the key challenge tends to be typically having   the training data because as much to start trying  to train the data you realize that the model needs   the data in a certain specific format and the  training data needs to be good all right the model   is not going to help you generate the training  data you know so so that work is still quite   manual and quite time consuming and that's where  and the quality of the model is going to depend   almost entirely on the quality of the training  data so you're going to need to have tools uh   internally in place to be able to train the data  first of all you're going to need to have your   data in a place where you can basically collect  enough data in your organization without having to   go to you know whole bunch of layers of approvals  and having to worry about oh where the data   actually is this is often the case people try to  start creating training data and then they realize   they actually don't know where the training data  is like where the data is because some of the data   was stored in the database that we haven't used  in last five years and now we don't know how to   access that anymore so there's a whole bunch of  issues like that so you have to be so that's what   I'm saying so I think before you start integrating  air models I think is uh it's generally helpful to   think about your current data stack and see what  kind of improvements you can make in your existing   databases that will in the longer turn a longer  run help you uh utilize AI tracker because it's   not just going to use AI models once and then  discard it that's going to be quite nice so if   you're going to use AI it has to be part of the  overall organizational strategy which includes   your existing databases and other types of Data  Solutions that are going to be interacting with   the AI systems and Border training as a follow-up  question from my side does that mean that with the   model that you've created it's always going to  be based upon the Enterprise data so it's first   up to the Enterprise to make sure that the data  quality and curation and data stewards ship so   that it's actually trusted data can I summarize it  like that yes exactly I think that's quite right   so what I mean it's like once you try to make a  model a useful and applicable you're gonna have   to productionize it which means that you're  not going to be able to on a talk basis add   training data you're going to have to have like a  streamlined data pipeline which ingests data from   your you know a data lake or data or beer house or  some other data storage solution feeds it into the   model on a regular basis so that the model can  update itself and that model then gets used by   the users so if the pipeline is not in place and  the pipeline requires more than just the choice   of the specific model that you're using commercial  or open source it requires that entire data stack   your databases and how they're going to interact  with the AI model and so that's as important as   the choice of the model or the uh what do anything  else perfect good and then one last question   um I know there's quite a lot of questions  coming in but we only have time for one more   um and this question is asking you spoke  about natural language understanding is   making it easier for the end user can you  give some examples of how easy it can be   um so I would say that the simplest example uh  like the same place it can make it is uh sort   of like a natural language queer engine sort of  like a Google where a user can just type a query   and the the language model can help identify the  most relevant answers and bring it to them more   like it's a GPT kind of solution uh because the  reason and but that can be quite a lot simpler   than for example traditional system because  as you know many of the traditional systems   the data has to be curated using SQL uh which  is not always something that a new employee   necessarily knows especially if the queries are  complex in which case you depend on some kind   of an I.T or Tech person to be able to run the  queries and give you data so the employee who   is going to use the data has very little control  or direct control or ability to be able to slice   and die so query the data in different  ways to be able to answer their questions   good thank you so much this was a really a  very interesting in-depth and the very um   understandable session so I really personally like  this a lot um thanks for your time in contributing   towards Big database and sharing your knowledge  and expertise with us that's what this whole   conference is about it's been really a pleasure  to host you today I'll tell you again thank you   for inviting me are you good so um in on the next  hour we will have in track number one another very   interesting speaker who is not going to cover  natural language processing and understanding   but is going to switch towards data literacy  especially within project management so completely   different topic note that on the other track we  still have the workshop going on so the Practical   um Workshop in case you're interested in joining  that and for those of you who would like to go   into in the next session and there we're going  to listen to Marcus in around 10 minutes from now   I will see all of you back in 10 minutes  when we start our next track thanks everyone

2023-06-02 11:48

Show Video

Other news