The Future of Enterprise: A Deep Dive Into Big Data and Natural Language Applications - Majid Hasan
welcome back everyone towards our next session and for this hour i had the great pleasure to introduce in my opinion a very interesting session where we're gonna take a deep dive into big data and natural language applications it's my great pleasure today to introduce to you Majid Hasan who will be presenting from Singapore today Majid is the chief technology officer at ESGenie he's a data scientist with an economic event and Majid is passionate about leveraging Market expectations and with alternative data and AI modeling he's currently working on an analytics startup that promises significant significant gains in the accuracy of economic forecasts at a friction of the cost of currently fundamental research and due diligence practices by reverse engineering the Market's Collective forward-looking expectations using Leading Edge asset pricing models Majid holds a PhD in finance from edhcc business school with over 10 years of experience in building Big Data products for small medium and large organizations he's currently the CTO at ESGenie where he's leading the development of an industry-leading product for unstructured ESG data processing and Analysis Majid it's my great pleasure to welcome you for the first time to Big data days and it's a pleasure to have you today and I really look forward to your presentation floor is yours all right thank you Jan thank you for the great introduction it's my pleasure to be in this great conference and I'm looking forward to the session and if uh for the audience if anyone has any questions uh during in the meantime please feel free to ping me on the chat uh I'll be I'll try my best to answer all the questions as best as I can alright so yeah so let's get started with the talk so the overall I think I'm going to talk mostly about um the natural language applications for Enterprise use cases as the top title suggests um so let's start with sort of the broader AI trends that are happening in the world right now and mostly in the context of a natural language processing so as I guess many of you would have heard about chatGPT um and open AI which you know I've got a lot of attention recently so that's one of the examples of a natural language processing but it's not just limited to chatgpt or open air there's actually a lot of advancement that has happened in the actual language processing I would say over the last five to seven years and broadly speaking you could categorize these uh advances under natural language understanding which is a short term for describing all the different types of AI models that can understand language in different ways so this could be vary for example so these natural language understanding models could be for example very general purpose models like the chat GPT which you know you can ask it to find answers kind of based information on information retrieval questions like you can ask it about who the presidents of the United States were you know over some period of time you can also ask it to write poems stories Etc but you also have more you could also have more specialized models that could do one type one or other type of language tasks for example summarizing text or classifying text into some predefined set of labels or question answering mode and all of these models essentially fall within the broad umbrella of natural language understanding and some of these models are also what we call generative language models so for example GPT is a basically classic example of tax generation model and then it's not all and then also other types of generative models for example there is video generating models there are image generating models 3D shapes generating models Etc and on top of this there's also multi-model generative models which basically use all the different modalities like that Vision um and images all together to be able to generate different types of things um and then on top of this board also these generative models beat language models Vision models we're seeing these new and new uh these newer applications and interfaces that are powering the modern web now so from for example chair GPT to also other types of uh so basically all of the web as we currently see it it is Shifting towards a more intuitive chat based interface where people can rather than having to learn the user interface complicated user interfaces people can now interact with applications using simple natural language descriptions of what they want to do and they'll app can understand what they're trying to ask and uh produce their desired results so that's kind of the context or the overall theme of the the talk and then for the rest of the talk I think I'll go deeper into uh how to think about integrating natural language processing or natural language understanding uh to for Enterprise use cases so as I'm saying so basically let's start with kind of defining the natural language understanding so it basically it means in the context of AI it's being able to perform different types of tasks that require some sort of an understanding of the language so it's examples of these tasks could include semantic search which is for example if you're uh if you have some Enterprise documents or even external documents and you're trying to search for some information historically majority of the search engines like Google they used to be based on keyword matching so they would almost literally match the keywords that you typed in your search and see whether those keywords appear in the documents uh and then find the documents that contain those keywords but with these natural language models because they have more of a more complex and more I would say nuanced understanding of language that no longer require explicit keyword matching if you ask if you're searching for something animals they can recognize that zoos are also related to animals birds are also related to animals Etc so they can find these documents that do may not strictly contain the same specific word but that are still closely related to that product another example is text classification so which is for example if you have some set of labels for example let's say you have different types of documents like your uh you have your annual reports your financial statements your sustainability reports you could use a text classification model to classify your documents into these different categories this is a also a very common use case in Enterprises for example you could classify customer queries or customer complaints whether it is related to some credit card related issue whether it is related to opening a new account or whether it is related to something else and that then helps you direct that query to the Right audience or the right Department within your organization another example is of natural language understanding is text generation which could be quite useful for uh care use cases where you want to generate perhaps for example legal contracts so some of these uh letter language models could be used to generate a specific types of documents these could be for example and will report sustainability reports uh your uh illegal contracts or even responses to your customer queries Etc uh and then another yet another category is text inference uh which is more kind of a like a reasoning task where you have some tests so for example question answering and you're trying to deduce whether something is the case so this is also a use case that is applicable to Enterprises for example in a setting where if you're a bank and you're trying to uh and you make you have to decide whether to approve a loan application or not so tax inference is quite useful in those situations because what you need to do is you you need to be able to read through the text and decide decide whether it does meet those specific requirements that you have set in your in your tax policy to be able to approve the law so that's where you could use a text inference model to make those specific inferences on whether or not this application is not approving uh it's a similar example which is quite close to tax inferences question answering which you could use to avoid specific answers to some questions for example questions about uh where's the let's stick to this law and application example who is who submitted this application example uh of what what is the amount of loan that they are asking for what is the duration of the loan Etc so you could extract these specific questions without having to read through the entire topic um so this is sort of the broad category of different uh kind of the the things that you could you do with natural language models uh that currently exist and one of the key questions that comes up once you've kind of start thinking about natural language what else and implementing them into uh into an Enterprise use case is what type of models because there is it is quite a diversity of models and how to basically go about implementing Which models should we use what are the considerations involved in deciding are all the models the same if not how to decide the pros and cons of different types of choices that we have and while there's a lot of diversity in terms of the natural language models that currently exist both in terms of their architectures their technical capabilities uh their performance in terms of their accuracy Etc from my experience I would say one of the key things that matters the most is um the better the model is open source or it's available or it's like a commercial closed Source word of uh and the reason that kind of matters quite a bit is because with open source is the considerations in terms of implementation are quite different so with commercial models for example chat GPT is a classic is a you know well-known example of a commercial model and errors um it's implementation in a way is much easier because it's a commercial commercially available uh commercially available API all you have to do is basically get the API key get a subscription to the API key and then you're pretty much good to use it there's not a lot of complicated uh integration or development involved uh in using the model while with open source models typically you have to have some sort of an internal data science team even if it's just one or two data scientists who can uh use the model and deploy it internally on your own cloud however the advantage with open source models is that you have much more flexibility and control over the model you can retrain or fine tune the model over your internal data and you can make it much more accurate or much more useful for your own specific use cases while for open well for a commercial apis typically they're all trained by the party that is maintaining the API so there is no necessarily a guarantee that there will be a perfectly good fit for your specific use case so it's because of the implementation considerations that I think this is one of the key choices that an Enterprise has to think about when they're looking into incorporating uh natural language understanding into their Enterprise use cases uh so let's get a little bit more into the these choices between open source and close uh closed API models so open source models that's the key consideration the key benefits would be that they can be hosted or deployed within the organization and that is also especially useful in relevant in situations where the data privacy is a big issue which as you know is the case for many organizations today especially for example in banking financial institutions it is sometimes by law that certain types of data sets cannot leave uh the organizational governance so in which case that consideration alone largely rules out using you know this kind of commercial apis because there the data must necessarily leave the organization boundaries while with open source model that is not the case you can deploy the model internally within your own cloud and the data will always stay within your organization so that's one big advantage of the open sources as it's good for privacy security reasons and then another um another key benefit in terms of the performance is the ability to train and fine-tune the model on your own internal Enterprise data which could make it much better than commercially available choices uh after the fine tuning the model could perform much better on the specific use cases that you have internally the key disadvantage however is obviously that it's more complex to uh it's more complex to deploy open source models because you can have to do that work internally even without the fine tuning and training you have to think about the kind of infrastructure the size of the models as the models become larger you need to uh you need to uh you need their considerations regarding the size of the GPU and how many clusters do you need to be able to deploy the model so that you can run it at uh at a larger scale especially if you're a large Enterprise and in contrast the commercial models or apis uh they're maintained by the specialized API companies so for example open air and then you can access the model through their API so it's much less work in terms of deploying the model you actually have to do no bar for deploying the model and in some cases because these are commercial apis they're already being um they've already been trained on lots of data they're already kind of ready to use so the quality out of the box can be better than an open source model so for an open source model in order to make it uh better you might have to train it some more on your internal data or fine tune it but the commercial apis out of the box their performance could be better but in the disadvantage is that you have less flexibility in terms of further making any further improvements on that uh quality of the model because it's typically harder to train you have less control over how you can how much uh to vertically or in what direction you can train or fine tune the model and then um the other disadvantage obviously as I was saying is that uh that around data previously so if the data or if you want to use a commercial API the data must uh like the chances are the data could have to leave your organization uh so for uh so yeah so this is kind of the key um so these are kind of the two key considerations I would say one is the data privacy and the other being do you need the flexibility to be able to fine-tune uh or train the data uh and maybe I'll explain a bit a little bit more on that and since the situation so I think the data privacy issue is relatively clear but the uh but how do you decide whether or not do you need to retrain or fine-tune the models or more I think that typically depends on how uh unique or how Niche your data is if it is something that is very specific to the industry for example if you're looking at some technical documents in manufacturing industry or healthcare industry the chances are then off-the-shelf model is not going to know that jargon the technical jargon and nuances abbreviations so in those situations typically the benefits of fine-tuning or retraining the model on the specific data set are quite sizable uh so so that's one way easy way to decide whether you should go with a commercial API you know of the sales model or whether you should invest in retraining or fine-tuning the model for those specific data sets if but if your use case is something which is fairly generic data in the sense that it is like uh it is it's not particularly specifically organization or any specific industry if it is something like uh like a novel or if you just for example it's like a customer open kind of generic customer questions the chances are an off-the-shelf model is as good as as a point to you so now let's uh have a little uh so now let me again a little bit into how do you leverage a natural language understanding for Enterprises especially in terms of how do we think about the Enterprise use cases um so I would think so basically I think as a starting point um the starting point should always be centered around some use case a clear use case in order to decide whether there is a need or whether there is a good fit for a natural language model uh to be employed uh for the benefit of the Enterprise so so start with this use case that kind of has something that kind of involves some kind of language understanding or uh a reading and then break it out into smaller or smaller language related tasks and then uh so before we jump so I guess what I'm trying to say here is that before we jump into incorporating any natural language model the first thing to do is to basically really think through whether the natural language model what is the use case that for which we're trying to use it and how specifically is a natural language model going to add value is it even does it even make sense to add a natural language uh to add some kind of AI capability into that use this and that depends on YouTube being kind of very specific and disciplined about what are the different steps that are involved in the use case which of those steps can be automated using a natural language model effectively and what would be the and what would be the benefit would it be the cost savings would it be the additional Revenue potential or would it be uh something else so quantify those kind of specific benefits that you could come from uh incorporating the different types of language models or different types of language understanding capabilities into your use case and that allows you to kind of uh uh at least get a rough sense of the possible Roi that you could get by using natural language understanding for a uh for that specific use case and that that gives you a sense of whether it is worth going worth trying to uh invest in time and effort into incorporating natural language understanding into your use cases because from personal experience what I've seen is what happens many times is like uh or sometimes organizations or people see a trend happening and they think oh this is great and you know we should integrate it but then the once this and they start implementing it before doing all of this kind of Roi calculation and then what happens is later on you realize that actually integrating a technology especially AI or Cutting Edge AI is actually quite a quite a costly and time consuming process and if you haven't fully thought through what the costs are what the benefits are what happens uh quite often is that later on you put in a lot of effort and time and energy into uh deploying this AI models internally and only to realize that actually we cannot sustain it it's we don't have enough benefit to be able to justify it and then the project doesn't really go anywhere so that ends up being part of waste um so I'll discuss some case studies to basically make it more kind of concrete about what are the some of the good examples where you know you could think about it including natural language understanding so for example one example would be uh so these are examples that I've seen from my personal experience so a climate Point building an internal search engine to track internal data so this use case uh is kind of related to semantic search capabilities that you could do with language models so what happens in this case for example in case of a climate fund um did they receive a lot of Grant applications from these climate friendly project around support and the climate fund needs to decide whether or not to approve uh each of those funding applications and which funding applications are kind of more so basically rank those funding applications and then make a decision one and what that process entails so if you follow the previous step-by-step guide what you can see in this use case is that there is clearly a lot of language related classic involved so in the usual workflow what would happen is that um the the fund would receive all this application and then some analysts would manually read through those documents identify categorize different types of funding proposals into different categories that this is a renewable this is a solar form project it's the wind farm project this is some Hydro project this is some other type of renewable energy project Etc and then there would be a specialized analyst that could read each category of those documents so for example all the solar power plant related applications and then rank them by some set of concentrations you know for example how much investment is needed for this project how much solar energy it is going to produce what is the cost of energy production per unit of investment what is the amount of CO2 emissions that will be saved by each project etc etc uh so the first task is which is you know ranking and classifying as you can imagine that can be automated using uh for example a text classification model so that is something that can be done using natural language understanding so that is support so we have identified one step of the process that can be uh used um that can be automated using their Energy natural language understanding can be used to utilized the Second Step which is classification um of which is basically identifying extracting the specific information or the details about each project that sort of thing could also be handled using for example the question answering order you could write down a predefined set of questions that you run on each one of those uh Grant applications to extract the specific informations uh so that's and yet another step that can be automated and the third step would be kind of ranking those projects once you've identified the key extracted the key information you could kind of do it's kind of like a text inference class you you could ask a model you could use the model to decide which which project or which Grant application promises the more CO2 emissions first unit of investment so that also can be uh so there you can do some kind of protection currency so there are at least three steps in this workflow where you could use the natural language understanding to basically automate your process which could you know save you a considerable uh time in terms of the manual post of processing or reading through those documents um and yet another example again from financial industry would be a financial advisory company that is using language models to summarize industry Trends risks and opportunities so this is uh so in this case if you think about it without using llms what would happen is some analysts or consultants in the financial advisory company would have to you know manually search the web uh look through the industry Trends or look through the news talking about industry risk and opportunities and then write short summaries or uh a create short presentations to be discussed and shared internally that then they can use to basically create advisory for companies in that industry or sector you know what are the kind of latest opportunities for them what are the risks that they need to uh they need to be concerned about what are the regulatory emerging regulatory uh regulatory trends that they need to be cautious about Etc so so in this case again if we use the step-by-step approach we can see that there is at least this Tech summarization task which we could perhaps automate using natural language uh models once we have searched the web we could pass the data to a language model and ask you to write short summaries about you know different types of Industry Trends risks opportunities Etc another thing uh we could also use natural language models to for example uh do semantic search on on the relevant information so once let's say if we have a news API that is sourcing uh that is allows us to Source a lot of different news about the relevant Industries we could use it language models to search the most relevant news um if related to the different types of Industry Trends or risks that we are interested in that we can then summarize uh yet another uh use case in this for language models would be to for example uh um um yeah so broadly speaking we could be a semantics to identify the relevant news and another to summarize social element news into um into a neat presentations uh yet another example could be an agritech company that uses AI to assess its suppliers so this is a lot of like uh as you can see the final step of this process is uh some kind of a text inference uh text inference uh problem so what you could do is so basically in these use cases uh the way it would work is if you're getting a lot of documents from your suppliers and you're trying to screen your suppliers for example uh in terms of their in terms of their commitment to various sustainability initiatives in terms of the alignment to your commitments you could use a language model to extract the relevant information from their documents and make inferences so your language model takes in uh your considerations and then looks looks through your supplier information to identify where there is a match where there is a mismatch um and a similar use case could be applied for the supply chain company that wants to classify incoming regulatory updates and classify them into different types of regulatory changes so that the relevant team can be updated um about these upcoming regulatory changes so that they can prepare themselves so this use case you can see it's it's a fairly clear-cut uh it has a fairly clear-cut focus on text classification so you could use the tech classification model to streamline this uh sort of use case and finally as I was talking about earlier you could utilize a text classification model or a text generation model to for example automate customer support uh so it could you could be done in two ways as you could have a language model that classifies a customer queries into different types of categories and then you pass it to a specialized customer service representative for that uh topic and that answers the query or you could just train a language model to be able to answer the user's question directly based on your internal data so you beat the model about different types of problems and how to best answer them and then the model directly interacts with the user's query and gives them an answer so now let me I I guess I'll get you a bit more in depth into one of these use cases so I I I'll go through the the climate fund example in this case the climate fund receives funding applications from green projects and needs to evaluate these applications to assess the most power promising projects and as I described um uh just now the first thing we need to do is to basically break it down break down the overall use case into specific language related tasks so we can identify where there is scope for incorporating a language model and what are the possible benefits so in this case the relevant language task would be semantic search to find uh the relevant documents or and to find the relevant information related to for example emissions avoided benefits to women Etc another possible um a use case another possible use of language models would be to summarize the text so for example if the same project talks about emissions avoided in multiple different places or across multiple different documents we could use a language model to synthesize and summarize that information into one a neat paragraph uh yet another example would be to make uh to use a language inference model to make specific assessments for example is it the case that the emissions avoided by this project are more than thousand times or is it the case that this project is creating more than 10 000 new jobs Etc and then finally once we have made this assessment we could use a language model to write a short summary about perhaps ranking the different projects comparing the strengths and weaknesses of different projects and uh the benefits of these of replacing the current workflow with the use of language models we depend critically on what the current pain points are so in this case for example the current pain point would be for example that it takes too long to read through the documents uh because and human labor is costly and the reading to the document is not the most valuable use of the analyst type so that would be the uh the kind of the core pain point or problem that the use of language models puts uh solve and then I think another important thing uh is that um it's also um there's also another kind of side benefit in using technology or language models which is that if your employee leaves a whole bunch of the expertise typically leads with them well if you're using a language model whatever training or whatever learning the model has done over the years it will stay within the organization it can become better over time and it can allow you to transfer Knowledge from one employee to the new incoming employees because they can then use the language model as a bit confined to educate themselves about what has historically been done what about the projects funded but what the rationales for them Etc so it speeds up the onboarding or uh training for the new employees uh So based on these considerations what we could now we can kind of begin to think about what the ROI would be you know we have uh identified what are the key areas where we can use the language models we have identified the current uh plain points or problem statements and so we have uh so all we need to do is identify if we remove these pain points how many of these pain points we can remove using the language models and what would be kind of the ROI from that so the way to think about that would be to you know we know that uh reading through the documents to identify relevant information takes time so if we replace that with an AI model the immediate benefit is going to be the time saved and the cost saving from the analyst uh the time for the analyst time and that cost saving is going to be roughly the time to read one document times the number of documents there are today so that allows you to kind of put some sort of a dollar figure on what is the value add what is the possible value add of adding uh a language model into that part of the workflow and then the second part would be to writing kind of uh synthesized summaries or to to basically make assessments on the projects and that again would be proportional with the manual um in the current workflow that um the cost of it would be proportional to time to generate one document or One summary times the number of these uh summaries to write and then and then one other benefit that perhaps is somewhat underappreciated is sometimes is incorporating these newer Technologies they also can save you uh cost and effort in terms of maintaining Legacy databases or our document stores which are not always as easy to you or as effective so when you replace uh so when you utilize a language model for example the language model can store a lot of the information internally so you no longer necessarily need uh to maintain all the um all the Legacy databases and so there could be like a tech benefits in terms of moving to a more uh in terms of moving to more efficient uh ways of doing things and yet another benefit as I was saying would be in terms of training of new employees or retaining the knowledge that has been learned over the you know over the years without and not having to worry about losing that uh knowledge if a few of your analysts leave the organization so now that we have a sense of uh our um our use case our problems and possible value add we can kind of decide the United States where we can decide you know what is the how do we go about implementing so let's say we have done the previous exercise and we realized based on you know the the course to reading these documents it makes sense for us to incorporate language models you know that will save us significant amount of time effort and the benefits basically uh there is an Roi so once we know that there is an Roi then the question is okay how do we go about uh implementing uh or what is the course so the previous steps I guess we've only talked about the benefits from the the course or the benefits from incorporating the model the next thing would be to understand what are the costs of incorporating these different types of models that are going to replace this manual work uh so in this case um you know the cost again will depend on your choice primarily whether you're using the commercial API or you're using an open source model if you're using a commercial API the course would be you know uh so the case of commercial apis obviously it would depend on what specifically is the business model of the API they're producing but let's say uh for the sake of argument that is going to cost us thirty thousand dollars per year to be able to replace all of that manual reading and you know summarizing of that data using the API and then we can say and then if you use like let's say an open source model for open source model we know that let's say the data is somewhere specialized and we know that we will need some kind of internal training before the model open source model is capable of handling uh that uh handling those tasks sufficiently well so for internal open source models it's generally uh the case that you will need at least one or two uh data scientists and let's say a bit currently do not have any so then you need to have it at least uh cover the salary that these data assignments and that would maybe cost you at least 100K USD and then on top of it the model needs to be deployed uh on some GPU and that would let's say post you another ten thousand dollars per year and then on top of it the water training for any additional training you will need to prepare some training data set and then you will need to run the training which would be time consuming depending on the size of the model and that could let's say cost you another 30k in in one time uh in one time so overall in this case as you can see uh after it's uh these kind of considerations that commercial using a commercial API makes much more sense because uh we only pay 30 000 per year while for open source models The Upfront portions of load fire so at least uh at the beginning it seems reasonable so let's start with a commercial API see how it works if it works well and we see that there is uh and if you see how the performance is and then if we have some gaps in the performance that we feel the need to improve the models we could always switch to a open source model that we can fine tune or train on our own data So based on these considerations both the costs and benefits of nlu uh using uh natural language models in our workflow we can now finally kind of quantify our Roi uh you know we have our the course of reading the course of manually reading the documents and posts of uh using the API we can say that it would roughly you know lead to so these are kind of made up members that it would uh but they're not completely made up numbers I think they are actually informed from an action case study um they are you know what we've seen is like you could get up to three x reduction in cost of manual reading uh you could get up to 5x Improvement in the speed of processing of information so that's another benefit so uh even if it costs the same if you can get that information processed relatively quickly you can make more decisions you can process more documents you can process more funding proposals you can basically do more business uh the same amount of time so that has a impact on your Revenue and then it could also get you up to two times reduction in the cost of data management system because a lot of your data is now handled lot of your data management is now handled by the model uh that you have uh deployed um and then on top of it so I would also like to mention that um so while these quantifiable rois are important in the sense that they're important for you for uh for an organization to be really clear that yes there is a use case and to get that project approved and have it uh sustain and have the budget allocated and have it you know uh moving through the the different approval layers between the organization they're also non-tangible benefits that may be difficult to quantify but could be quite important so for example one benefit that we generally see with uh sort of integrating natural language models is that it improves the usability of that area because with the traditional Data Systems the users have to learn SQL or some other complicated programming languages to be able to just interact and manipulate the data and that limits the a lot of the internal users or internal employees ability to actually do anything meaningful with the data if they're not familiar with if they're not super you know experts in Excel or SQL or some other programming language well the natural language model it becomes much easier or much more convenient for any employ any user to engage with the data so so there is a so there's that benefit in terms of the organelle it improves the data driven culture within the organization and to employees engagement with the data which could have a whole bunch of whole host of you know uh aside benefits in terms of the employee engagement or uh employee efficiency productivity Etc so these also ideally the organization should have uh think about and kind of try to incorporate them into the ROI and uh so sister as I'm saying so this is informed uh this case study was informed by an actual case study so and then obviously it's there's always a good side but then there's also uh with any new technology they're also uh the risks and challenges so the key challenges in this kind of a case study uh that we've noticed for uh that the AI models have uh this tendency to what we call hallucinate what that means is that sometimes for example if you're using a model to do question answers sometimes the model will generate answers that were not uh actually in the text so if you're for example asking a model uh so you could also see that in for example chat CPT if you for example ask it uh sometimes a question about some author or some religion active States it might give you an answer that has nothing to do with that person uh so that's quite a natural feature of the way that the language models are currently uh trained and implemented so that is something you always have to if your use case requires accurate information that that's something you always have to uh to be cautious of and typically have some sort of a layer of manual inspector to make sure that your model is either sufficiently trained to be able to produce accurate answers or if the model is not sufficiently trained then you have an analyst just go through it and then you know flagged any after that seeing the city and so yet another issue uh typically tends to be for a technical or companies or industry specific jargon that off-the-shelf models do not understand so this could be for example examples of um acronyms or abbreviations so companies use abbreviations in order to describe different things which models do not understand or they confuse it with something else another example is sometimes uh things that have a specific technical meaning within the organization for example um what does it mean for the uh for the let's say in the case of this climate Point what does it mean for a project to have uh to be uh to have a sufficient commitment to uh renewable energy that would have a specific definition uh within the context of the organization that the model by default uh so either you need to train the model or it would make errors if asked a question that depends on that sort of understanding and yet another issue with foreigntegrating these uh language models with existing a data stack so that's also kind of an important point I would like to highlight because what happens is like when you think about AI models it's not just about how good the model is it's also about how is your current data stack is how is the model going to communicate or interact with your existing data if your existing data is stored in a way that is difficult for the model or the API to interact with it then you're not going to get any real benefit out of the model so you also have to think not just about the technology that you need to bring in but also the technology that you currently have and how compatible the two are and you might actually find that the best bit forward is not so much to jump on the new uh bringing the new AI models But first you think about your existing data stack and of whether uh and and to change the existing data stack so that it becomes easier for you to try it out different different types of the airport because the ROI of that investment in the your existing technology stack or database stack would be much higher than basically uh investing in making an AI model work with your legacy databases or technology stack uh so to sum up this kind of a use case of the key benefits that we've found from this case study about that they're both first of all tangible and intangible benefits the tangible benefits in case of natural language understanding and especially for this use case this sort of use case typically revolve around cost cutting so language models taking you know remove a lot of the manual work in terms of reading summarizing uh annotating or classifying the text and that's a clear uh that offers a clear Roi as somewhat indirect benefits can sometimes come from Employee Engagement with the data as well as from improvements uh or having to do less of a maintenance of your legacy databases as you more of your data gets moved to the language models key challenges tend to revolve around the models hallucination which is important for uh the use cases that you've had a high degree of factual accuracy uh and the the model the inability to understand some key terms uh key company specific or industry specific terms um yep so thank you for your time so that's all I have for my presentation I'm very happy to have uh have take any questions if you have thank you Mike for this wonderful presentation um I personally have to say that I learned a lot in the process I did know a lot around this and I think this is a great example of how you could use this on a very practical basis uh I think I'm joined by a lot of others because I saw quite a number of questions coming in I'm starting with a question that came in quite early in the in the chat already and there was a question from Jean-Paul karongwa if I'm pronouncing it correctly and this question is for commercial apis what's your advice to ensure interoperability and avoid vendor login issues yeah so that is a great question so that uh typically always is an issue um and um so um my broad answer would be that uh the best way to avoid a lock-in is to uh is to not rely overly uh is to have a think about your Tech stack from the beginning so you so my advice would be under no circumstances should you um devise your roadmap in such a way that you're overly dependent on one commercial API you should always uh have a uh you should always set up things in such a way that you can change uh that you can if you try different apis from the get-go uh to be able to do the same task and in general of if you use the model of the Shelf it is the case that they are not directly interchangeable but with some um sort of additional work around it you can always uh so for example if if you'll already use a bit more technical um language so what you could always do is you could always write your own wrapper functions in your programming languages let's say you're using python that basically add another layer of processing on top of the response that you get from the commercial API so what you then do is you can have different apis in the background but then on top of it you have your own wrappers so that at the end of the day the responses that you get is always standardized so it doesn't matter you can always switch from one API to another without affecting your Downstream tasks and I think that is really important because otherwise the models AI models are not like your regular software you know even if if one model is currently the best it's working better than all the other choices there is no guarantee that will continue to do so the models get retrained and the retraining changes their behavior and even if it improves their behavior overall there's no guarantee that if you improve their behavior for the specific use case that you are using it for and if it then turns out to become a a sub-optimal model for that use case you might very well have to switch to another model spirit I think that's some great advice in terms of looking at it more from a holistic perspective in just uh in instead of just a specific point solution great we still have time for some other questions let me project one on this on the screen then it's from Wayne when you're saying this seems like a useful case study that I can propose to my organization but in terms of the natural language understanding that don't understand internal company jargon what would be the challenge in training the model to understand a better fit for my organizational purpose um so for that question I think this is also a great question um I think for the perspective of training models for on your internal data the uh the key challenge tends to be typically having the training data because as much to start trying to train the data you realize that the model needs the data in a certain specific format and the training data needs to be good all right the model is not going to help you generate the training data you know so so that work is still quite manual and quite time consuming and that's where and the quality of the model is going to depend almost entirely on the quality of the training data so you're going to need to have tools uh internally in place to be able to train the data first of all you're going to need to have your data in a place where you can basically collect enough data in your organization without having to go to you know whole bunch of layers of approvals and having to worry about oh where the data actually is this is often the case people try to start creating training data and then they realize they actually don't know where the training data is like where the data is because some of the data was stored in the database that we haven't used in last five years and now we don't know how to access that anymore so there's a whole bunch of issues like that so you have to be so that's what I'm saying so I think before you start integrating air models I think is uh it's generally helpful to think about your current data stack and see what kind of improvements you can make in your existing databases that will in the longer turn a longer run help you uh utilize AI tracker because it's not just going to use AI models once and then discard it that's going to be quite nice so if you're going to use AI it has to be part of the overall organizational strategy which includes your existing databases and other types of Data Solutions that are going to be interacting with the AI systems and Border training as a follow-up question from my side does that mean that with the model that you've created it's always going to be based upon the Enterprise data so it's first up to the Enterprise to make sure that the data quality and curation and data stewards ship so that it's actually trusted data can I summarize it like that yes exactly I think that's quite right so what I mean it's like once you try to make a model a useful and applicable you're gonna have to productionize it which means that you're not going to be able to on a talk basis add training data you're going to have to have like a streamlined data pipeline which ingests data from your you know a data lake or data or beer house or some other data storage solution feeds it into the model on a regular basis so that the model can update itself and that model then gets used by the users so if the pipeline is not in place and the pipeline requires more than just the choice of the specific model that you're using commercial or open source it requires that entire data stack your databases and how they're going to interact with the AI model and so that's as important as the choice of the model or the uh what do anything else perfect good and then one last question um I know there's quite a lot of questions coming in but we only have time for one more um and this question is asking you spoke about natural language understanding is making it easier for the end user can you give some examples of how easy it can be um so I would say that the simplest example uh like the same place it can make it is uh sort of like a natural language queer engine sort of like a Google where a user can just type a query and the the language model can help identify the most relevant answers and bring it to them more like it's a GPT kind of solution uh because the reason and but that can be quite a lot simpler than for example traditional system because as you know many of the traditional systems the data has to be curated using SQL uh which is not always something that a new employee necessarily knows especially if the queries are complex in which case you depend on some kind of an I.T or Tech person to be able to run the queries and give you data so the employee who is going to use the data has very little control or direct control or ability to be able to slice and die so query the data in different ways to be able to answer their questions good thank you so much this was a really a very interesting in-depth and the very um understandable session so I really personally like this a lot um thanks for your time in contributing towards Big database and sharing your knowledge and expertise with us that's what this whole conference is about it's been really a pleasure to host you today I'll tell you again thank you for inviting me are you good so um in on the next hour we will have in track number one another very interesting speaker who is not going to cover natural language processing and understanding but is going to switch towards data literacy especially within project management so completely different topic note that on the other track we still have the workshop going on so the Practical um Workshop in case you're interested in joining that and for those of you who would like to go into in the next session and there we're going to listen to Marcus in around 10 minutes from now I will see all of you back in 10 minutes when we start our next track thanks everyone
2023-06-02 11:48
