hello I’m John Stueve I’m a manager of automation anywhere training and this is introduction to IQ Bot uh first of all when we start talking about IQ Bot we're talking about moving semi-structured and unstructured data into structured data or digital text and there's different ways of doing that the normal way with as far as RPA is concerned is using OCR which has limited support and some limited features within strictly RPA there are lots of different vendors in the intelligent OCR space and these are there are some limitations to that as far as just the complexity and the investment it takes to get up and going with those solutions so we've gone through and found the best of both worlds we've got worked through 14 years of our domain expertise in RPA to combine the right set of ai technologies in a cognitive solution that can be trained and operational at scale with about 60 straight through processing within a few weeks our computer vision uses a technologies like that which is used for self-driving cars we use natural language processing and fuzzy logic machine learning language translation sentiment analysis and chatbot those three can be leveraged using third-party ai libraries through our RPA tool so the integration of both cognitive technologies like computer vision and fuzzy logic and machine learning along with language translation sentiment analysis and chat allow us to handle business processes across several domains what really makes us more efficient is the cost it takes to get set up with intelligent OCRs there's a long lag time we can get IQ bought up an hour a couple hours per bot for different types of document training can get them up and moving so that we can produce the benefits that we have from transitioning from semi-structured data to structured data within the automation processes this is the typical process uh before where which results in about a thirty to forty percent straight through processing uh people have to classify documents and sort them out then they go through an OCR attempt and processing failed box requires people to look at the documents and make a determination whether it was a good scan or not and then they validate that processes or the outputs of those documents through the back office at the end of that those two manual processes we can go back to automation and trigger the approval workflow and then people at the end can can uh can finalize that review with IQ Bot and RPA we have uh automated the extract and digitize using the cognitive technologies we have majority of that work is done with uh just RPA with with our cognitive services and then after the training of that the bot handles those documents completely on its own there is a human in the loop for any failed documents but as they go through that validation process it uses machine learning to inform the cognitive solution how to to process similar errors in the future so over time straight through processing grows and grows after the process is finished that we validate and trigger like we would before and then we can also have assisted approvals process where a bot and a human can work together this is pretty much what every business process has in common data capture enrichment validation processing doing the actual work reconciling that to make sure we have good audit and accuracy and then analyzing reporting our carnot solution IQ Bots helps us with the data capture helps us with the data enrichment all those things can occur prior to the robotic process automation and then a smooth handoff between the two this is the real benefit of using our cognitive solution in accordance with our RPA solution so next thing I want to do is just go through how we how how this starts how can we start an IQ by process by going through a learning instance or what we call training alert learning instance first off we need to have pretty good quality documents we need to have them in a in a valid image format could be either vector or raster or combination tiff png jpeg vector all those are different image formats that we support we also support regular vector or raster pdf files or even plain text pdf files it's best to use original scans rather than scans of scans and it's best to scan a fairly high dpi or 300 dpi is what we recommend documents should be less than 10 megabytes and should be should have a fairly structured maybe a semi-structured format and each document should be in a separate file the process we're going to move through as we go through this is to create the learning instance classify it and let the the cognitive solution go through a classification step train the documents and let the process learn where to find the information that's looking to extract and then finally move that cognitive solution into production where we're working with real uh documents that have been scanned and removing them through the IQ Bot so with that I’d like to go back to my demo portion and today I’m going to be using the community edition of IQ Bot with this we have the ability to get started by creating a new learning instance and the first part of this opens up the learning instances portion of the IQ Bot control room okay so we're going to create a new learning instance and this new learning instance needs to have a name what I’m going to do with this learning instance is process a set of invoices so I’m going to call my learning instances my learning instance invoices for imagine I can provide a description further of what this learning instance is going to work through and then I need to create the document type we also call this the domain in here we select invoices and then we can select different types of languages if we want to when we select invoices at the bottom we get the different types of information we can extract from an invoice and these are the top three are selected by default and there's additional fields we can get and in an invoice what we usually see and let me bring up an example of an invoice here this is what an invoice would look like there's the top part of the screen is the form and these things occur once per document including some things at the bottom that occur once per document and then there's repeating rows and these are called the table part of the document where we want to capture each row from this so we have two sections in this document when we work through it we have the sections for the common form fields and then we have the section for the table or repeated fields for this I’m going to select invoice date invoice number invoice total I’m also going to select billing address and payment terms so we have an understanding of when we're supposed to pay this invoice and how long for the repeated segment I’m just going to capture the item description item total and quantity and that sets up the domain and tells us what the output's going to be when we get our output file from the IQ Bot we'll have columns for each one of these fields in that output so we we've standardized now the headers for our csv files the next thing I need to do is browse and find a selection of documents and if we had many documents we'd want to choose a representative sample and I know from experience that these three are a representative sample of the different documents we have available for this uh exercise so we're going to choose three and send those up to the IQ Bot and that includes the creation of the learning instance now when I click on create the instance uh what happens is it uploads it creates the the package or the boundaries for that image within IQ Bot and we'll be able to now describe that under that learning instance name now it's taking these documents that have been uploading and analyzing it it's going through a classification which things occur in which places which labels occur in each places it does some semantic analysis and aliases and then it's going to return us to the place where we can continue to train our IQ Bot and with this it's going to open up an interface where we can see the document on one side and then the fields we're trying to collect on the other so here's an example of the first document and it automatically formats these if we have similar documents it'll it's going to group them into one and we can see each document by clicking here and view each group so this is invoice one it looks like this and we can move to the second invoice and choose either of these as the training target and in this case when it reveals finishes processing what that document looks like we'll see the the appearance is the same but many of the labels are the same format and the format of the document is similar so it's put both of these invoices in the same uh in the same field so we're going to go back to our original the one it picked first as its training sample and examine what it was able to do through its analysis of this document and documents like it was able to say I’m able to confidently select the invoice date by this label so the invoice date field has a label on the document as invoice date and the field value is here 22nd august 2019 or 18. long time ago uh you'll notice in the scan you'll have little blue lines these are called system identified regions so it understands there's information in these blocks and it identifies which block goes with which label automatically so all these green check marks is IQ Bot saying this is what I think is right please check my work so I can check with invoice total we have I mean so invoice number it found a label called invoice no and it found the value of 10 20 80. for invoice total it found invoice amount at the bottom and associates that with the right data billing address it found a simple word ultron and what I want to collect is this whole billing address so if I can drag one of these handles and expand the box out to be able to capture the whole all those serves within that region for sirs so that's good I've corrected it a little bit on the payment terms the same thing payment terms as shown down here and it's picking the first of two terms according to this invoice so I’m going to drag it out to make sure we get the net 30 in addition to the two percent due in 10 10 10 days so that concludes the form fields the thing that stays static on the on the document next we're going to go to the table and the first part of the table settings has some interesting things some things we need to define the first is to I the label along the table that's associated with one line and has no additional information underneath it is the best way to describe this so item total while it's unique for each line in the amount section so item total here if we look at what's found let's find the amount column that's fine for this but it's got additional information below that what might be confused with this column if we go back and we look at other columns the best one is quantity it's here by itself it qualifies one line per data item and there's no additional information underneath it so it's going to save us with any confusion we can also tell the bot where the table section ends in this case we're just going to select a label subtotal that happens at the end of the of the repeated lines section once we've done that we can go and check where it's recognized the different things we're looking for so it's recognizing the column in this column we've got this little hyphen and sometimes hyphens are recognized as separators and we could it could end up becoming two different groups so stretching this out to cover that whole column is sometimes helpful but it's not required for item total it's nothing it'll grab this without any issues same thing with item quantity it'll grab the single digits here so at this point we've gone through each of the different fields we're interested in capturing we've identified and corrected IQ Bot where it might have made a poor selection and we can actually run this through the IQ Bot and see the extraction results this is taking this document running it through the classification system running it through the analyzer and pulling out information it'll show us the document it used and then also the information it was able to extract from that document so here's the first document has the one with the with the scripted a logo the information that extracted is here and we could do a quick check as well as the table repeating sections and seeing that we're getting the full descriptions for each one and then up at the top we can see these arrows enabled to get to the next document I can click on that wait for it to process and then we'll display the second document and the information extracted from there so here's the second document again not as stylized but still the same information and we're still grabbing the correct information the address is complete the full payment terms are complete the table repeated sections we have the right information for each one of those so we're done pretty good with this and those are the only two documents in this group so then we can go ahead and go back to training if we want to we can export this to csv it'll go through and take this information and put it into a csv file and we can check that against what we're using I’m going to go ahead and go back to training here's the csv file let's go ahead and open it and this just previews what this would look like at the other end when it is completed and it's going to open in excel and this is what the excel file looks like along the top of it we see we have the same data element headers as what we defined in our domain and this is the data it extracted for form field information like the date invoice number invoice total uh those are always the same so they repeat and then the payment turn the item description item total and quantity are different for each row so it flattens the data into a single file and that's what we'll see on the output as well now we're going to go back to training and this takes us back to the beginning where we identified labels and when we identified the labels we identified the position in the document where it's at IQ Bot does a fairly good job of handling registration issues if the if it was scanned slightly askew it can handle that they can also handle lots of fuzziness in the images if there's ever a time where the fuzziness of the image impacts the resolution IQ Bot will say I wasn't able to get good data and it'll put into what's called a validation queue where a human can assist and help make corrections to that and as the human is correcting those defects in the document or the defects in the scan the bot's going to learn from that so if it's something that's repeating quite a bit smudge on the glass of the scanning document or something like that IQ Bot can actually learn from that and improve over time to actually incorporate that that defect into its analysis and get faster straight through processing okay so now we're complete we've gone through it we looked at the extraction results we've seen the information here so we can go ahead and save this group of documents and go to the next group so remember we uploaded three documents we've seen two and now we're going to go and train on the third document which is a slightly different style than the first so we're going to wait for that document to load here's the second document from angel traders and it's formatted slightly differently and so it requires a different training or a different group to be trained same information invoice date it's finding correctly invoice number it says invoice number sign as the label but it's finding the correct information here invoice total same thing down here at the bottom it's capturing the correct label and the correct value for billing address we have a similar thing where it's only capturing the first name of the company we can expand this box out again to capture all of the address field and then come down to payment terms and here it doesn't have the green check mark it doesn't know it can't find what the payment terms are so we can inspect this invoice and see oh here's where the payment terms are here in the other comments label so the highlight here is field label if we click on the label other comments it's going to run through a quick analysis and figure out what it thinks the correct value should be in this case it's handled tax rate and that's not what we want we want it behind here so we can come and click the draw for this field value click on the draw icon and now I have a crossbar I could take this crossbar and then expand it over the place where I expect payment term values to be and then it associates the other comments label with this area to extract the details so that handles that comment for the table settings we're going to use the same thing we're going to find the best column that describes one per row and also has nothing underneath it so quantity again seems to be our best bet here the next thing we want to end the table at a certain section and again subtotal is the label that marks the end of the repeating rows so we'll mark as subtotal there and then we're just going to check to see how our labels are defined uh IQ Bot does a good job of identifying where the breaks are so as long as the header in this case the description is can aligned with where things are we're good with that and we can check and see the extraction results of this one just to double check and make sure that our information is correct okay so here's the output for the angel traders document and we can see the information was extracted correctly along with all of the items from some item to December soup everything was extracted correctly so we can go back to training and when we're back with training we're going to save and close this one and we'll be done with training this instance so we're back to the training situations we can we've identified all of our fields we can go and save this instance as well so both groups have we've reviewed and trained each group we're going to go back to the learning instance screen and once it says there we it gives us information about what to do with here and we can use upload documents with RPA to be able to process things we're going to set this to close and then we're going to set this learning instance into production training is complete and we can set this learning instance called invoices imagine to production and now it's ready to handle documents uploaded from the RPA side so I’m going to go to my RPA and I have a bot already loaded up here that's going to go through and handle these documents and upload them to the server and the process of this is just to identify which folder my documents are in so the folder that I’m working with here is uh c colon IQ Bot sample docs invoices which is this directory here which has all 10 of the different documents we're going to be working with then we go through a loop through each file in this folder and we're using folder path as a variable to identify that the output of this folder goes to a dictionary of strings and if you read the fine print of there there's two keys in this dictionary one called name and one called extension and we can use that in the upload process and when we go to the upload process it's going to communicate with the IQ Bot server to identify which learning instances are available and this is from a preview and then we loop through this folder and the output of that folder is a direct dictionary of strings and in the fine print it says there's a name key and an extension key for the file extension so inside this loop through files and folders I’m using those three elements the first one is the folder path for the directory that I’m has the files and then there's a direct dictionary variable with name key a period and then a dictionary uh variable with an extension key will upload these documents so with all that I can save my bot and I've selected the learning instance we created here from this drop down menu and then save the bot and now I'll run the bot and it should run and upload to my upload those documents to the IQ Bot server and as it's uploading the IQ Bot documents to the IQ Bot server we can see that now it's processing these documents and it's in the yet to be classified it'll go through a classified section and continually continue to move through these documents as it's going through the system so there it's received all 10 documents that I've uploaded it'll start sorting through these documents here when it's finished with the process we can go into IQ but I’m going to create another bot to download the documents or the extractions that we were related from this so I’m going to create a new bot in private and create a new bot for extracting the information so we're going to call this IQ Bot download to download documents we'll need to create a bot that uses the IQ Bot download to do this so we're going to download all documents for the download all documents step we're going to choose the learning instance name uh which document status it is success or failure you can choose success invalid unclassified or untrained in this case we want to get the results of a good training we're going to paste in our cognitive output here so that's the directory where we should see this and I'll go ahead and delete the files on the server when it's happening and save my response to prompt assignment apply that and do a quick message box to show the results of this process by showing the message that's stored in the prompt assignment I’m going to click on apply click run oh make sure I save my bot and then click run success that's what we like to see and success from my bot and we'll see the directory that I had the cognitive output has a list of files each of these lists of files has a unique identifier plus the file name that we use so the first file that we sent it is the tiff231 this one if we look at the results of it we can see we have a single csv with a single row on it so this is tiff one select all rows double click to see all the data and we've gotten all the data available for that it also tells the truth here so let me let me show that one more time close this open up this csv file and we'll see the results of the extraction from that one document so we can expand this out and see all the columns it has all the details from the form elements it has the details from the repeating elements and whether or not it was successful or not comes in the output and so these files would then be the next part of the phase that's done through RPA can be used to fill out things in erp or crm and this is the results of the IQ Bot process and I want to remind everybody to go be great have a great day
2021-02-21