[Music] hello and welcome to mendix world extending mendix with drama my name is alistair and i'm on the evangelist team here at mendix i've done mendix for a long time but i come from a technical background and so one of the first languages that i fell in love with was java and so it brings me great pleasure to talk to you today about java actions what we'll cover today is what java actions are and what they look like today how you can use java actions in your environment with some of your favorite ides as well as all of the libraries that you need to get things done and lastly we'll have a look at how you can extend and reuse your job actions simply throughout your organization in software today it's very rare we find ourselves reinventing the wheel when it comes to enhancing the capabilities of our apps there are so many businesses out there that are solving common problems that software faces in all technologies and what's more impressive is the open source community which has massive ecosystems that support businesses solving problems for a potentially much lower cost and when it comes to integrating with these services and this data it's not always the case that soap and rest services are the best way to go perhaps the service simply doesn't exist or there might be a performance implication to take into account when it comes to point-to-point re response time and latency of course there's a security consideration to be made when it comes to reducing the number of calls that are made outside of your network and the reason might be as simple as the fact that the client library code is written in language that is familiar with your team and therefore will be easier faster to implement and easier to maintain when it comes to influencing client library code and sdks in mendix this is done through java actions java actions exist within your mendix project and allow you to extend your minix application through custom java code and in fact because mendix's runtime is based on the jvm or java virtual machine it's possible to import libraries that are compiled in other jvm languages like scala and used within your menix application now before we jump into the demo i'd like to say that if you missed anything along the way all of the project files that we'll be using in this project will be available after the presentation so don't panic now before we jump into code we have a problem to solve let's say that we are an online travel agency and we provide functionality for our customers to upload photographs along with their reviews of travel services such as hotels car rental or tour guides now with this example and with many others like it there's a common problem in the digital world today privacy is extremely important and it is critical for us to protect our customers privacy and their personal information so one interesting way of doing this in our case for the example of uploading photographs to the platform we would like that whenever a photograph is uploaded by a customer and if there was a face either on purpose or by mistake included in the photograph we would like for that face to be automatically detected and blurred out so that the privacy of that customer can be protected as well as the privacy of others we are going to solve this problem by using two job actions the first job action will be to detect faces within a photo and for this we will use google's computer vision client library api the face that will be detected will come in the form of an x and y point which will indicate the beginning of the box where the face is located as well as a width and height which will indicate how large the box where the face is contained in is now the second java action will make use of these face locations in order to blur them out inside the image or photograph so let's jump into it here we are in studio pro with a mostly clean project all we have is a photo entity which inherits from the system image which allows us to upload photographs we also have a credentials entity which inherits from the file document and this allows us to upload the credential files that are required to integrate with google's vision what we also have is a simple page that allows us to upload photographs now in order to store the face locations we're going to create a non-persistent entity because we're only going to use this information temporarily we'll name it face location and this will contain all of the attributes that we need for the face which will be the x x and y points and the width and height which indicate how large the face box will be now once the entity is created we can start looking at how we create our job actions so i'll start by creating a new job action and we'll use the naming convention ja to indicate that it's a job action and we'll call it detect faces now when creating java actions we can add any number of parameters to pass through into the java code and these parameters can be of simple string types or numbers but they can also be of complex object types and these complex object types can be represented by the mendix objects that exist in the manx domain model that we have created so in our example we will be using a source image which will be the photo that we pass through to search for faces and this will be of type photo that we created earlier next we'll need to pass through the credentials file which will be used to authenticate with the google version api and this will be object of type credentials which we also created earlier and lastly of course we want to pass back a list of all the phases that we have detected inside the photograph so for the return type we select that it will be a list and we determine that the type of the list will be the face location which is the non-persistent entity that we created earlier now before we jump into any code we need to make sure that all of the libraries that we need in order to integrate with google's vision api are inside our project and ready to go so for this minix allows the use of automated dependency management by using tools that a lot that enable dependency management such as maven and ivy and so for our example we're going to use iv if we have a look at the iv.xml file we could see it's configured with a list of dependencies that we need in order to integrate with google cloud's vision so all that i need to do in this case is to make sure that i'm inside the correct project directory which is our which is our project and then i'll need to run the iv command which will take this configuration file and look for all of the libraries that i need inside my project based on the google vision requirement and now it will start loading the libraries and including all of the java files that we need in order to achieve this functionality and once it's complete it will let me know the status of the project and whether all the libraries have been copied in correctly or not and of course if the libraries already exist it will not it will not reply and not remove them or have an issue it will just let me know that they haven't been touched so now all the libraries have been included in our project and this will allow us to carry out from here now we need something to edit the java code so in mendix projects are deployed in such a way that they allow you to use popular java ides such as eclipse intellij jetbrains and netbeans so if we open eclipse we can import our project into eclipse using this existing project all i need to do is to make sure that the correct project folder is selected with all of my project files in it and click finish now all the java that i need at my fingertips is available once i've found my new java action and opened it we'll see immediately that minix has created some examples some boilerplate java code so what it has done is wrapped the action in a class it's also included all the parameters that i've selected in the model to be passed into the java and it's also provided an execute action method for where i can where i am to implement all the code so here we will implement a simple method for integrating with google cloud vision now the next thing to do is to make sure that all of the imports are correct for the classes that i need now that we've done this we can start to look at the code so the first thing we're doing is setting up the credentials for google vision next thing we do is convert the photo into a format that's usable for google vision and we set the feature type to facial detection and then we prepare the request for the api once the request has been sent the response is in a list and the list and the response that we're interested in is a list of faces that have come back from the google api so we iterate through each of these faces and for each one we access the what's called the bounding poly which is the box where the face is located and so we can use this to get the x and y that we mentioned as well as the width and height to determine the size of the box now we want to get this information back into mendix to use in our next java action so mendix also creates with to map to each mendix object that was created in the domain model a proxy java class which contains a constructor and setter and getter methods for each of the attributes that are contained contained in the entity and this allows us to create and interact with objects in the minix domain model so for our example i'll use the constructor to create a new face location and i'll use the set methods to set all the attributes that we need to pass back and the last thing i'll do is once the face location is created i'll add it to the arraylist that we created of face locations and pass this back to mendix we'll also see inside this java action that there is a there's a slot that's been given for extra code and this can be used to write any private classes you need to run inside this java action or any additional methods to make your code more readable so in our case we'll implement a credential provider class that's required to authenticate with google now that we've created our java action that will detect faces let's have a look at our next java action or let's have a look at our blurring images when it comes to the digital space images are stored in an array of pixels and the act of blurring an image is the simple task of taking a pixel that exists in an area that you would like to blur and mixing and merging it with the pixels that directly surround it so let's have a look at our actual our new drive action we'll create a new drive action and we'll use the same naming standard but this one will be called blur boxes when it comes to the parameters for this action the first parameter is going to be the source image which will be the photo that we added before now in our example we said that we plan to replace the original image with a new blurred image but if we're really going to make this java action so that it's reusable throughout our organization there is a use case where we will need to add a destination image as well which will allow us to retain the original image so i'm going to add a destination image too and this will be of the same type now the last parameter we'll need is of course the list of face locations that we detected using google vision so i'll add the list and um and i'll specify the type being the face location which we created earlier in the domain model now finally for the return value we're going to keep it simple and just return a boolean value which tells us whether the action was successful or not now we can rerun to see our new drive action displayed in eclipse and so if we refresh the folder we should shortly see our new java action so let's open it and have a look so this one has been created much the same as the others except the parameters are slightly different so now we'll implement a simple piece of code which will allow us to blur images based on boxes and again we'll just make sure that the inputs are correct once that's done we can start to look at our java action so in addition to proxy classes for all the objects in mendix minix also generates a load of helper classes and methods which allow us to integrate with features of the platform so in this case we make use of a call function called get file document content which allows us to store the source image inside a file input stream which can then be used later for image processing the next step is to determine what is called a convolution matrix which is used to do a blurring filter once that's created we can then iterate through each of the face locations that we have received in the parameter list and we use the dimensions that we received to create a sub image of the photo and in the last step we apply the blur filter that we've created earlier to that sub image once the loop is complete and all the faces are processed the next step is to write the new image to the destination parameter that we specified earlier and before we do anything else we make sure to clean up any streams that we used and of course return true if everything was successful now that we've done all the hard work it's time to put our java actions into work so we already have a microflow here that will be used to process our image the input parameter will be a photo that we have already uploaded and the first action is to retrieve a credential from the database which will allow us to integrate with the google cloud api so what's left the only thing that's left to do is to integrate our java actions the first job action we're going to use the detect faces so we drag that action on and configure the parameters and the source image will be the photo and the credentials file will be the credential file we just received from the database the last thing to do is to add a nice name for the output parameter which is the list of face locations we get from google next we can use our blur action so we drag that on afterwards for the parameters the source image will be the photo and as we said before we're going to replace the original with the new blurred photo so the destination photo will be the same and lastly as the input parameter we select list of locations that we got from google after we set a nice name for our return value our micro is complete now before we have a look at our results let's talk a little bit about how we can package and reuse our job actions at the moment if we have a look at this microflow the java actions look a bit ugly and a bit generic and in reality if someone else were to use them they would be very difficult to find and they wouldn't be categorized so what we're going to do is we're going to expose our java actions as microflow action activities which will allow us to add icons and categorize them into groups so if we open one of our java actions and expose it as a microflow action we'll notice the first thing we can do is give it a nice name so we can call it detect faces in image which is a little more intuitive next we can categorize it so in this case it'll be under computer vision and we can go one step further we can add an icon which allows to show for familiar brand names and functionality for example we'll use the google cloud version api icon and so we can do the same with the blue boxes so we'll expose it as a micro action we can improve the caption to say that it will blow boxes in an image and now we can categorize this one as image processing and lastly we add an icon to represent that it will be a blur effect so if we go back and have a look at our microphone now we can see there's a big difference now anyone who is using or implementing this microflow or these java actions can clearly see what the purpose is of them and what's also been done is it is much easier to find them so if we have a look at how we would find a job action again we have to select java action and we have to search for it in the right module but when it comes to these java actions all we need to do is search out of the correct category so under here you can see we have now have a computer vision category and we can use the detect faces action the same under image processing for the blue box so this makes it a lot easier on your colleagues for when they are reusing job actions within the organization and they don't even need to know that it's written in java all they need to know is how it works and where they can use it so let's have a look at our results here we can see we have an example a customer photo that's been uploaded with a review this is a great photo of a hotel room which contains the bed in the cupboard but the problem is it has a face in it so let's see what happens when we try to blur the face using our java action this will be sent this image will be sent to google as api the faces will come back and then our java action that will blow the boxes will attempt to blur them inside the image okay it's complete let's have a look and now we can see our photos now become anonymous the personal information of our customer or perhaps that of a bystander is now protected but what happens if there's more than one face in the photo let's see what happens when we apply our blur to this image again the image will be sent to google's api the face is detected and their locations will be sent into our blur action now you can see the photos have been blurred so the privacy of these people have also been been protected now i have very much enjoyed creating these java actions and if you feel like you've missed something or if you want to take part all of the project files that are available that we've used today are available in my github at mxcrawford and if you're wondering what are the things you can do in java we've seen customers implement custom documentated documentation generation with pdfs or word documents we've seen custom implementations of single sign-on as well as custom request handlers and many many more so i extend a challenge to you to go and have fun with java and mendix and go make it
2021-03-24