Use machine learning & artificial intelligence in your apps (Innovation track - Playtime EMEA 2017)

Hi everyone thank you for being here I hope, you are all comfortably. Seated, so. My, name is Albert Renault I work in the Android and play business. Development team looking after app developers. Across Europe and I'm based in London. So. Today, as. You heard this morning with, being kind of announcing. A few things and we. Kind of excited. You kind of do, all these announcements, we've. Been working it really hard over the past few months to kind of bring, keep, on innovating, within the console trying to bring new tools, trying to bring new functionalities. So, you can then come back home and also. Start. Innovating, so. We really kind of exciting, about having you here I hope you enjoying your stay in Berlin I hope that you're enjoying your day so far and as. You can see in this innovation, track we've got a pretty packed agenda we're, going to start talking about machine. Learning /. Assistant topics, and then, we're gonna discuss. VR and. Have. Some kind of panel, discussions and we'll finish with sharing, like insights, and research findings, about design. So. Let's. Start with the first presentation. Focus, on machine, learning that I will be presenting together, with, Daniel, from memorize and Hassim, from I am so. Earlier, this morning how, she gave, you a presentation by machine learning giving you like an overview of the key principles, of machine, learning as well as why. We, believe at Google that, this is important, in, this session we'll try to have a slightly different angle and, try to give you like insight on how as a developer. You, should be approaching it and thinking about it and also, how some of your, peers are being approaching, it recently. Within the app. So. I know that machine. Learning artificial, intelligence, probably. Hear a lot about it a lot of people talk about it it's probably one of the top buzzword, of 2017. Probably. A lot less people are actually really doing it but, and. I'm sure that many of you are kind of excitingly. Confused. About this but, if you hear at least, that means that you are interested who. In the room is actually, of, doing. Some kind of machine. Learning related project, within the app. Okay. So we go like maybe good, 20% and who is considering. It for the next six, twelve months.

Okay. So we go up to 50 percents I hope, that's at the end of this talk we're gonna have more, than 50% but. That's. Great so any. Case Isis you mentioned, machine. Learning has become the number one priority for Google we're moving from a mobile-first. World towards, a AI first, world and the, way I think about it is sometimes. Thinking, about five ten years ago these companies that we're. Not really considering, Mobile's seriously, in their strategy right. And actually, the success, of many, of you is linked, to the fact that all traditional. Players were not taking. Into their strategy mobile, seriously, and. We. Hope that you not gonna do the same mistake with machine learning today. So. It's. Good. Machine learnings already a reality as we saw this morning from. Kind of image recognition in. Google photos, content. Recommendation, on YouTube over a hundred, project, production. Ready project. Are actually, using. Neural network technology, at Google, but. You probably, thinking right now yes right this is great but, how does it apply to me right. I'm not Google I don't have the resources I, don't have maybe the skills, to do it how, should I be thinking about it and how should I even get started so. Doing this presentation we're, going to try to get look, at the key aspect you should be considering, when, kind of thinking about machine learning and. Starting. By thinking, okay are. You machine learning ready okay so what are the key initial. Conditions. Before. Taking, up a machine learning process, a project, so. First of all do you have a problem sounds. A bit silly but. This is probably one of the most common mistake we observe in industry, like people are, looking at what ml, can do and try, to kind of randomly, apply that to their business however, you should take the problem the other way around right you should look at your problems, and.

Not Simple. Problem like mission-critical, problems, and try, to see how machine, can help you solve these problems, this is important because you might end up, building. A great model super successful machine, learning feature but, it's useless because it's not actually not solving, a problem within your company. Once. You have a problem obviously, you need data you need a lot of data you need a lot of quality, data large, volume and you need to be able to kind. Of access. It and process, it acts at scale so that means that you might have to go through a process, of improvement of your infrastructure. You have to clean your data and you might have to even think about collecting, labels. For your data and, finally. You need to identify the. People that are going to execute this, project, within your companies and if. You don't have anyone please don't feel discouraged right away there are a lot, of resources, online that are available to everyone to train your teams Coursera. Udacity. YouTube. And all the kind of github. Etc, etc there's, a lot of resources out there that you can leverage, but. Ideally in an ideal world what we tend to recommend is to combine. Two, type of skills, on one hand what, we call data scientists. So people that are able to kind of conceptualize. Mathematical. Model that, kind of take into account your business requirements, and on, one and the other side we call data as engineers. That translate. This into code, able. To train the model and to kind of run it on production. So. At the beginning of you're going to be kind of tapping into whatever resources, you have but, the more you grow the, more we encourage you to think about structuring, this team into one centralized. Team. That, is then going to diffuse, their knowledge into the different division. And process of the organization. So. Now you have a clear problem you, have skills, you, have data you. Need to start thinking about choosing your model here. See this morning mentioned, describe. A few of them from. Classic image classification to. Reinforcement. Learning within self-driving, cars you need to choose your model and even, maybe a combination of several model and it's often where the magic happens. One. Key aspect of it is the, presence, or not of, labels. Within your data so, in the example, of image recognition you might have data that will tell you the object that is present, within your image right, but, unfortunately often. This might not be the case in, this case you might have to consider starting.

To Collect, these, labels, or, even think about choosing another model. That will not require these labels a. Lot. Of companies, in the machine learning field believe. In what we call. Open, research. Right. So including, Google all you need deep mine we believe in open research, this means that a lot. Of the research finding, a lot of the libraries lot, of the models are being developed are being put like published, online, available. Open source to everyone, this. Means that as a developer you. Will have to choose between, the. To what extent you will have you will rely on existing models, and to what extent you will actually develop your own models, right. So of course obviously, at the beginning we encourage you to tap as much as you can into our exist existing, libraries. That are built but. Quickly you might realize that this doesn't, really fit exactly your needs and you might have to either train. Your own model, with your own data or even. Like build your own custom model. As. Far as Google is concerned. We. Kind. Of we. Have this trade-off between simplicity. And. Flexibility. We'll also kind of determine. What technology. Called kind of solution, you're going to be using so as far as Google is concerned we. Can help. You along, the whole spectrum with, on the kind of ready to use side of the spectrum we got solution. Like Google Cloud machine learning api's which, gives you cloud dishin speech. And, translate, and natural language and of, ATIS. As well as well as action on google that we're going to be kind of discussing. In, the next presentation, but. If, this is not enough one. Of the most commonly. Can. Of use, use. Framework, by developer, is tensorflow, HT mention it this morning it's. Has. Become the most common use machine. Learning framework. It's you, can use it you can have build, your own custom model, you can use it to pre to train your model you have a lot of libraries, available. Out there you can kind of run it on different processing, units on different cloud platform, etc, etc and, as, he. Mentioned this morning we're, going to be launching soon tensorflow. Light which will enable, running. Machine, learning models. Directly, on device, with. This I will, pass it over to memorize. And I am do Daniel and have sinned who are going to share with you how, they been approaching machine, learning, within their own app thank, you very much. Hello. Hi, so. Thank you very much Hubbard and thank you for having us here I think it's an amazing conference thank. You very much so. Right. So my name is Daniel I'm the CTO of memorize and if. You don't know memorize we're one of the leading. Language learning apps, in the world with. Over 25, million users and 200, language. Combinations, to learn from we, really strive to make language learning as joyful, and effective, as possible and very simply, we want the best app on. Google Play we're very proud of this achievement. And. Today, I'd like to talk to you about how we build, product. Feature using, machine learning so I think I would ask you guys how many people are interested or playing. Around with using. Machine learning to before but how many of you actually have delivered, product. Using machine learning. Right. So not as many so it's obvious that everybody's really excited about the possibilities, of machine learning, but.

It's Not as easy to get from having. Ideas and using the technology to get something out there. So. We. We're. Doing hackathons, on a regular basis like every six weeks and we really want to give people time to play around with ideas and technologies, they think. Might work well on the product and this. Particular feature started as a hat so we try to think okay so what could we do it's object recognition and, after. A couple of days with developer, flying around it they. Came up with a very neat concept, which is turning the the, world.the to its own dictionary so imagine taking your phone looking. Around and, we, feel on German then you look at the screen you learn how to say screen or TV, in German, and there, was the idea so once. Once. We had that in place after the hack there was thought okay this is really cool how can we take this and actually make, this, something that we can give to our users, so. Don't we we started kind of. Looking. At the product and try, to define it better and what, we what. We knew we want to have we want to make sure that it's, very accurate because, we teach people languages, we can teach them the wrong thing it has to be in the right difficulty, level it and it has to be really fast and give like oh wow this is my like really bringing the, technology forward. The. Interesting thing is that, actually. Getting a machine, learning model or in this particular case. Object recognition was, really, the easy part, the, more. Difficult part was to get it to recognize the right things. So, if you guys look at that picture what do you see. Yeah. So a guy a man if I look at that which actually I see James who's one of the people that worked on this project now, we took this image and we gave it to Google. Vision API which is an amazing piece of technology and, this is what it came up with so, hair, facial, hair beard. Well, this picture is definitely very much fun but, I think that's, not the first, thing when we look at the picture we think it so, I think context. Is really really important, it's not about recognizing. Object is recognizing, the right objects and, we, spend most of the time really. Focusing on that. So. Once, we. After. Playing around with, a lot of models and the quite a few out there we, actually decide to train our own model so, we used the, based on was using Inception v3 which is a model which exists on on top of that we trained our own model so, what, we actually did is we, got. Product. Managers to to, collect a. Lot, of images, we first define the images that we care about so we, do a lot of user surveys, which was done somehow our users use the app, and. We know they often, time for example they use it at their homes so we thought about okay so which objects, are available, in their immediate surroundings so, we try to define that list of objects which were in the hundreds and then start collecting. Collecting. The images, and actually. Product. Managers spend a lot of time collecting images. From. From. Different sources and we found, out that actually we need quite a few example, and like, some, objects needed more than others for example people so as you can see here there, is a one example of has a secured engineer which is not quite a hair dryer maybe, needs one but. But. Yeah but so this was an example of something which was a bit difficult and as you can see there there. A picture, of bicycle, and a picture of glasses and they're both recognized, as glasses, something, that we wouldn't imagine that, it will you would never look at the bicycle sir oh it's glasses but, it's actually a problems, that emerge and we had to do, a lot of back-and-forth collecting, more and more images, focusing. On amateur. Photography, and. Also different, sizes, and shapes of the, objects that we really. Care. About, so, there are a lot of interesting examples in the slides were there were problems. But. By, iterating, and understand what it is that we want to build and what we care. What. We care about being the final result we, came up with a really neat product. So, it's just a few images from the finished, product that actually trekking has really, nice things like airplanes, and stuff like that and now were in the process of finding how. To integrate it best with, our product. So, just a few lessons learned, I think, like, I like. I said I think, even. With a really good model different. Data sets really equal different, products, so like you see so in the Google vision API it's. Not that it was wrong it just gave us we. Didn't give us what we wanted it to be really, carried in, our product we wanted people to look around them and learn, how objects are called so, really cared about single, objects, how.

The User is experiencing, it, and. Additional. To that I just want to say that I think machine. Learning I think as Albert said as well it's a tool you, want to educate people on how to use it there's a lot of resources out there educate, your teams and. Try to encourage. Them to use it and try, to. Find the right problems. That. Machine, learning can solve, thank. You very much and I hand over to Hoshi from iron. Morning. So, today. We're going to talk about how, we use machine learning at I am. To. Help improve, our workflow. For, the photographers and. What. Is I am so I'm in cyclic, combination. Of our community so that these are photographers we have around 20 million photographers who, upload their content onto our platform and we. Also have a marketplace where people can sell their photographs, for, for life thing which can be used by, brands. For their PR campaign, or advertisement, and to. Connect the right photographer. With the right buyer. And we, build the technology and, the. Technology we, build is called I envision so the our vision focuses, on understanding, everything. About a photograph so, what's inside a photograph, there's, a dog jumping a dog leaping a, stick. Frozen. Frozen, lake atmosphere. Outdoors. But we also want to know how good the, photograph. Is composed, how, does it a photograph, uploaded by any person, on the web compared. To the photographs, taken. By professional, photographers and that. That's what we captured through aesthetics, so we have a deep learning model machine learning model which can rate a photograph, a score. Between zero and hundred hundred, means that it is as good by a professional photographer and. So. The core of training, any machine learning. Machine. Learning model is the problem the, problem is, problem. You want to solve and machine, learning model, needs an objective. An objective, in this case, would be given. A collection of images give, me the right answer that, I expect, and when you don't give me the right answer, go, correct yourself so, this is the feedback that the machine learning model gets and as, you can imagine the, more complicated, your problem the, more complex, your model needs to be and that. Complexity, usually, translates, into, the size of the model in megabytes, or gigabytes. And. That's. That's, a concern when you to go mobile so if you want to run a machine, learning model on mobile you're constrained, on. The size of the machine learning model that can a phone can run on a CPU or a GPU, so that's an additional constraint that the. Model, has to take into account you. Can you can actually take, an existing large-scale. Big, complexity, model and then, shrink it down to D to the size you want there there are many methods out there, quantization. And compression, is one an analogy, would be changing, a PG PNG. Image, into a JPEG image for, all practical purposes their. Visual quality is similar but the more attention. You pay to the detail you start to see the cracks and. This. Is this is all and well once, you have a, machine, learning model that can, run on the phone. It. Can produce all the information, we need to know about about, an image you can detect, the tags you can detect what's inside the photograph but you can also get a score a score, how good the for the, photograph is composed, compared, to professional, photographers. And. Once. We have all the information the question is you. Can send, all the photographs to, a AWS. Machine or a Google, cloud and get.

All The answers you want why, why, go why, bother with, miniaturizing. Your mobile miniaturizing. Your machine. Learning model what's what's the advantage why. Spend so much effort into. Something that, is already working for you. One. Key aspect is, privacy so in in our case we are working, with. People's personal collection so it's your personal photos it could be photos, of your partners it. Could be pictures of your kids or private, moments and it's. Very tedious to upload your, your, whole of your collection, to cloud and get it analyzed by the algorithm and the. Second key aspect is that the. Current functionality is that multiple, people upload, their photos and they go to a single, server but. With mobile phones the, we have a perfect scenario we have one user and one computing machine the, handheld device and the. Third third. Reason is, it's. Available everywhere you don't need to be connected the internet you don't need to have Wi-Fi. And that. That makes sense because give. It taking example, of machine translation you, need the translation, the most when you're traveling abroad when, you don't have access to internet or Wi-Fi. So. You want your machine learning model to be run to. Be able to run everywhere. So. If you are a machine learning, engineer. Or data, scientist, that's that's, your part of the problem but this is the problem half solved now as the product. Managers to managers. Turn to, turn that into a feature and that goes through various cycles so once you have a prototype that could come through a hackathon that, could come through an, eternal effort you, build the first prototype and you, let the PM's play with it and they they can think about how they can use this machine. Learning model. In a feature and that's. The place where either your clients team the Android iOS and, the designer come together to, build a specific feature, the. Next step is actually getting user feedback so let let real users. Of the app or the feature play. With the model and get their feedback and this. Is this is the critical this, is a critical part of actually, retaining. Your machine learning model and. Collect. A new data set that new data set should reflect the feedback that you get from real users and that, the next stage is actually retraining, your machine learning model to, better fit. With. That in mind if we lost a feature a few months ago : selects, which, runs our deep learning models on the phone you, know it analyzes all your photo collection you could have thousands of photos and then. Then it suggests what, the best pictures you should upload to the platform.

This. Makes sense you don't want to upload your private photos and also. If you're a new user -, I am. You. See all the good photos and good photographers, and you get intimidated, what should i upload am I good enough this, machine learning model helps us help. Them find, the best photos they can upload onto our platform and this entirely runs on tensorflow we haven't yet moved intensive, low light but, that's the it's. Gonna come very soon. If. You have a machine learning model you can assess its performance, by various. Metrics there is precision, recall toughen. Accuracy but, you also wanna measure, the performance based. On how. Well your target feature is. In our case we wanted the users to upload not just more photos but also better photos and, so. When. A user uploads a photo that is recommended by a machine learning model we automatically, add, a hashtag to it called I am selects and then, we assess how, does that how. Does that photos, with that hashtag compared. To everything, else we have on our platform, and we saw that people were actually uploading, better content, over, time I. Think. That's it if you have questions for for, us please leave me. You.

2018-02-20 12:43

Other news