I hope, every is doing okay guys have a good morning Sasha. Session is pretty good and either Scots, or Pradesh's, wherever you wherever you spent your morning. This. Is one of my favorite shows there's a real left-to-right, view from sort of a technology, perspective a business perspective and. So I love this one this sessions, an interest, or a bridge session right. Saachi's probably, spent maybe 10 minutes you know power platform, and dynamics 365, and Scott might have gone another 10, with a demo this one's an hour the, intent here is to stitch together some of the pieces so, Sachi talked about sort of power platform, for extensibility, common. Data model a common data service, sort of data at the heart a. Little bit of power bi and power platform, for extensibility, and and an, allowing line of business application. Developers. And is v's to link things together so. What we want to do in this hour is try and stitch the pieces together and so, will will spend, about 3/4 the time on demos, and. You, know show and tell just. A little bit of an intro and then everything, we do here there's probably another hour deeper dive, so we're that bridge session between the two now, I I was, inspired this morning that that Sachi and I sometimes. That. Didn't work. Nice. Job. Where, did my power point go here. All. Right somebody want to give. Me a little hand on where. The power point went on this thing. That's. Interesting. Little. Why cancel means I'm gonna I'm gonna hide behind the podium here and I'm gonna pop back up and, pretend. Like nothing happened. He. Did a better job of it than I well probably but he had Scott come on stage and help him with that I had Sean so it's pretty similar yeah. Hi. He's a little Whittier so, myself. Orville, and Justine will spend, about an hour on this. Such. I you know he started, off in the gaming section with a video because he liked to look sort of backwards before he look forward he thought he brought out Donkey dot bass I thought okay line, of business applications are. Just as cool as games and therefore, we have old videos that are just as cool as that and so I went searching the archives in. The hour we had and I want to sort of go for it here so this is. Not. Quite Donkey dot bass but I think you'll be impressed with how old it is. Whether. It's IBM, RadioShack. Apple, or any of many personal, computers, they all have one thing in common software. From. Microsoft, Microsoft. Developed. A computer, language for micro computers, called basic, which, made possible the personal, computer, revolution.
80%. Of personal computers, are sold to businesses, now, for the first time knowledge, of computer, language, is no longer necessary, to do financial forecasting. Thanks, to multi tools expert. Systems, this, new software asks simple, questions, in plain English using. Just a couple of keys a complete, financial picture can be entered into Microsoft, spreadsheet, called, multi, plan then. Change, a variable and you can instantly, see the implications. Our. Life was simpler back then. It. Was pretty exciting too so I kind of love that one now there's sort of good news and bad news there the. Good news is we've made a lot of progress, the. Bad news is some of the core fundamentals, are still the same and. So if you think of most sort, of line of business applications as forms over data we. Still have you know a long way to go because that is the basic model the the three pictures here in the 70s, we were probably sitting in front of a green screen and, during, some information, on top of a in. That case a mainframe, in 92. By. Then we probably had a three to quarter inch floppy disk. Some. White likelihood, we were connecting to a server on the back end and in 2015. You know it's likely as a, solution but in all cases still forms over data Shawn, points out to me that the ergonomics, and the lighting are better as you get you know further along in history but. We have some work to do and change in terms of the model that we're working on if, you think about sort of the, line of business applications. And the journey we're on it's. An interesting time I mean if you look at sort of just. The realities, of where we are look, there's a lot of folks now that Kent, can and should be contributing into the solution, set our, challenge is often the contributions, they make aren't reusable, and so how do you help subject, matter experts, develop code that we can reuse or. Tools, or pieces and not have it be throw away. Programming, languages, tool sets, the. Environments. They're all growing you, know cloud hybrid we've, got old existing code we've got new code we've got more languages, you'll see us you know play around with that Visual. Studio code with Visual Studio with, the Azure portal. Lots, of places so we've, got all the. Number of devices growing the one I should highlight is data I'm, Sachi sort of hit this this morning we've.
Gone From sort of databases, data, warehouses. Data lakes look let's just call it a tidal wave we've, got more and more data coming in we're. Going to be asked to reuse that data to add either insights, or intelligence, as to what we're doing I would. Bet garner, that everyone in this room has had at least a few people asked about AI how, are we going to implement it how are we going to use it and, that all starts with data and so, all of these things are just the realities, of what, we deal with and then we have some opportunities how, do we use AI how do we extry ality how to use I owe t or, blockchain. Which was announced today I'll, spend most of my energy today on the AI side of it as we think about line of business applications how, do we subtly, infuse AI into, the the, work that we're doing and, on. That particular one through, each of the stands as we go through, they'll, be AI woven, in but the thing I'll highlight now is my, belief if you do a good job with AI it's. Not about sort of tada it's about making things work better and so if you do a good job with AI and you infuse it well things, just work better, that's not sort of some you know grand grandiose, change so earlier. You saw a visual studio with AI add it in we have AI in office we have AI and Windows but you don't see us say hey Windows inking brought, to you by Azure AI or. Visual studio now with Microsoft, ai it just works better and so we'll try and show you sort of the subtle additions, into the different you know phases today as we go because that's my belief in terms of how you make that work now. We've, been on a bit of a journey ourselves, the the, transformation, of all of our solutions, it's, been a sort of 10 year plus project, in office we went from office client, client-server to internet our SAS solution, on the, server side we've been on the journey from Windows, Server sequel, server BizTalk server up, through Azure and sort of the entire Azure conversation, today on our, line of business solutions, which started, with acquisitions, division. Great Plains, Solomon. Etc, we've, been on a journey for those as well to pull out the common elements, sort of Satya talked about power apps power, bi is the extensible, engines the common data model right. All built on top of Azure and then, turn the dynamic. Solutions, themselves into, SAS services, that people can build on top of so they're both solution, plus platform same, by the way for office 365, so. We want to move our you know line of business applications and our solutions, from, just forms over data to. Systems of intelligence that are feeding each other that make the whole process better, and make our solutions, better and capabilities. And as you, know users, of line of you know business applications, that Microsoft myself for 25 years as creators. Of them you. Know there's opportunity, here all right there's a lot upside so. For this session, we're going to break into three parts we're going to start with the data all, right if you're gonna have a conversation, around AI first thing we ever think about is how do we get the data sort. Of data ingestion data, cleansing, data wrangling pick your term how do we make that easier and then, the second piece of this is look let's look for insight first I believe, in bi before AI like, if you can't find insights in your data let's, not have a conversation on adding intelligence, so let's get sort of the data component, done and started with sort of begin there the second thing then is how, do we connect to and then leverage the power platform, how do we allow subject, matter experts, likely. Not professional, developers, but, to take advantage of some of the capabilities we're providing and create. Services, or extensions, or capabilities, that then can be added into the overall solution then. The last piece is how do we sort, of take all of the pieces Plus D 365. O 365. And build n 10 solutions, and for that we'll show off to is V solutions, and, some of the work they've done all. This gets stitched together we, have customers, get. Together and sort of create the extensions, for the common data model and sample apps they'll, allow us to more quickly connect, the work going on between different. Line, of business solutions, and different ISPs and different developers what, Satya commented, on this morning was look, there's no one sort, of apt to rule them all in any industry, and sort of anywhere to go and one, of the hardest challenges in this space is there's lots of smaller apps or lots of different vendors how, do we make it easier to connect the dots and that's what the accelerators, are about we're, going to use healthcare as our scenario we could have picked retail we could have picked manufacturing.
But, We want to pick a specific industry use. That a scenario, and then, our walk our way through each of the three pieces I've. Got a couple you know colleagues here to help me out the first section, of this is going to be data plus the power platform, I'm gonna ask just seeing the Justyna, lose Nick to come on up take. The first stands of this and then I'll go. Back in between all. Right it's all yours. Awesome. So. Today we're going to be looking at a feedback loop that's really relevant for any healthcare provider which, is patient satisfaction and, as, part of this demo we're really going to be hitting on three things firstly. How you can do self-service, data prep for big data on top of the azure data Lake secondly. How you can use Microsoft AI, technologies, to do AI enrichment and finally. How we can leverage power bi to build amazing reports, find insights and share those insights with others so. To start off with we actually inside, the power bi service and. You'll see alongside our reports and dashboards we actually have this new tab called data flow and we've, brought the ATF Louvre directly into power bi data. Flows are really powerful technology. That, allow us to create reusable and, repeatable, ETL processes, directly. Inside power bi you'll. See over here inside our patient satisfaction workspace. And, we have a number, of different data flows that we've created around, patient surveys Hospital, KPIs, as well, as patient historical, data so, we're actually doing here is when jesting, data from lots of different sources we're, transforming, this data cleansing, this data and saving, it in these repeatable, ETL, flows called data flows data. Flows can also be linked together so, you can actually length data from lots of different places and create a consolidated. And unified view of your data so. In this case we've actually created this patient satisfaction view. Which is bringing data from lots of different places, another. Really cool thing about data flows is it's actually built on top of the azure data Lake so as an analyst as you're bringing data into data flows you're actually operating, on top of the azure data Lake without even knowing it so, if we go ahead and actually switch this into this kind of list view so it's a little bit easier to see and we're actually going to pop up the azure storage Explorer, you'll, see that over here inside, are a storage. Explorer we're inside a TLS Gen 2 and we're inside the power bi blob container, inside a tea last gen 2 so, I can do is navigate to my patient, satisfaction workspace. Which, is the one I'm in inside power bi and you'll see all of these data flows see, this common data model format a common, data model folders, so, that means that all of the data that the analyst has brought in is now able to be used by others in the organization as, a reusable, data asset, so, for example if my let's, say para apps developer, wants, to go ahead and build a nurse app using, para apps the great news is power apps is able to read and write CDM folders too so, we can now collaborate, on top of the same data our, business applications, are also able to write CDM form and so, do all of our different Azure data services, so this means that all the different personas, inside the organization, are now able to collaborate together, so. Before we actually start, creating our different reports, I want to show you guys how easy it actually is to, do data transformations. Using data flows so, I'm going to navigate to my patient, surveys data flow over here and I'm, going to go ahead and, wait for this for a second and click, on edit over here and. I do not want to recover it from cash I want to actually other my entities, and, over here you're actually going to see we have brought in patient. Survey data so, we have you know lots of different patients who have filled in surveys about our hospitals, and we have a number of structured, questions, that we've asked them so we have some structured answers but, if we go all the way to the end we can see that also the patients were asked to add some additional comments so, we have some free text here as well now, if you've used power bi or Excel before that you'll see that this interface should be looking very familiar, to you and that's because we've actually embedded, power query directly into, data flows to, make data prep for, on top of the big on top of the azure data like really really easy because, we're actually asking, our analysts to use tools that they're already very familiar with now. I want to extract some insights from these additional comments, over here but, it's kind of hard to work on the BI tool with unstructured text, so, it can actually do is you'll notice we've added this new AI insights, button to, our power query in the web and what this allows us to do if we go ahead and select this is we can actually call the cognitive, services for, the azure cognitive, services directly.
Inside Power bi and so far we've brought in image detection, models as well as textual, analytics, models, so. If I want to for example extract. Out all the key phrases from my text all I have to do is select the column. That I want to analyze so in my case this is the additional comments and I'm just gonna invoke the function now. If I wanted to do something like this inside. Let's say a shirt I'd have to spend up the cognitive service I'd have to get out the subscription, key I'd have to figure out how I'm gonna call it you know I'm gonna store the result in power bi and basically two clicks as an analyst I can, go ahead and call this function and you'll see I've gone ahead and created myself a new column, over here that, has all of the key phrases that were extracted, so, if we go ahead and zoom into this for example we can see someone here talking. About the fish and hospital food so you know really really important things about their, patient visit you know they're commenting about how they found the quality of the food in the hospital, now, if I wanted to see whether they were happy, with the food or unhappy, with the food I could call an additional, cognitive service, which is my sentiment, function and figure, out if basically, you know they're happy they're writing him you know good comments or they're writing not so good comments so I've gone ahead and you know skip those steps over here for you guys and we, can see that in this particular case sentiment. Is zero point 197, sentiment. Generally between zero and one so generally, here they are not very happy with the food in the hospital okay, so now that we've gone ahead and actually added some additional insights, to our data I want to actually start creating my reports inside power bi so, I'm gonna jump into the power bi desktop tool, over here which, is our author in canvas, for reporting, and you can see if I've already gone ahead and started to create my reports, so, on the left hand side you'll see this map and as I do my demos you'll see I really love maps but, over, here in the map we've actually started leveraging, the sentiment, that we've extracted out earlier and we can see very quickly is you know which hospitals, have generally, low, sentiment, versus higher sentiment, so I can start focusing on the areas that need improvement on the, right hand side because I've consolidated all my data together I can start cross correlating, different metrics such as my hospital, KPIs like length of stay with, my sentiment, across time and, you'll, notice over here I have a placeholder visualization.
Which I haven't plotted yet here. I want to actually plot my survey scores across time and I want to use a heat map to do that now, power bi doesn't have a built-in heat map but that's not a problem because, power bi is fully extensible and you can actually alter your own AI visualizations, using custom visuals or, in this case we own actually, plot a Python, visualization. Here so, you'll see I've already started plotting, a Python visual if I select this you'll, see the Python visualization, is selected, and what I can do is very easily just bind it to the data that I want to use so, in this case I'm going to drag in my category my score and my week number I want, to find the average of, my score across week and to, save a couple of clicks I've already prepared my Python script here and I'm just gonna copy paste it in so, over here this is a little bit different usually power bi you know kind of your drag-and-drop but here I have my Python script editor and I can just drop this python script in and I can run this now, if I wanted to go ahead and use my own IDE I could totally do that using an external IDE script, but we've actually also embedded, vyas code directly into this power bi editors if I go ahead and start typing you'll see intellisense, actually pop up directly. Inside power bi so. I'm gonna make this minimize, this right now and you'll see that we've gone ahead and plotted, this heat map over here so, now we can we're ready to start analyzing our data so, let's go ahead and keep this a little bit local I'm going to select a hospital, and Washington, over here and, gonna select King County, and, I'm. Gonna do that and let's, zoom into the map a little bit and I'm gonna select a fictitious hospital, here in Seattle, where the sentiment, is looking, pretty, bad it's you know currently at 0.1. And you'll, see when I selected, this the whole right-hand side has, cross filtered so I can adjust this are getting insights for example I can see that some of my KPI such as my length of stay and my, wait time are generally higher whereas, readmissions, rate is lower compared, to the average, sentiment. Used to be kind of okay but thinks I've really been degrading. Across the last couple of weeks and if we zoom in to a heat map over here we've got all our different question categories listed, and generally. Things look kind of okay like for discharge, doctor care nurse Kerr scores, have been pretty stable but there's two categories where things have really deteriorated. Over time which, is hospital environment, and hospital experience so, what I gained some more insights about that so I'm gonna actually flip this from looking at my data to a more qualitative, view, and I'm gonna look at all the key phrases that have been extracted out for this hospital in the past month so. The good news is I don't see anything about fish trending, so you know that's kind of good given the last that we looked about but, I can see something, like you know words like unsanitary. And cold, and dirty popping, up and you, know if I am a health care provider these are the last words I want to be seeing, you know to be described, my, hospital, abouts so that's, a little bit worrying for me so. I really want to kind of focus more on this, particular hospital, but I want to actually start running some statistical, analysis, to see what is actually driving sentiment. To go down because currently with casino selected this hospital we've seen how the metrics, kind of change, and we've been piecing things together but, I want to really understand what is driving sentiment, to go lower so, I'm gonna flip tabs over here and you, might have seen this during Scott's keynote, we've, introduced, or we are introducing, the concept of AI visualizations. Into power bi and the first visual that we've altered is called the key influencers, visual which helps you understand, what factors are driving a metric that you're interested in analyzing so. I'm going to actually alter this from scratch over here I've, set.
It Up a little bit so I've told power bi that I want to analyze sentiment. And we've started the question to ask what influences. Sentiment, to decrease, and this is set up to be looking at our Seattle hospital we were analyzing earlier and now, all I have to do is start dropping in factories that could potentially be, you know influencing. Sentiment, so for example maybe the department that the patient visited has some sort of influence on their experience, and as, as soon as I drop this in power bi she runs a regression model behind, the scenes and automatically. Tells me yes this is indeed important, and we can see that when patients visit the emergency room generally, their sentiment, is 11 percentage points lower than when they visit other departments, inside the hospital and we can see a visual representation of, that to here on the right-hand side but. We don't stop here we can carry on dragging, in additional, factors and every, time I drag in a factor you'll see power bi reruns, analysis, andrey, ranks all the factors in real-time so you can do interactive, you know statistical analysis, in real, time and power bi and over, here now we see that you know the factors have reracked the, top factor is indeed, dirty you know so my deduction, was correct those key phrases didn't matter and when, patients you know would describe the hospital, as dirty, generally, you know their sentiment, would be really really low but, not only can I do this you know with one Hospital, at a time I can start comparing different hospitals, to each other so, what I've done here is I've actually set up two hospitals they, have very similar sentiment. Scores you, can see this one zero point one three this one zero point one one but. If you look at the factors, you know we took a look at the right hand side one on the left hand side we can see things like surprising. Things day of the week is Tuesday, so, patients for some reason when they come in on Tuesday they, are a lot unhappier, than they wouldn't they come in for the rest of the week another, you know metric that came up this isn't that surprising is, when they wait over three hours but, the important part here is even though we have two hospitals very, similar sentiment scores the, key influences visual has surfaced different factors as the ones that we really need to focus on improving, so. That I want to just jump back into slides and do a really quick recap of what we just talked about, so. Let's, move to the next slide so, over here we know we started with data and doing the data prep and we showed you how if data flows and analysts, can really, create reusable and repeatable, ETL processes, using, tools like power query which they're already very familiar with when, they store data and CDM, form in the data Lake they can really collaborate, with their data scientists, their data engineers, and their business users really, easily. So we minimize these redundancies, and create reusable data assets with, AI technologies. Built into, the data transformations. We can enrich data very easily and, with power bi with things like Python visuals and AI visualizations, we, can really you know do things like find meaningful insights, and share it with others and with that I'd like to pass back to Steve thank you. Thank. You Justin Tina. So. Sort of date, at the heart to start with all right for everything we want to do for both line of business applications, and AI and, yes the the notion, of being able to do sentiment, analysis on the food at a large event in a word cloud is likely to be, the fallout from this for build, the. One thing I want to highlight in this last slide is looked, at a a, AI infusion, right throughout all of the aspects, of sort, of our bi tool set and power bi weather, it's sort of the cleaning, of the data and looking for insights or sentiment, whether it's the ability to work with Python, whether it's the ability to pull in Azure. Ml, or Azure frameworks, it's, not sort of some big different thing it's just woven, in seamlessly and you'll see more and more of that over time and so that's sort of first step so thing. One data we've, cleared that so, the next part of the conversation then becomes okay how do we sort of allow subject, matter experts, outside, of the data scientist in this case but let's, say in a particular industry, or in a particular area to. Either modify, some of the tools we've built and you'll, see a bit of that with a virtual agent and sort of the notion of the somebody who's the subject matter expert or within, the. Other. Areas, of the code or the platform we want to build so we're gonna we're gonna leverage power apps for that we're gonna leverage a little bit of the new AI toolset.
One. Thing I'll point out on this one is in, our April release of both Power Platform and sort of Dynamics 365, we, released a whole bunch of new capabilities and, so as we go through this we're building, on top of those capabilities and, there's more and more being added into the platform both. From the developer, perspective and from an ISV perspective, and in terms of building end-to-end solutions, so. With that I'm gonna hand off I sort, of fast, forwarded that to my colleague over McDonald who's going to pick up the next piece of this which is the power apps what'd. You call it citizen developer, or subject matter expert. There. Is so, for. Me all right it's all yours man thank you Steve. It. Started here. Perfect. Oh actually Alvin you switch over to eight, great. So. As many of us in this room know developers. Often have way, more on their plate than, they actually have time and resources to handle but, kind of at the same time with in many of our organizations their. Subject matter experts, people, who really know their areas very well but, they have a problem they don't necessarily have the toolset or the skills to, develop the apps they need to kind of get their job done sometimes. What. Ends up happening is, that they'll then go and, hack together a solution and often. At least based on my experience, that, solution goes viral throughout, the organization. Everybody. Adopts it and then, guess what the. Developers, get stuck trying to figure out how to maintain that app and then, they're stuck with the dilemma do, they just keep shining, it and try to make it a little better or do, they throw it away and kind of start anew wouldn't. It be great if we could actually have the same tools used across both the developers, and the, subject matter experts, so that way when things are built that asset, becomes a better, and bigger piece for the organization, well, let's see how we could do that using a combination of, a flow, BOTS. AI and, a wide variety of technologies, so. Right here we have a bot. For once again we're continuing with healthcare for lamina healthcare and I'm. Gonna go through and I'm just gonna activate, it so it says you know how, are we doing today, not so well because my son is sick and for. Those of you who have written BOTS this is just the azure bot service, working behind. The scenes we have Lewis so we're able to pick out the intents on this. Case my son has a headache and not. Only does he have a headache but I think it might actually be a head injury which is really unfortunate so. I click yes now. I'm given an interested, interesting. Prompt right I have a choice I could either go into the office to get my son checked or, I could wait for somebody to come visit me in my home, anyone. Who's familiar with the space knows that a head injury is actually a very serious thing and it needs actually more immediate, attend, a situation. Like this may not necessarily rank. The highest on the developer buck chart but, a subject matter expert would, actually know how to go in and make the necessary fixes let's. See if these tools that we have might help us out so we're going to go through and we're going to take a look at the virtual agent designer so, at Microsoft, we spent a lot of time creating BOTS and one, of the teams that I used, to be a part of created. The dynamics 365, AI for, customer service and with that we, came out with some interesting authoring, tools as far as creating, our dialogues, so. You can see here we have a list of all user topics, one of them being sick and at, this point this is where people start to get nervous because, if you go and you update a dialog, you. Can introduce a bug and even worse you, might have a subtle unintended, consequence.
That People, can't necessarily pick, out you. Can see here I have my dialogue, tree and this, is just one component this, is the dialogue tree for when you're sick you, could imagine as you saw before in our basic example, there were only 25, but, it's not uncommon to get up into the hundreds, when it comes to dialogue trees so, being able to pinpoint where your problem is can, be can, be very difficult in, this. Case we have some tools that we could help with we. Have our virtual agent and the feature that I personally like is this ability to do tracing. When. You do tracing, as you type things so as one. Of my co-workers told me BOTS have feelings too so I'll give it a nice kind greeting with hello it. Will actually go through and it will pinpoint we're, in the dialogue tree we are here. We're at the greeting stage and I'll zoom in a bit so people can follow along so. We have a greeting once again I'll go through the same floor I previously did so my son is sick. Put. That through and it will actually move out from the greeting dialogue tree and dynamically. Automatically, go into the sick dialogue tree you. See I had the same options as before and I'm. Actually flowing through within, the dialogue tree on kind of what exactly is happening once. Again I say that he has a head injury and. Now. I have my list of options so. For subject matter experts, this is great they can pinpoint exactly where, the problem is but, this is now where everyone starts to get a little worried you, might be thinking in order to update this dialogue tree maybe I need to the JSON the configuration, files and make, some hand tweaks to it not, a good idea however, right here I actually, had the option by just clicking on this plus sign that. I could then update the flow in. This case I'll choose kind of what the bot says you'll notice I get an error message you could safely ignore the error message it's, just that I was walking through a trace and I've now interrupted, it because I'm gonna go and edit it I'm. Gonna go and add some text and thankfully. For all of you you do not have to watch me type it now. This copy. And paste right there and. What this message says is pretty much that hey. Give us your phone number we'll contact you, and if, it's an emergency like maybe you should just call 911, so. This is great we had the message updated, but, this is where once again the, details, really matter right, in order to capture this phone number phone, numbers can be in a variety of data, types it, could be an integer it could be a long it could be a character array it could be of type string and that. Decision as small as it may seem could, have ripple effects kind of throughout what's happening you don't necessarily want someone, who doesn't understand, software engineering, to be manipulating.
Something Like that but, once again we have something to help us out as, I. Go through here I just click on user says I don't need to capture text, but, I would like to capture a variable, in this case I just simply say to add and to, create a variable I'll give it a name of a phone. Number and. You. Can see from the drop-down I have a variety of options but for this demo I'm just gonna keep it simple with text. Go. Through I click done we, see that my dialog tree automatically. Updates, and, remakes. All the connections that are necessary, so this is great right I'm feeling good I have, my phone number variable. But, the next problem I had now is I want an action, to be triggered when this happens for. That we could actually take advantage of flow and can I see a quick show of hands of how many people in the room abuse flow before oh, great. Ok good, usage for, those of you who haven't used it I highly recommend, it um you, can think of it as workflow. Orchestration. Or kind of an RPA type tool and in, this case when. I receive a request you're gonna see it's just basic, HTTP, I have, my post URL endpoint. I have my schema, and, what I would like to happen here is to thing one. I would like a notification to go out alerting, that like hey we need to see this person and then, to because I know the nurses only have so much time I'd, actually like to schedule an appointment to, block their calendar, so, that way they, have enough time to do what they need to do in. This case I'll just click on adding an action and. Here. I have a variety of options I could do different things like the notifications, which I'll do I could, take advantage of o365, I, can even do things like send out tweets I highly. Recommend if you're doing anything with sensitive data like healthcare finance. Maybe. We should have send tweets but, I personally still kind of like this feature there. Are good instances, for this so for example if you, work for a utility, and the power goes out in your neighborhood right, maybe being able to automatically, send out a tweet letting everybody know like powers, out don't call us we'll keep you updated our going on is, a good feature to have in. This case I'm just gonna add a mobile, notification. And this. Is where it gets interesting as well because when, I send out this notification to the nurse I want to provide some information, right but, I often worry about hard coding these things because then that gets very hard to update but, there are some capabilities, that help us out here so. In this case first I'm going to put down kind of the boilerplate text but. Then over here you'll notice I also have dynamic content. By. Just clicking on that whatever is put into the to field will automatically, get populated, and just. Like that the subject matter expert, is able to adopt some best practices. But. That's not the only thing I want to do I also want to create the event you, can see here I could create a parallel, action, so this will happen at the same time as well and. I'll just go through and create an event, from. The drop-down I can see the calendars, that we have access to once. Again I'll put the same, title. In, here. And also. Include the dynamic content so this is great here. You're gonna see me actually enter, some hard-coded, dates but. That's only because this is a demo in real, life what. You would actually want to do is, have. Some dynamic variables, that go in there as well so. This. Is great so I have this kind of updated, and the other thing you can think to is you could actually go and create your own kind, of custom. Components, and flows and make those available for people in you organize a, so that way the subject matter experts, don't necessarily, have to play with this so it depends on kind of what your comfort level is without. Different people kind of updating your code so. It's going through here and saving, and this, is great but sometimes I might want to use this type of thing even outside a flow in. That case I had the option of actually exporting, this you. Can see here that I'll just go through the export, and I can actually export as a logic apps template, and in, interest of time I already have it open here in code so you can see it's your standard JSON right so, we saw very quickly how we were able to kind of create that and send. That back out for use elsewhere so. This is great but, we still haven't finished solving our problem we want to be able to take advantage of this notification, once.
Again, The. Details matter you might often be wondering like oh this would be difficult to take this variable plug, it into this other piece but. Not exactly, just. Go through here select my action, and, then. The list of actions become easily available we see that we have our notification. And. With that we, just simply map it to the phone number and, we've updated our dialogue tree so, you saw how very simply taking advantage of kind of the virtual agent designer and flow. We were actually able to get subject, matter experts, to, work well with their developer, counterparts, isn't this something all of you could use. Come. On I know like. Great. So that, was a good start but now let's take it on to the other side right so. In the case with our nurses there's. Times where they'll need information, at their fingertips and, they would like to have apps that could help them with that as Justyna. Showed earlier she was able to take power bi take. Data from a variety of sources, wrangle. It together and make it usable to get great insights, if we, wanted to we could actually start on the power app side and do the exact same thing import the data we need and do the data wrangling there but, because justine has already done the heavy lifting for me I don't see the need to duplicate. That so, I'll just actually take advantage of the work that she's done and plug, that into my, application. As. We move over to power apps you could see here that I have my canvas, app and the. One thing I want to call those within the tree view you can see that I have a variety of components, so. You could actually develop the components, that you think. That subject matter experts, would like to use and just make that available that way you maintain the right security the, right data model, the right consistency, of the user interface and let's. Just go and run this and we know that power apps can run within the browser or they could even run as a mobile app here. I'm gonna click on the nurse link and you're. Gonna see profile, information for the nurse so this is something that's unique to our organization. Maybe sits in some type of sequel, data warehouse, or database somewhere and, we're just able to pull that in on the. Other hand there's. Also sensitive, information that we'll want to take care of because we're focus on healthcare for this demo purposes, I'm going to take care I take advantage of the fire standard, so, fire what that is is it defines kind of the common data exchange, for healthcare information it's, an industry standard just like many other industries, have their standards, as well and I'll, go through and I'll click on my patients, I see my patients listed, and down, here I see my son Ashton as I.
Click On that it actually goes out to the fire server pulls. In the electronic. Medical records and displays them here however. With that I might also want to go and take a look at an appointment that I have with Ashton as I, click through through the appointments, we're, then plugging into all through si0 365. So. You could see that even within my power app I'm able to pull from all these various data sources yet. Create one consistent, user experience and, with. That as I showed, before with flow whenever. A records created within Dynamics, I could, also go through and then create an event for the nurse and also. Export, that out as I'll show here in a visual. Studio you. Could actually see a bit of the JSON. There that's, used for connecting for the fire server but, instead of walking through the code what I'll do they'll make it easier is actually, if I switch back to the, PowerPoint presentation, actually, here on one-five. You. Can see the architecture diagram, that shows as we pull in data from a variety of systems of Records everything from demographic, information, clinical. Trials, patient, information I can, then flow that through to an ingestion process, and then, take advantage of the azure fire server in order to kind of persist that data and then, present it out through our various endpoints, as a system, of intelligence. Know that this is just one sample implementation there's, a wide variety of implementations. You could do for health care data and, once again this pattern will also work for other. Industries. As well so, as a quick recap we were able to see how with subject. Matter experts and professional, developers, we, could create the right tool sets across the, virtual agents assignor the. Power platform, health. Accelerators. Of which we have more accelerators, as well and the, azure fire connector, so, with that I'll hand it back over to Steve. Excellent. Work all. Right so we're continuing. Our tour de force the, last thing I want to do is just kind of quickly show off to IC solutions, and we'll loop back around on the on the power bi side the one. Of the things that both Justine and Orville talked about was this, ability to get, data into the common data model folders, to reuse, it one. Of the things we have to do though is the the core, common data model gives us the entity structure for I'll call it the standard things we might all think about but, as you go industry by industry you need more specifics, so in the healthcare space we need the notion of a patient. Or a provider or a healthcare record etc so, what we've been doing, is going and working with, industry experts at them the ISV level the SI side and customers, to extend, the common data model for those specific industries, and so we've got a healthcare accelerator, out there that's what we're using today we've. Got one out there today for, the. Nonprofit. Space we have one out there in education, at. The show here we'll move into private preview for one for automotive, and and to actually in the financial services, space but. These are the the tools that we're working with the try and give a kickstart, for each of these industries, and allow us to stitch pieces together and so, both. Of the, next. Scenarios I want to show are samples, both. Of the participants, have worked with us on the accelerator I want to show two hands-free, health. First is a, new actually device with, software that's meant, for the home in healthcare and the second is into gene which is more of an end-to-end solution hands-free. Health is a new start up Mike Hardy who was the Cardillo. I said that wrong former, Aetna CEO, kicked. This off about a year ago and. You can actually see the device here on stage if you think of any of your in-home devices. That you talk to this, is your healthcare one so a hands-free health that's what it's for I'm gonna switch over to seven. Did. We get the switch which is my mobile phone here and so, this is the if I have a device like this we need to set it up and. You could I guess talk to it for a while and try and set it up it's a little easier to configure through a mobile phone app so. We've got the app here loaded and, you can see the the, profile, the, medications. Etc and, this, is nicely, built for weather the person who's using this is setting it up or if I think of my parents who are elderly or if their IT person, aka me or their health care provider, you know in-home healthcare provider is setting it up and, so you assume I'm setting this up for this. Person if I go in for example two medications um, you'll see these are the medications this person takes if I want to add one add new, medication, you'll, notice I could enter the the text, here myself I could.
Speak Into the device if I wanted to again if I think of okay my folks the answer's no that's not working but, I do want to take advantage of some of the capabilities we've, been showing today so, here I've got the camera I've got the pill bottle it's. Full of Medicine and. I'm. Just gonna take a picture of the label see. If I can hold that steady. Use. Photo. And. What its gonna do is it's gonna fill in the form and. It's basically says this is aspirin and, that's what's in this bottle but but, here I've used the cognitive, service the new OCR, capability, that taught you talked about this morning I know he highlighted it but it's in the release and. This, is a pretty simple form so in this case it's just looking for keywords it extracts, them it fills in the form if this, was something more complicated let's say we were looking at a. Real estate form or something with multiple pages or had handwriting, in it we might want to train the model in this case though super easy to use I can save this and that'll, get added into the record right, so now we've. Got our device it's set up it's in the home if I, switch over to let's. See which. Machine is this eight and. I switch into my desktop. And. Let's, see I want, to go to three and. I want to come in too. Here. We go so this is the well be app and we're going to two simple things here we have a basically, an azure function, setup and we're sitting in the azure portal the, function, triggers two things if I go, into this it's gonna Osan, to start with the cognitive service, that was the first thing I did so, let me pop back. Sorry. About that let me pick off the first thing here which is basically, taking advantage of the cognitive service, so here, we're sitting in xamarin we've got an application, for, both iOS and Android so, we use xamarin inside of Visual Studio and, all I have to do is call essentially, the cognitive service I can look for faces I can look for images we're, just going to capture this particular image and. Then we're gonna use that to fill in the form so that was sort of step one and you a very simple piece to do then. If we take we take the device, itself now we want to take advantage of the. Device so we've set up a trigger. Here we're, basically just listening, to the endpoint for this device I've. Set it up to do two things if I, what's, supposed to happen here is I've entered the medicine, and I've entered how often they take it the, person's supposed essentially, it let's say four o'clock in the afternoon my, mom's supposed say hey I took my medicine if they, don't then we've set up a trigger to do two things one is to send. A notification on, that device which will turn on the light says take your medicine, two is to place a record into, the into, the provider, whoever the providers.
Is. For this particular patient into. Their record using, the flow. Flow. Toolset, basically, Orville showed us this so, we'll go ahead and post a record in - in. This case we'll use it back into the azure or, sorry the accelerator. We'd built the healthcare accelerator, so that way anybody else can take advantage of it so if I go into the. The. Welby system. Here and I go ahead and run the app I'm gonna do a quick run it. Should trigger in a couple minutes here a green light over on this device which in this case is the signal it would tell you hey go take your medicine. So. We'll give it a second. In. My. Practice that took a couple seconds. Hmm. Takes. A couple more seconds, a couple. More seconds, somebody. Tap dances. Voila. There we go green light comes on I know, as crushing. Crushing soul-crushing, when they don't work I, have, lots of good demos, that don't work stories if you'd like afterwards, so. Well. Be device set up and go we also wrote a record into the into, the healthcare record. In the accelerator, so, second piece of this then is if. We go ahead and pick up the end of gene system. Here. Then is more of an intense solution, now indigene, basically, as a company, provided. All of the records, from or all most of the records from pharmacy. Companies, to, doctors, and health care providers on the information, about drugs that, was their primary business about, a year ago they said hey we want to build more of an end-to-end solution we. Want to take the capabilities, we have we, want to build a patient. Doctor. And and, farm a connected. Tissue essentially, a cloud for all three of those well I want to be able to, you. Know share the information across them and so, they built this up from scratch and so if I go in here this is the same patient so, here we have the core patient information if, I look at the health timeline, you'll. See a trigger here this is the aspirin the miss dosage that was the medicine, I added they missed their dosage again, the accelerator on the backend allows the connection it allows hands-free, and indigene to talk to each other without knowing each other I don't have to write a custom connector for each one of the providers. Here I don't actually have to know each other I'm if I look at touch points you, see this device. Conversation. This, device over the over time will allow you to ask questions so in this case the patient has asked a question the device question. Comes into the provider provider. Can either answer the question or if they want to they can go out to the pharma company so, if you think about those feedback loops where we have data coming in from a new sort of IOT third party device that's, feeding the system we can then use that data and cut it across from, I'll call it supply chain in this case with Pharma to customer, in this case which is doctor, to, you know third party which is patient, and so we're trying to create those loops together and, allow all the systems to connect and. So to. Sort of quick examples in, this particular case. Flip back to. The. Slides. That's. Me so let's get me for a second in. This, case on the indigene side this was about a year project, they started with three of the dynamics, 365. Solutions, sales, marketing, customer service alright they took advantage of power apps they took advantage of flow they. Connected into Azure in. Terms of the sequel, database blob, storage then they took advantage of cosmos DB they. Used kubernetes, they created, the connection to all the different endpoints and ultimately, were to create able, to create an end-to-end solution, in about a year take. Advantage of all the pieces and that's sort of the the, modern application. Sort of life you, know line of business application, scenario from scratch using, all the tools using, the accelerator, and off you go so whether you're building from scratch or you're extending, you know we want to make sure that whether it's the power, bi side the. Power app side the D 365, side they actually use office and, short, of sharepoint all the pieces can be connected together and if you want sort of a more detailed breakout of this that, would be a good follow-on session that will do the.
Last Thing I wanted to do was I wanted to sort of come round trip we. Started, in power bi, with. Using a little bit of AI to look at the data that, sort of backwards, looking it's very useful, helps, us with the information the, last thing I want to do is sort of go, from sort of looking backwards to looking forwards how do we use sort of data in power, bi to become predictive, alright, so in the last scenario we used a are in the cognitive service, we use in a couple other areas in this particular scenario I want take the last step of the journey and then so I'm gonna ask Justyna to come back finish. Off our healthcare sort, of record, in our healthcare journey with a predictive. Solution in power bi sounds. Good. All. Right, so let's jump back into power bi over here and. As. Giggs mentioned, you know so far we've used power bi to really, analyze your, data and extract out insights but now we want to switch gears a little bit and go from a retrospective, view of our data to, more of a predictive, view so, I'm back inside data flows inside the power bi service and, the scenario that I want to look at is really estimating. Bad's, demand, so, you know there may be certain, hospitals, that have over capacity, under capacity in the following week and when we want to get better at being able to proactively you, know kind of move patients around to fill our hospital. Bed allocation, better so. We've got some historical, data inside, our data flows over here around, there unplanned, admissions, and we want to do is actually build a predictive model to be able to forecast, our admissions, for, the next week and you'll, notice this little. Icon over here inside our data flows which is this, brain or the lightning bolts as you can see it's very very smart, and what, I'm going to do is I'm actually selected. And you can see we can add machine, learning models directly, inside power bi data flows and, what we're actually using behind, the scenes some Astra technology, called, automated ml so, you can use this in Azure today. I think, Scott talked about it inside the keynote as well but we've actually embedded, that directly, into power bi to our to give our analysts a point-and-click, experience. For building machine learning models and I'll show you how easy it is to do in just a couple of clicks right now so. The first thing I need to do is actually tell power bi what, it is I want to be able to predict so, we're gonna go through this list of variables over here and we've, got our weekly unplanned, admission so this is again historical, data so this is my known data and this is what I want to train my machine learning model on and, I'm going to select next, and power bi is going to give me a few options of the types of models, that I'm able to build and I'm gonna go ahead and select a regression model because I want to estimate a numerical, value so. I'm going to click Next power bi has already done an intelligent, selection, for me of the variables, that thinks I should be using for the model but, if I want to change these, around and say no I want to use these other fields as features I can override the selection, two and. Really the last point I have to do is just give my model a name so, I'm gonna go ahead and select something like admissions. And I'm, pretty much done with building, my machine learning model now ultimately the Mel is gonna run behind the scenes and is actually gonna do all the heavy lifting so, it's gonna do the feature selection it's gonna do the algorithm selection. It's gonna do hyper parameter, tuning and there's gonna iterate through lots and lots of different models until, it finds me the best possible, model that it can build but.
Instead Of waiting you know for this to happen for next 15 minutes I'm gonna cancel out and I'm gonna show you guys a model, I've already built earlier on, top of the exact same data so. Over here I can flip from my entities to my machine learning models I'm gonna, select this and I'm, going to go into my performance, report and what, this is going to show me is actually how well the model I've built is doing, so, we've got some of our you know important. KPIs, here, such as how much data did we train on what, was the final model that was fitted, we see our model performance, and we see other diagnostics. Such as my predicted, versus my actuals, I've got my residual, errors and the right-hand side we've actually got a detailed, explanation for, analysts, so that we're not just throwing data science, jargon at them we're actually explaining, to them what all of these terms mean if, I'm happy with the small and it looks like it's you know performing, pretty well I can, apply this model to my data and what, this is going to do is look just like we solved the cognitive services earlier, is going to create a new column on top, of my entity and in, this case instead of extracting, out key phrases is actually, going to estimate the amount of unplanned admissions, for the following week for, every single hospital we have inside our dataset so. I want to do is fast-forward a little bit and show you how you can use this data inside. A power bi report, once, you've actually gone ahead and, added it to your data model so, over here as I've mentioned I like maps a lot so we have another map and what, I'm gonna do is again I'm gonna select Washington, as the area I want to focus on and what this does is it actually filters, this table on the right-hand side to show me all of the hospitals, in Washington, and it, gives me my predicted, over capacity so. We've gone ahead and we're able to do actually row level, predictions, using, this machine learning model and I can very quickly see you know there's about three hospitals that, have very high predicted. Over capacity but, the good news is there's a lot of hospitals, that are actually under capacity so maybe now I can start planning how I'm going to reallocate, my patients, more effectively, in the week but, not only does automated, amel give you row level predictions, it also gives you row level explanations. So, if I go ahead and select let's see this hospital, in Federal Way which, has a high predicted, over capacity.
You See we've updated this waterfall chart above it and what, this actually shows me is gives me an explanation of, how the machine learning model derived that prediction so, all the pink, reddish, bars, as you see at the beginning of the graph those, are the ones that are driving that capacity, to be higher and the bars at the end these ones over here are the ones that are actually lowering, my risk and this, is how we you know end up in this predicted capacity, of over 22 over here so, we can see you know factors, like this for example which, is you know my last week's unplanned, admissions that's the top factor which isn't you know that's surprising, if last, week I had of love unplanned admissions, there's you know some sort of time, you know allocation. To this next, week I'm also predicted. To have you know a lot of unplanned admissions, but then we also have some more interesting factors, such as for example my temperature, trending, downwards, so, this is telling me it's not the fact that you have low temperatures, it's, the Delta the change in temperatures, that's actually driving admissions, to go up so, if your temperature is trending downwards that's when you're you, know admissions are going to increase so not only this will give you the prediction, so you can program today sort allocating, your patients to different hospitals it also gives you an explanation, so, you can learn more about what is going on with your data so. Again I highly encourage you guys to try out automated, ml in power bi build, these predictive models and with that I'm going to pass back to gooks. All. Right so recap on this last piece we, saw a little again a I wove, Ihnen we saw the custom vision API in the Welby app we, saw as your functions, we, saw very quickly just a little bit of indigene building an end-to-end solution on top of all of the components we showed, and then, we ended with the automated machine. Learning moving to predictive, inside a power bi I'd also point out in that last piece asatya, talked a little bit this morning about ethics for AI and transparency, transparency. Is one of those things that people are very interested in and so that piece Justyna showed you at the end which is not just the absolute answer but, what are the pieces that drove. The change and the information is, going to become incredibly valuable, over time we're going to be more and more of that so. With that I. Want, to again, come back around to the industry accelerators, tomorrow, morning. Will publish a blog it'll. Talk a little bit more about the one for the two for financial, services and the one for automotive so people want to participate, in that or get information on the private preview there'll. Be a form to fill out we'll, have a little more information on this particular session, and some of the follow ups will, publish it tomorrow given, the amount of noise that's out there today in the blog.
Space For Microsoft, so we'll let that go in the morning outside. Of that look we we hope this is useful it's the first time we've tried to say okay let's take the power bi side and the data side the power app side the, end-to-end pieces, and try and stitch them together because, from here it's, all deeper you know Justina's got a couple sessions there's a power, bi deep dive there are power apps deep dives. There are flow deep dive so there's deep dives in all the different pieces and. I you, know please give us feedback our goal was to try and say hey here's, how to compose it all if, that works great if not give us feedback we'll work on making it better next time I wish, you a great rest of the build and thank you for your time Cheers.
2019-05-08