New Breakpoint: Episode 5 - Aussie Architecture

[Music] bye [Music] [Music] dude hey everyone and welcome to this week's episode of new breakpoint my name is simon wait and i'll be your host for today's show we've got a great show lined up for you today uh we'll get to our main guest in a minute but if you'd like to join in make sure that you log into your youtube account and feel free to join in the live chat and answer or ask any questions that you've got either for myself or val who'll be joining us in a minute uh there if you are catching up on the show later on you can always tweet us at our microsoft au dev twitter handle and in fact during the show as well and those questions will be brought into the studio so that either myself or val will be able to answer them for you as always please make sure that you take an opportunity to review our code of conduct to make sure that when you are asking questions or collaborating with members of our broader community that you're doing so in a respectful and appropriate fashion so what are we going to do this week so this week we're joined by val cevenko who's one of my colleagues in the azure business group here in australia and al's val's focus is on iot but she does a lot of other things she's a passionate developer she's really hyper interested in ai has done some really interesting work uh in her time at microsoft australia in this space as well today what she's going to talk about is ai for developers last week with damien we looked at ml ops and we showed how you can bring your devops expertise into the world of machine learning this week we're going to look at how as a developer you can get started in the world of ai which can seem a little bit daunting when you first take a look at it so let me bring val onto the show just do this two seconds here hey val how are you hi simon hi everyone how's it going oh it's fantastic i'm really pleased to have you here and this is an amazing topic i think we have some really great opportunities today to explore simple and quick ways for people to get started with ai and having seen some of the demos you've got as well i think it'll be interesting for people to uh suddenly see the power they have at their fingertips without necessarily having to be machine learning or artificial intelligence experts so without further ado i'm going to hand over the show to yourself foul to to do your thing uh let me get your slides up here so that um you can get started all right there's your stuff and i will see uh you all later so val over to you awesome thank you simon well hi everyone um today it's i'm gonna talk to quite a you know an area of passion for me personally um is the ai for developers and you know maybe a little bit about me before we get started um as simon mentioned my name is valeria cevinko born and raised in russia hence a tricky double eye in the name came to australia to study electrical and biomedical engineering a few years ago it's almost coming out to 10 years and now currently leading the iit business for australia based in brisbane i've done a lot of work sort of with you know being an electronics engineer in the devices space done a lot of soldering is part of my degree and came to microsoft to sort of combine the cloud and devices uh and that's how i was introduced to the world of ai because when you connect you know small little chips to the cloud there's not really value in just looking at the data value comes in when you analyze it and ai came about to be a very interesting technology that enables you to do that um i think ai is quite a buzzword and i'm sure everyone who is now watching in some in some shape or form have heard about it before um and i was introduced to it through many projects that i worked with customers uh prior to this role i was in the fast track for azure team working directly with customers and building the projects and today i'm just going to share a few of the demos some of the projects that i have built myself personally and kind of help you maybe get started but my main goal if we take all that aside is just to get you excited is to get you inspired to go and then play with some of the services um because in the past couple of years um microsoft azure i would have released a lot of really cool cognitive services to help you get started and i'm going to show you some examples but if you're an organization and you know you're very new to ai and you've heard of it you know what it stands for like what is it and you know being a developer the first thing that i do is i i google radical and stack overflow and i try to figure out what is this thing that let's say my you know my job wants me to do and i took this snippet from the wikipedia because it's you know doing a lot of research in my electrical engineering degree wikipedia was my go-to sort of website to understand what is that technology that i'm trying to work with and if you read through it i mean it's it's pretty complex right and how can you take this description into something that you as a developer can go and build right and i don't want to go and read it you can look it up on the wikipedia but it's a little bit convoluted it's it's just hard to get your head around um and if you don't have really like a data science degree um or a machine learning degree it's it's it's quite a um it's quite a steep learning curve so that's why today i'm going to share my own personal experience right coming as a developer working with customers um across multiple projects and i guess the power of my role is i'm not just siloed with the you know doing iot um usually when i work with customers and projects it's all end-to-end from your devices to your analytics to some of the insights so today i'm just gonna you know share my own personal um journey we're gonna look through a few two use cases the first one is kind of the speech analytics and a customer story that i worked with my really good colleague tim laden um and then we're going to go through how you can use ai on your iot data if you do have iot data as part of your organization how to get started i think we should have 15 minutes and simon please let me know if i'm going over time i'm pretty passionate about this topic so i can be talking um for quite some time we do have a few demos and i had to pre-record a few snippets of them just because the streamyard sort of software makes a little bit tricky to share the audio so we'll share some of the pre-recorded snippets but everything will be live and i have devices now streaming data and let's hope that my hotspot will not turn off halfway through the demo alright so if we get started not sure why this is not showing but let's not pay too much attention to it um if you want to start in sort of in azure and doing ai machine learning kind of at the top of this right look at the top you see your they mean specific pre-trained models so something that microsoft spend a lot of resources time and data to pre-build the model they can then reuse and if you're familiar with it they're just called just called um they're called cognitive services and it's basically an api right and with developers we like apis um it's quite easy to integrate if you do have different sdks and things like that to help you use those services then if we go down um all of the tools right so i did a bit of work with one of our customers using sort of jupyter notebooks in the visual studio code you know i spend most of my time in vs code when i do any development work so we do have really nice integration with all the rest of the tools popular frameworks um you know tensorflow if you ever worked with custom vision is an example right custom vision is built on the concept of a cognitive service but it kind of helps you customize it to your particular scenario so you can upload a few images um let's say of if you're training you know to recognize different types of bolts as an example you can upload the data um to the custom vision portal and train it and then you can actually export that as a tensorflow model so if you want to see how that works you can do that as well we also have different services like azure data breaks um cnaps azure machine learning vms like we have one which is called azure data science vm that has gpus so you can if you want to go down that route um sort of use and build your own model using the infrastructure that we have so it's we have cpus gpus and fpgas um as well so a lot of really cool resources to help you get started but today let's you know let's let's stay at the top and if you want to go down we do have resources available on the microsoft learn um to to to help you get started so if we just stay at the top and focus on our you know apis so they're called microsoft cognitive services um and i'm sure you've seen a lot of demos you know microsoft from build um and actually when i just started a few years ago i didn't really find them useful this is i'm just sharing my own personal opinion i didn't know how to use them i didn't know how to apply them they they seemed like they didn't work as well up until the point i had actual um projects that i had to do with customers that i saw their you know their capability and how amazing and how they have simplified uh the project that we've been trying to create and just accelerated it um so i i'll i should sort of have an example of the particular project that i worked on so when we look at the cognitive services they're basically ai you know pre-trained ai that expresses a rest api um when you you know you post you can post an image to it and returns you the json is the output and you can visualize it and then you know maybe share that information um on the web page of some sort um and there's a few pillars right that we have cognitive services across things like vision speech language web search and decision so if i go to the next slide why cognitive services we actually have a lot of sdks for different languages python knowledge c sharp um so depending on you know what you're comfortable with um we do have things that are exist really out of the box and samples for you to get started um and it's it's quite easy because developers quite familiar how to use the rest api so um it's quite a no-brainer to some extent um and if we look at the broader capabilities there is quite a therapist if you if you have some time today maybe after the session if you're watching um you know online after after we've gone live have a look at what we have for let's say vision right we have a motion recognition uh we have text and handwriting recognition so if you have receipts and you want to even for your own personal project and you want to automate you know getting that information from the receipt which is either handwritten or typed into some type of a knowledge base you can build a quick demo as well speech is definitely an area of my interest um because we have services like speech to text as well as text to speech where you can train the model on your own voice type in the text and it will basically synthesize that um you know kind of put that turn the text into the speech using your own um your own voice we have a few things for language knowledge and search so if you want you can take a screenshot um but if you just type in your you know in your web browser cognitive services i will be able to see a variety of different um cognitive services for you to get started with on the platform but now i want to kind of make it real and share with you an example that um i work myself and we've actually had a customer story posted on the microsoft news um which is pretty exciting um so it's the team that i work with the fast track for azure if you want to look more at this story um this is the website maybe take a picture if you would like to and the solution itself um is posted on github it's my colleague's personal github and we sort of with time keep reiterating over the solution we had the solution deployed across a few of our customers and they sort of extended that to fit their own requirements but at a high level what did we work on um so we worked with the um hopkins center with the team at hopkins center at griffith university if you're based in brisbane and queensland it's one of the largest unis here and we work with professor elizabeth kendall who is the executive director for the center and she leads a large team of researchers investigating rehabilitation and disability service um and it's important work that brings together you know community that that stands physiology neurologist speech pathology um and etc and what they usually do is to gather the data to do research they usually do interviews right so they call them up they either do it via the phone or do it in person um and as part of the research they actually need to like record the call and then transcribe it for sort of research and analytics purposes uh and what the team at griffith has told this is sometimes it takes months for transcriptions to arrive these different third-party services but if you think the audio could have pii data rights you need to make sure it you know it uses different compliance um and privacy um regulations around how you know the data is stored and for the griffith team it's really you know was really quite challenging to manage that uh and definitely it's the cost and the time that it takes for them to receive the transcription um and because they maybe use different agencies to transcribe uh it's not very the data is not is not the same right so how do you bring that consistency across all of your data sets for your research and analysis so we worked with the team at fasttrack and nick cross verso who is the acting director for e-research serves a group with uni and we've built an accelerator more or less that the grade 15 that they used to transcribe the recordings so if i now stop this [Music] and you can still see my screen i will hide that away so if you want to read a little bit more this is the story um it's quite amazing and i'm super happy that the griffith team has taken the accelerator i done some amazing things on top of it the accelerator is posted on github as i mentioned using the cognitive services um on the team laden so it's my colleague github and it's called speech to text if you ever want to play with it just make sure you use transcription viewer three as i said it's still work in progress um and we keep updating it as you can see team was helping me to get it working for this for this demo a few days ago um so make sure you choose transcription 3 and if you scroll down i've tried to write a high level overview of what it does if you want to deploy it it uses arm templates and custom deployment to deploy this solution but this is sort of the high level architecture i'm going to mostly focus at the top part because i think i need to to fix the bottom part has been a few months since we've touched the bottom part and the apis have a little bit changed so i don't think so it works um but if we focus on the top part dead yeas we use the speech-to-text cognitive service uh and batch transcription team wrote an angular app where he's hosting sort of the web the web page of it for the researchers to come in login uh upload the audio and see the transcription uh we sort of just use azure functions to you know send the requests um and receive the transcriptions once it is done so if you do come in and then you deploy it um so it looks like this um so if you just come in and you have never done this before i'll just open a new time got it you will just need to specify the where you're deploying it so just the region whether it's in australia or us um it will take a bit some time there you go all right but while that is taking time um once you do spin it up it creates a few services right so this is your app azure function blob storage for your data and just application insights let's see if it has provisioned custom deployment there you go um so you just need to create a resource group specify the region and the speech location so it's just a drop down and review create it will take a few minutes um so just keep in mind we'll take a bit of time once it's provisioned you'll have this resource group you can go down to the app service and click on the and click on the url maybe my laptop is not coping with um sharing and demoing at the same time cool while that is loading let's go to the one that i have already created because that would be a little bit quicker so this is the transcription viewer um this is the work that team has done um i selected color purple that's one of my favorite colors what you can do is you can come in you know select the recording or you can upload one so just upload new choose file uh and if you think you know how long it will cost uh initially uh just um we did it over a christmas holiday last year um so me and tim just decided you know why not to just create a quick quick sort of accelerator to um to help the customer so it took us a few days but sort of pretty fine making it in a way that the customer would be able to reuse it did take some time but it was pretty quickly with the sort of the power of cognitive services cool let's now choose recording tool you can put the type title so i can just say um demo live description [Music] live then you can click submit um as you can see there is a few transcription that i have already posted if you go before let's say obviously you can do it one by one like i'm doing it here uh by team did an amazing thing where you can just select which containers you can as a researcher um just upload all the files and batch transcription will transcribe all of them um sort of yourself so you don't need to do it one by one so you can see this one is running um it does take a few a bit of time uh but let's go to the one not two months a few minutes um so let's go to the one that has been already transcribed and then i'll show you this one because i think that's the one that we have pre-recorded um as you can see this is you can listen to the recording here at the bottom i won't show it just yet um but you can look at the offset at the text because it returns it adjacent with a timestamp you can easily layer that out quite quite nicely um so let's say if you're working for a um you know for a company where they do call center uh well they have a call center so having something like this you know for compliance purposes um or for how well let's say the agent is sort of working you can do quite a quite a similar um project um so what else can you do here you can change the playback speed um you can show confidence right so i'll just show confidence here um confidence is quite useful um because obviously the transcription has become really good but there are still you know maybe different accents uh they might affect the the transcription a bit um so having confidence in knowing okay this is 91 you know i'm quite quite confident that that's what the person has said uh this is 73 so you might not be as confident and you can take that you know with it with a grain of salt when you're doing your research um simon any chance i can get you to play the recording so folks that are watching live can sort of see how it works in yeah hi my system is network solutions yeah hi my system is down and i need to speak with a technician oh okay um let me gather some information um and see if we can as you can see that was uh myself and a good colleague um who is irish doing this demo so we had to pre-record it um listening to myself was always the worst thing uh but as you can see you're able to click on the transcription that will take it to a particular um time in the recording um which is quite a pain handy feature for the analysis of it um this is kind of the first demo i didn't want to go into the nitty gritties of the car just the amount full of time but if you want to spin it up it's it's in this resource group um this is what it will create and if you want to run it you know past your team like and say hey you know let's see if maybe we can do something like this you already have a quick accelerator to get started uh one more thing that i wanted to show was this one and i think i'm being a little bit over the time but the other really cool thing that we have is ability to do text to speech right so you can retrain the model on you know on audio type in some text and that will give you that that audio in that particular um voice you have portrayed in the model um the cool thing is you can either even do intonation you can make the voice sort of sound more happy or more sad um so if you want to play around with it if you go to the audio oops audio content creation um and you want to play around with these styles you you can so for example you can be cheerful um if you go down narration professional and you can see how that changes the same text based on different um speaking style would you have one demo maybe someone if you can um really quickly input the the prerequisite all four of my grandparents are born outside of brazil they're all immigrants into brazil as i look back at this year i am very proud of the progress we have made as a team we have made incredible progress day to day across our people customers teams technologies businesses and products as you can see right now there was the recorded and the synthesized right so the recorded was the actual person talking and the synthesized was the ai that has generated um the the audio and as you can see it's it's really hard to separate that and if you think of various applications um you know maybe in the um could be if you're building uh like a chat board as an example um you can pre-train the audio uh for a very nice sounding voice and play that um as well maybe you can you know as a first step play around with your own audio and create a replica of your own voice um so it's really really cool technology um then i will just do the other demo uh which is the you know contoso uh let's let's think of a simple company called contoso archiving and specializing in the shipping artwork right and they have been receiving a lot of concerns from their customers that the parcel comes damaged or something that has arrived has been spoiled or someone has tempered with the package and they want to track that information and they want to be alerted if something happens to the parcel right so you can read a little bit more what is it sort of saying here uh but in general we're trying to run maybe some ai or analytics over the iit data right and in general if you want to approach the problem maybe you need to build um like a data science or machine learning model so because i'm a developer i don't know how to do this um i had a look and saw that we have a service which is called azure stream analytics and azure stream analytics is a service that helps you analyze live streaming data and as part of that services has the capability to do anomaly detection so we have pre-built a model that does anomaly detection integrated that into stream analytics and you can just use a simple sql query like language language to use that model so at a high level we have a device streaming some data will have temperature being streamed into stream analytics and then we want to run stream analytics query to identify let's say if the temperature goes up or goes down within a time frame i do have a device i'll just show it hopefully it will i think it's like this there you go sorry if i'm giving you a little bit of a headache but this is an mx tube which is a very simple arduino board with a few sensors on it but it's just streaming some data into azure and then i have a few services in azure it's called iot hub that catches the data and then pushes that down to stream analytics so let me hide that away so i just have a device running and then if we go into into the into this sort of um dashboard um if you want to have a look more on the sort of the stream analytics part we have a normal detection azure stream analytics which sort of describes you can watch the videos quite long uh but it sort of explains how it works and it uses that um sort of framework called spike in deep to be able to see when the spike occurs uh or when the deep occurs and based on that it will alert what i did is that i just copied and pasted this sql um query um and i have a stream analytics job or just just just before i go if you want to recreate this end-to-end demo um in this case i'm using mxcheap but that could be any other device you can go to this github repo and it helps you sort of build step by step the whole end-to-end demo it's actually pretty cool so this is what i've done and the idea is this is your query and what it does is that it's looking at all of the data coming from all of my devices right and it's looking at temperature reading because i want to have a look at the temperature in real time uh and it's looking at the temperature and all you need to do is you need to use this anomaly detection underscore spike and deep function over the temperature and here i list it as what 112 1200 seconds sorry 1200 and 1200 seconds of the data and then we want to output that information into my endpoint right so i'm going to look at the data see if the anomaly has occurred within a certain time frame if it has occurred i'm going to output the anomaly score so how confident i am so as you can see across cognitive services conference is quite a big big part of it because you want to know how confident the model is and the actual anomaly has it occurred or not i do have a time series explorer right here which is the service where i output the data if i now just zoom out a little bit in the last few seconds there you go maybe i can press on my temperature reading so now hopefully the temperature will go up so this is i'm just gonna show what what i'm doing but more or less i'm just holding the sensor you can see i'm just kind of holding the temperature sensor on the on the device let's just refresh just takes a bit of time it's got to flow through all those pipes to get to where it's going to yep to get or maybe my finger is not hot enough for the sensor but as you can see before uh while it was running um it detected that the temperature was going constantly down which is definitely not normally so if you do have you know a box with a sensor and you want the temperature to be consistent you want to be uploaded every time it's going down and as part of the stream analytics you can output the data into let's say an event hub and then you could have a logic app that can send an sms um anomaly detection is one of those cognitive services apis as well that you can you can bolt your stuff into awesome awesome demos val um a lot a lot for people to take in there um i think for me the cognitive services is a really great place to get started when you start building intelligence into your apps and the great thing is you don't necessarily even need to be running your source application inside of azure when you start that journey you can start by baking those rest apis into your app today and as you migrate into azure you can start to expand your capabilities outside of that as well hang around while we wrap up the show by all means as i said please feel free to reach out via twitter or post comments on youtube vows details are on the show log let me just bring that up on screen here so if you want to find out more about the source code for the demo that val showed the article to do with the griffith university work that val and tim contributed to along with that super cool speech speech website where you can go and have a bit of a play with the the neural text to speech it's a bit freaky seeing alex kipman kitman speak like that and then being able to convert that into text to speech but some really amazing possibilities there to generate for content creators i think as much as anything else so without further ado i'm going to wrap up today's episode thank you very much for joining us today val a reminder for those of you watching the show next week on thursday we have our community show it's going to be an amazing discussion on ways that we can tangibly support women in tech with a great group of our community team and also some community members from outside of outside of the contributors to new breakpoint um val do we want to give everyone a wave to wrap up the episode so just a big wave at the camera thanks everyone thank you thanks a lot um and we'll see everyone for the next episode see you all again [Music] soon you

2021-03-23 04:31

Other news