[Music] welcome everyone and thank you for joining us for today's session on accelerating edge ai solutions i'm steen graham general manager of our edge ai group at intel corporation and joining me is joe coco who leads the partner group program manager for the azure edge devices platforms and services and today we'll be talking about how intel and microsoft are collaborating to accelerate the development of aji solutions that deliver business outcomes at the edge and as we all have seen computing is becoming woven into the fabric of everything we do and as the physical world gets connected to the internet we see an explosion of data an opportunity to transform industries and rich lives this decade more than half of data will be created and processed outside the data center closer to the point of data creation or service delivery the physical world of factories hospitals cities and more this will enable us to transform industries and enrich lives as i talked about one of the compelling challenges though in delivering on this promise is how to take these incredible ai applications that i'm prototyping in my pc i'm training and building in the cloud and deploy them in the physical world we live in today a physical world that has installed existing infrastructure that has multiple os support and you know enterprises are challenged with the cost of deploying new infrastructure that can run ai microsoft and intel are partnering on a set of compelling technologies that allow enterprises to deploy ai at the edge on new and installed infrastructure across operating systems environments giving you the total cost of ownership and a faster time to business outcome joe can you tell us a little bit more about the technologies you're developing to allow developers to prototype in their pc and azure and deploy seamlessly at the edge sure steam it's great to talk to you today and to talk with everybody else today um and share the collaboration with intel that we've been uh having over the last several months so you know as as steve mentioned uh you know ai is really gathering steam on the edge and the challenge is how to actually take advantage of the benefits of ai and so what we've been doing recently is really bringing the best of windows and the best of linux together and because many of the ai workload today are packaged in linux containers and so then the question is how do i take advantage of those assets if i have a bunch of windows iot devices in my factory in my or my retail store and so we've created this solution called eflo or edge for linux on windows and it really brings the best of windows and linux together what eflo does is it provides a lightweight production oriented vm for deploying linux workloads on windows devices your windows 32 apps can run side by side with your linux workloads and interoperate and so imagine the world where you're a developer you have a pc that's your development box running windows you can develop using wsl too wsl is our windows subsystem for linux so right from your windows box using vs code you can develop your linux workloads test them out train the models and then you can deploy that linux workload to those devices in the field onto eflo the lightweight vm for production linux workloads and so we're super excited about that the the beauty of the this is that we integrate with openvino so you can get the hardware acceleration and the performance that you need when you're doing your ml inferencing on the edge and and you can test that on your development box as you're developing so it's really a great end-to-end solution and really brings together the best of both worlds so joe that's extremely compelling so it gives developers the opportunity to develop on their pc in linux based applications deploy an existing infrastructure with windows with those applications that we know and love on windows with all the windows great support services and it connects to azure services so as we deploy that new ai infrastructure we can actually manage that with azure as well i think something that not everybody knows is you know with intel and microsoft we've developed an incredible ecosystem of industry-specific purpose-built hardware platforms everything from ruggedized gateways to medical grade human machine interfaces and with the toolkit we have openvino we can take advantage and enhance the ai performance on those existing hardware platforms and we can also connect it to that great ai developer experience in azure using e-flow as well so this is an incredible value proposition for developers that want to deploy solutions in the physical world today and don't want to wait for new infrastructure you know or want to pair new infrastructure with existing infrastructure one of the things that we've been working on at intel is we've actually used eflo with openvino in both a reference application and one of the key use cases we find in manufacturing is predictive maintenance and we want to be able to predict and understand when manufacturing facilities are going to break down we actually took the eflo technology on windows 10 infrastructure and we actually took a bearing data set and we analyzed using lstm neural networks whether or not those bearings were going to fail and we did that in the cpu with openvino getting the ai performance we need to predict those failures and we got this up and running in just a few days and it was an incredible use case using eflo and the open vino toolkit joe you want to talk a little bit more about the other compelling use cases you're seeing with the community that's using the flow already yeah absolutely um you know you you raised a couple of interesting points dean one is you know the types of devices that are out there that are running windows iot today it's really a huge variety of devices in many different vertical markets so you mentioned you know industrial use cases industrial hmis is a good example uh manufacturing there's we see a lot of defect detection in manufacturing processes whether that's looking for defects with cameras on say widgets that you're manufacturing or even looking at uh say fruit on an assembly line and picking out the bad fruit using computer vision as a couple of examples another one would be in retail settings windows iot is used in many retail settings as well whether it's a local gas station or a large retailer so looking at things like [Music] whether the shelves need to be restocked is a great example where the people are within the store these are all things that you know we're seeing an increasing use of ai in and so being able to bring the open vino technology together with eflo and those linux workloads on those windows iot devices uh is a really powerful opportunity for for our customers um the other thing you mentioned as well steam was the connection to the cloud and one of the interesting things as we dive into the kind of the details of the architecture of eflo is that we have included azure iot edge as part of this lightweight virtual machine and so you get uh the virtual machine for running your linux workloads but it's also integrated with azure iot edge for seamless connectivity to the cloud and that virtual machine actually uses our mariner linux operating system mariner is a linux operating system for microsoft that we use for our first party products and so uh we're using mariner and building upon that um and then the beauty is the interoperability between the 132 apps and those linux workloads that's really a key piece of it and when you think about you know the strength of windows and the strength of linux it's really a powerful combination because windows allows you to build these applications with advanced user interfaces for the you know for the for the end user with natural user input so say with a an ultrasound machine for example you know a touch interface or even a pen interface that's a huge benefit to windows devices and of course the huge win32 app ecosystem but in addition to that you also get 10 years of long-term servicing with windows you get enterprise-grade device management and security and it's just an out-of-the-box solution so it's really a strong value prop but linux brings a number of other value propositions right one is it's flexible and customizable low cost of entry uh and as i mentioned earlier at the outset a lot of the ai workloads are packaged in linux containers and you know some of the modern microservice architecture pieces like kubernetes have also been built on linux and we're big fans of linux and so we wanted to bring both of these things together you can run them on one device at the same time and everything can seamlessly interoperate yeah joe so many things you said there are so compelling i think you know we've actually been able to you know deploy and prototype ai in the cloud and deployed at the edge um and what's the game changer here as you alluded to is to be able to do that across linux and windows with existing and new applications on existing and new infrastructure and why that's such a game changer we think is because that gives enterprises their fastest ability to deploy these new ai applications and their best total cost of ownership in doing so which makes this an a really incredible value proposition and you know joe i think we're busting a few myths with this as well where some people think you know you can't run ai applications on windows and some people think you can't run ai applications on you know intel based host processors but with the advancements we built out through eflo you can certainly run those cloud native ai applications and manage them through azure on eflo and similarly with the openvino toolkit we're unlocking enhanced ai performance on our existing cpus and integrated graphics capability as well and of course if you need additional acceleration you can have our purpose-built vpus for aji applications as well and ultimately why why that's so compelling again is this ecosystem that we've collectively built decades upon decades of purpose-built customer equipment and this is equipment with the right certifications for industries the right i o the right range of power for the physical world obviously there's ubiquitous computing resources and you know huge amounts of power in a data center but when you deploy at the edge power constrained custom io to ingest those devices so there's so many things we can unlock with the power of eflo windows capabilities and then getting the enhancements in the ai performance in openvino so joe what do you think um you know what do you see over the next few years as far as edge ai applications and enterprises deploying this technology yeah i mean we're really seeing a strong demand um for using e flow to bring ai to the edge we we have a uh eflo is in public preview right now we have many uh customers uh building uh solutions on it and again we're seeing a lot of a lot of demand in the industrial sector and the retail sector those are the those are two really popular ones but even even you can imagine the scenarios uh you know when i say retail it's a pretty broad spectrum of scenarios right uh you can imagine a self-ordering kiosk as an example that has cameras in it right and you're say recognizing people how many people are in front of the the kiosk it might be a self-watering kiosk or something or or something in a large big box store to like you know check the price of a product or find where something is in the store and so that really that integration of cameras is really happening across all these different industries um and so you know we're just seeing such a variety it's really it's really inspiring um you know but the other thing is it's not just about as you you alluded to it's not just about cameras or even ai for that matter i mean really you can run any any linux module it can be something that you know some sort of manufacturing application some sort of opc standard that's running there there's all kinds of interesting hardware and devices that you can talk to so it's really the broad set of of code that you can bring together both from windows and from linux and you can do that without having a you know a linux operating system team let's say you have windows devices in a facility you don't have to go buy another box a linux device you know maintain the operating system built often build the operating system yourself microsoft maintains all of that for you so we keep not only windows up to date we keep the vm up to date with the mariner linux operating system inside of it uh and keep it all secure and you can continue to use the existing device management tools and update tools that you already have and that you're familiar with so you know there's really a wide variety of scenarios uh steam yeah i think one couple of things that you talked about that i think are really moving the industry forward in this direction you talk about the the industrial sector and you know with protocols like opc ua we're able to kind of unlock this whole new data set now but we want to be able to process that data at the edge because some of this data has a very short half-life it needs to be analyzed and acted upon you know in near real time and so by putting the ai workloads at the edge co-located with that new opca data set we're able to act on that near real time and solve some of these challenges in the industry like predictive maintenance like defect detection they're really compelling use cases for industrial applications and you talked a little bit about you know cameras as well and i think you know there's industry standards like onvif who are seeing the ability to kind of connect those endpoint data sets from the cameras to this intelligent ai node and that intelligent ai node now with eflo and openvino was just a pc you turn on that pc now and you can actually analyze and act upon that information and you know what's so important about this value proposition is the programmability of it is so much easier with these common industry standards that are feeding in that data set and then we're able to use pc programming methodologies and even make it more seamless with a combination of azure services at the edge e flow and then unlock the performance we need to run ai on that existing infrastructure with openvino it's kind of all coming together and for someone like myself who has been working in in iot for a decade you know we we've always been asking hey what are we going to do with all this data this zettabyte scale of data that we're creating today that far exceeds social media data and one of the challenges was just being able to analyze and act on that data in near real time and now with deep learning in the infrastructure and the tools to deploy deep learning you know we're able to to drive real business outcomes you know in these compelling sectors such as industrial such as retail you know and much more so an incredible incredible set of value propositions um joe anything else you're you're really excited about um delivering here with with eflo any new previews of complementary technologies you're bringing to market yeah that's that's a really good point steam so one of the things that we've brought to eflo is our live video analytics solution so this is a azure service that runs both in the cloud and on the edge for for when you need you know near real-time performance or low latency or you know you want to optimize the data bandwidth and we were easily able to bring that over it's a linux based uh edge solution today and it just runs on the windows device using eflo and so that's a great example of um bringing you know other services from microsoft even you know you can go even further you know sql edge or what you know there's a number of different services that we have you can run whether they run natively on windows or their linux solutions running an eflo doesn't really matter they all work together so you'll see uh many more services from microsoft taking advantage of eflo on windows devices and i think the other point you know you you talked about is the the beauty of the developer story as well right it's been really hard to develop these applications it's and then it you know there's so many new new pieces of technology from training and and then how do you deploy them and how do you scale them and really with the ability to use vs code and the great development tools on your pc with wsl develop there test there and then deploy to production using eflo is just a really great seamless story from developer to deployment and again going through often azure iot edge or iot hub to actually deploy those modules then so that cloud connectivity for uh you know being able to manage those modules from the cloud that deploy them to edge devices update them as needed it's a very powerful a very powerful i think capability that uh we haven't really had that end-to-end solution before yeah i just want to kind of expand on that because you know what the secret is is that you know not only you can prototype in wsl2 on your laptop right but you can also unlock another layer of performance with openvino on that same processor and you know i i was actually launched the neural compute stick too and you know it was a great technology used our purpose built vp used to add acceleration but once we actually install openvino on your your desktop or your laptop system you're actually getting just as good or better performance than that purpose-built ai accelerator so there's really no need for that usb based acceleration but the really cool thing about prototyping in wsl2 turning on openv now and deploying that in the edge this is this incredible ecosystem of hundreds of purpose-built devices because that same code and that same processor is available today in you know hmis and ruggedized gateways in den rail form factors and coms express modules with custom carrier boards it's just a ubiquitous ecosystem that's been built decade after decade after decade which is really compelling i want to go back to something you said earlier because i'm tremendously excited about it and i you know i hope the folks that are watching are as well as you talk about live video analytics and i think you know for those of us that geek out on this we understand that the the windows media pipeline is great and what you're essentially doing is you're you're pairing the windows media pipeline with you know this linux based development environment for ai and then we've also done an open vino integration so you can get the best of your existing hardware as well with that and as we all know about 80 of the ip traffic in the world today's video so what do you want to run analytics on what do you want to run really fast wow you want to run video applications really fast so that's an incredible value proposition joe that you talked about the other thing you hit on which i think is really compelling is you know we talk about this data being created and obviously you know it makes a lot of sense to have your databases in the cloud but some data you want to cache at the edge and you talk about the value proposition of pushing out the sql database technology as well which is probably not surprising that we've also done an open vino integration there so you can run analytics on that sql data and so you know ultimately we're making it really easy for developers to run ai on that data at the edge so they can drive business outcomes and get the benefits of installed applications new applications installed infrastructure and new infrastructure it's really an incredible um value proposition that we're seeing in the market yeah absolutely it's uh these are exciting times indeed that's for sure so in our our session today joe and i have talked about compelling solutions in the retail environment like shopper analytics inventory management or industrial such as defect detection or predictive maintenance for industrial machinery we talked a little bit about the innovations we're seeing in applications like smart cities where we can do traffic flow optimization and even in healthcare we're able to use existing infrastructure to run ai applications in areas like medical images and x-ray equipment and so much more if you're excited about those innovations that joe and i have talked about and you want to learn more about the technology that's available from microsoft from intel and our incredible partner community today you can go to the intelligentedge.com and check out that incredible set of solutions and learn about how you can deploy them today and drive those business outcomes at the edge joe i had such an incredible time today talking about this technology that we're building that allows developers to both prototype in their pc use these incredible cloud services with azure and deploy on existing and new infrastructure and support new and existing applications it's really an incredible story and thank you so much for joining me to talk about it anything you'd like to add to our audience steam well thank you i've really enjoyed the conversation it's always fun chatting with you about you know the the intelligent edge and ai and and the role that you know intel's played and really making that a reality and uh so i'm also excited about uh where this is going i encourage folks to check out the intelligentedge.com there's lots of good resources there and i encourage folks to also give the eflo preview a try it's available for download and you know send feedback and hope you can build some cool applications with it fantastic thanks everybody for joining us today thank you you
2021-03-05