Building Digital Twins Mixed Reality and Metaverse Apps | BRK223
![Building Digital Twins Mixed Reality and Metaverse Apps | BRK223 Building Digital Twins Mixed Reality and Metaverse Apps | BRK223](/pic/building_digital_twins_mixed_reality_and_metaverse_apps_brk223/RVhoUklLSGR1QzA_.jpeg)
[Music] hello and welcome my name is sam george i lead azure iot and i represent a large engineering group across microsoft that's bringing together one of the most exciting new waves of computing that we've seen today we're going to be talking about this exciting convergence of cloud and edge computing of digital twins mixed reality as well as autonomous systems and it's all coming together with other technologies as well to enable breakthrough new capabilities that businesses are already starting to harness it starts with the intelligent cloud and the intelligent edge our mission has been to empower this and if you think about intelligent cloud and intelligent edge it's that the new durable compute paradigm is that solutions span both the cloud and the edge that intelligence runs on the edge and in the cloud and in fact both work in harmony and are self-referential and make each other better the edge is not a means to the cloud it's its own destination with its own place in this new durable pattern and one of the most exciting uses of the edge and the cloud is digital twins so today we're going to start by talking about what is a digital twin and what can you do with it one of the first things you do with a digital twin is model something model a physical environment that can be buildings energy distribution grids warehouses factories anything that's important to you and the most important thing once it's modeled is to bring that digital twin alive and synchronize it when you synchronize it with the physical world you're able to start applying software techniques to that model one of the first things you can do then is monitor this physical environment through the digital counterpart it provides operational insights into exactly what's happening at any given time in these environments the next thing that you can do as part of monitoring that is then track its past once you've tracked its past you're able to then start taking advantage of analytics to find insights into past states into anomalies into anything that's important to you in the physical representation that this digital copy is representing and that enables you to do something incredibly powerful which is predict you can start to predict future states you can start to predict when equipment will need servicing before it does you can start to predict occupancy needs before they happen anything that you can apply software to you can now predict its future states and the killer app for all of this is simulation simulation enables you to take copies of the digital twin that's effectively the operational visibility of that physical environment make copies of it and then run simulations on it and those simulations can then goal seek to find optimizations that are too complex to find by monitoring the physical environment alone you can do all that with this software copy of the digital twin and then once the simulation is complete then you can apply the insights that you gained on the copy of that digital twin back to the real digital twin and affect inputs into the physical environment this enables you then to start providing autonomous control you can start automating routine and mundane tasks make them better with systems that learn and improve over time this makes the physical environment better and better and more optimal and one of the most powerful things then you can do with this is interact with it you can interact with this digital replica overlaid onto the physical environment you can get rich metadata and insights about anything that you're doing in the physical world from this digital copy you can also in interact in pure virtual space over distances and with any colleagues and experts anywhere in the world now there's many names for this and it's far beyond simply digital twins it's beyond autonomous systems and it's beyond mixed reality as more and more companies start to take advantage of these capabilities and interact with each other and partner in these virtual worlds we're really creating the metaverse we're creating the intersection of all of these digital realities that are then optimizing the physical reality let's take a look at this in action imagine our world today people vehicles businesses buildings hospitals factories entire cities and the billions of devices that connect us to this world and to each other this is the physical world but what if we can replicate that physical world into its digital twin a digital twin that is bound to the physical world in real time that we can experience in mixed reality that we can collaborate with that we can run simulations on and find what is important to us that we can apply ai to learn predict and act to save time and money to reduce carbon to preserve natural resources to improve safety to bring us all closer this is already happening today across the world organizations are taking advantage of this new wave of innovation that trends to an interconnectedness that enables the metaverse everything modeled in the metaverse mirrors the status of its physical twin including the interactions and relations between everything else by applying the power of the cloud we eliminate boundaries of what this can do the potential is limitless we can now track and analyze data from connected environments to identify patterns trends and anomalies we can now simulate any possibility evaluate outcomes determine the impact of any change or condition we can now harness ai to perceive the physical world to improve and automate tasks to give superpowers to your frontline workers we can now give everyone in your organization the power to build apps and workflows to collaborate with each other in this virtual space to share and receive expertise at the right time with the right context in front of you to move through the physical world and get relevant information about its digital counterpart when and where you need using mixed reality the microsoft cloud brings these capabilities together with the limitless computing power of the cloud and intelligent devices at the edge and creates this framework for building immersive and impactful digital twin mixed reality and metaverse [Music] solutions now as you just saw all of those rich capabilities that we introduced in the beginning are layered on top of each other they interact and if you look at the different services that we at microsoft offer we provide everything from iot all the way up through mixed reality with all of these products that you see here we're the only company in the world that brings this entire stack together with a rich partner ecosystem that makes this all possible with companies across the globe anheuser-busch and bev is a great example of this with more than 200 breweries and 150 000 employees across the globe anheuser-busch is the largest brewer in the world they're dedicated to the highest standards of quality and consistency they're experts at what they do in process manufacturing and they're taking advantage of this entire stack in order to dramatically improve things let's see this in action with more than 200 breweries and 160 000 employees across the globe anheuser-busch invev is the largest brewer in the world with an unparalleled portfolio of brands and a global presence with operations in 50 markets av inbev is an expert in their field and their forward-leaning and innovative spirit is leading to new breakthroughs in brewing through their digital factory and supply chain of the future initiatives in collaboration with microsoft av invev is focused on transforming both breweries and their global operations by empowering their frontline colleagues with digital solutions that enable world-class manufacturing mobility data automation and business insights it begins with the brewery coming to life with ai azure digital twins and the microsoft cloud the brewmaster is responsible for making the best quality beer she has unprecedented visibility into the brewing process so she can predict and monitor the complex chemical and biological fermentation parameters required to produce the highest quality product frontline operations can remotely monitor critical quality and traceability data from the manufacturing execution systems using the mobile brewery technology built using azure with built-in energy and utility management solutions the digital brewery supports av inbev's industry-leading sustainable development goals after the perfect batch is ready to be bottled and shipped av inbev extend their digital twin solution to support the packaging line operations to ensure it's done right these can plans are adopting azure ai and microsoft project bonsai's deep reinforcement learning to deliver line balancing optimization that detects and automatically compensates for bottlenecks in the complex can manufacturing operations at ab inbev their 100 uptime mindset is enabled through predictive maintenance and global support a frontline operator can efficiently address problems through remote collaboration with a maintenance technician ensuring no unplanned downtime of packaging equipment the perfect beer is packaged and ready the digital technologies of the av inbev supply chain of the future minimizes carbon footprint while making sure the right beer gets to the right customer at the right time this beer's journey is complete and av invev and microsoft are taking bold steps into the future [Music] it's incredible to see abn bev taking advantage of this new stack taking advantage of azure iot azure digital twins azure ai project bonsai what abn bev has done is a first example in the brewery that is then gives them the confidence then to extend that to their distribution as well so let's take a look at the stack in action starting with the physical world as we saw the first step up is azure iot and azure iot enables abn bev and any customer to connect into their physical environments it enables them to also run cloud intelligence directly on-premises disconnected using azure iot edge it enables them to sense anything that's happening in the physical environment and one of the new breakthroughs that we've just announced recently is azure percept azure percept is a brand new set of developer kits that then our third-party hardware partners are customizing for different environments it enables you to take advantage of ai on the edge with easy to develop tools and easy to manage capabilities as well for audio and video and that audio and video enables you to sense things that weren't possible before you can catch product defects as they're happening in real time you can ensure worker safety by making sure that workers are wearing protective equipment or they're not in dangerous zones and all of this is being powered by azure percept and of course cyber physical security is absolutely paramount in these types of environments so we have azure defender for iot which enables you to connect to not only new devices like azure percept but existing devices that you've been using for decades it enables you to quickly monitor those to scan them and to check the security posture of those in real time and if there's any problem you get automatic alerts that automatically gets surfaced in azure sentinel moving from azure iot to the next layer up we find azure digital twins and as you heard with ab and bev azure digital twins is being used to model that physical environment now azure digital twins is then fed by azure iot in order to make the model come alive to keep it live and up to date with the live operational state of the breweries azure digital twins has an open source digital twin definition language that enables you to model anything including the complex relationships that exist in these models and we've also open sourced ontologies which are industry specific versions of digital twin definition language tailored for those different industries on top of digital twins we find azure maps azure maps provides the location of things azure maps has a breakthrough new capability of indoor private maps which enables you for example in these industrial environments to create indoor maps that help you quickly find any asset and any co-worker with location and routing services and again that's all entirely private to your enterprise now as data flows through the system it flows into azure synapse analytics and azure synapse analytics is our breakthrough new service that enables you to have a secure comprehensive set of data services that all work together to provide rich analytics to track the time series history of things in these environments and then also to apply analytics to find insights to them now that all that data resides in azure synapse analytics we can take advantage of azure ai as well as microsoft project bonsai azure ai provides breakthrough insights into the data that was collected and project bonsai enables a low code approach to both machine teaching as well as creating intelligent autonomous systems those autonomous systems learn over time and get better and better and of course because they're connected now to the digital twin they can influence and automate the physical world moving up we have the microsoft power platform power platform enables anyone in an enterprise to quickly build applications power bi dashboards as well as take advantage of power automate to automate anything power platform enables you to take advantage of everything below that we've talked about already from the digital twin to all of the data that's being collected about it and really take advantage of that for anyone in your enterprise and then moving up we have microsoft mesh and hololens this enables anyone from across the planet to collaborate together in these mixed reality environments it also enables the digital twin to be overlaid onto the physical process so workers with hololens or tablets and phones can see the digital information about the physical environment directly in front of them this then completes the stack that started with the physical world goes up through azure iot and ends in mixed reality and importantly there's solutions of course that are built on top of this you saw one of those from ab and bev but there's others there's ones like this that microsoft develops we have isvs that are developing these as well as well as partners from across the world some of the solutions that we as microsoft are developing are things like connected store which creates a digital twin of retail environments and gives you insights into how it's being used and how to optimize it there's connected field service which automatically calls for a repair technician when a machine needs help and helps enterprises coordinate that there's guides which give you visual wizards overlaid on top of physical reality about which steps to take to complete something and if you're still having trouble remoteassist enables you to call an expert from anywhere on the planet they can see through your eyes they can annotate the world in front of you this technology stack is transforming every industry and as you look at every industry there's a rich set of partners from across the globe microsoft is a platform and partnering company and we believe that this technology will lift all boats and so we've partnered with all of the companies that you see here and many more to start delivering these solutions to customers from across the globe parasaclay is a french equivalent of silicon valley dramatic growth in the paris suburbs is creating a challenge accommodating energy demands while still meeting their environmental and sustainability goals over the next 20 years their goal is to reduce greenhouse gas emissions by 75 percent as well as reduce the price of energy by more than 45 percent by modeling 27 city areas and its infrastructure using azure digital twins they're able to track changes and ask what if questions they're taking advantage of cosmo tech simulation services to ask those what-if questions what if we use geothermal as an energy source where can it be placed and what is the impact of the placement on distribution what new energy sources are required for business parks or higher density suburbs and where should they be placed what is the impact of higher efficiency buildings on energy usage in the future with the solution in place they give confidence to their city investors that their investments will balance growth and sustainability goals avoiding being out of compliance with energy regulations swedish grocer ica has 1300 stores across sweden their goal is to differentiate with a great customer experience from having shelves that are always stocked and a spotless clean store ica turned to avanade ombori and microsoft to build a digital twin of their store that enabled them to capture contextualize and surface data that allows employees to improve their store experience by ensuring products are always stocked and help customers faster with the foundations built using azure iot and azure iot edge ica used azure cognitive services custom vision video analytics and partner modules from onboary and avanade to give them real-time visibility into the store this visibility is projected into azure digital twins of the store that models a real-time representation of product and machinery including azure maps indoor maps azure digital twin connects this data to systems like the employee app presenting any changes taking place in the physical space of the store product quality is maintained by monitoring the refrigerator and freezer sections and in-store services and queues are monitored so that customers can quickly check out when they're ready total is a major energy player which produces and markets fuels natural gas and low carbon electricity are committed to better energy that is safer more affordable cleaner and accessible to as many people as possible one of the many things total does is blend fuel with previous solutions blend quality can typically only be determined after the blending process has been completed this is caused by uncertain variations due to complex fluid dynamics from different input flows equipment configurations and recipes total worked with our partner tcs to create a digital twin of the blending equipment and processes and then applied ansys complex dynamic fluid simulations to help them assess if a recipe was possible with the equipment first minimizing waste and reducing costs brookfield properties is one of the largest property owners in the world one of the challenges they face is how to monitor service and maintain their fleet of worldwide buildings which are all unique starting with our iconic one manhattan west property they worked with our leading digital twins isv willow to create a digital twin of this 2 million square foot 67-story skyscraper in new york city they used azure iot and azure ai to remotely monitor and predict maintain equipment they also drive decision making and optimize systems like energy ventilation and temperature power bi dashboards drive facility managers to give them live operational insights at the tip of their fingers and azure digital twins then transforms the building into an intelligent network of digitally connected spaces combining systems across retail office hospitality and residential environment now all of this is happening now and you can get started now as well now with all of these examples that we just went through one of the things that i want to stress is that you can progressively adopt this not every layer is required to build these solutions but by starting with these layers it helps you expand into them as your business needs improve and as new scenarios come online so it's time to talk about how to get started with this today the first thing that i want to cover is azure iot and digital twins we have a lot of amazing content that you can see here that shows you how to get started how to take advantage of azure iot how to model things with digital twins and how to start connecting the two in order to take advantage of some of the scenarios that you saw today azure ai and microsoft project bonsai you can see how to get started here and take advantage of both of those to not only find insights from your data but to create autonomous systems that you can apply to these digital worlds microsoft power apps again are low code no code platform and here's how you get started using microsoft power apps and finally microsoft hololens and microsoft mesh you can learn more here you can even see at the bottom the azure digital twins and mixed reality microsoft learn module that shows how to use these two together so as all of this comes together we have a bold ambition to make this easier and easier to use to make these layers more and more seamlessly connected so that any business anywhere on the planet can reach their full potential taking advantage of this thank you so much you
2021-05-30 16:40