[Music] welcome to our open source session at build we have some exciting news and demos to share with you today i'll share with you our open source journey at microsoft and how we built azure as an open cloud so that you can build and run apps with your favorite languages platforms integrations and databases while still optimizing it costs i'll then turn it over to our other fabulous speakers to cover the flexibility of deploying cloud native applications and how to unlock innovation faster using ai and data at microsoft we're committed to open source software and we're working on a culture that embodies open source values our developers are using open source software to get their jobs done more effectively virtually every team at microsoft is involved with using or contributing to open source software you can see this in the numbers and yes these numbers are real um so the first number is 60 000. that's 60 000 unique open source software components that we use at microsoft and because we use those in multiple different places different teams might use them or we might ship them in multiple products we have 9.5 million different instances of those components in use that we're tracking open source is about creating an open mindset collaborating and contributing to software to create added value and to build on the work of others we collaborate with communities and partners and our customers to bring the best of technologies to helping organizations and individuals to achieve more microsoft's contributions to open technologies span across multiple areas a few examples are highlighted here linux has played an outsized unique role in the development of open source culture at microsoft our upstream contributions to the linux kernel span over a decade we contribute to the kubernetes ecosystem with projects like dapper or open service mesh to make kubernetes more enterprise friendly and we continue to make upstream contributions to a variety of programming languages web frameworks and technologies like net node.js python php and many more net is ranked as the number one most loved framework and has more than five million active developers on github every month our mission is to enable organizations and individuals to achieve more and open source is a very real part of the solution by creating open solutions we enable our customers to build on the work of others to innovate more to achieve more based on the mckinsey research open source software adoption is the biggest differentiator for top performing organizations both in terms of innovation and for developer retention we're actively seeing the benefit of adopting an open source mindset and so are our customers our customers are driving their digital transformation journey using open source and deriving significant benefits from investing in open source software technology i'd like to highlight two examples of customers that benefited from adopting open source technologies the first one i want to tell you about is bosh fosh was looking for a cloud option to run their core business logic with zero changes on top of a new infrastructure aks offered a simplified managed kubernetes service in the cloud without needing to manage infrastructure the developers were able to use the open source tools they were familiar with and love and run the solution virtually without any code changes the other one i'd like to tell you about is shell shell was looking for an intelligent cloud with an intelligent edge that could use machine vision to automatically identify and respond to safety hazards in near real time shell was looking for a cloud platform with the fastest response time with the tools necessary to get the job done shell deployed several open source technologies azure data bricks with the latest version of apache spark for analytics gpu processing in the cloud an end-to-end devops pipeline apache kafka for streaming and an open source computer vision library for their deep learning machine vision applications opencv azure provided the right tools for the job that work together in seamless fashion no single person can make the progress that we can all make together using software in the open open source has fundamentally changed software development we're excited to show you how we're making azure the best cloud for developers azure can help you innovate today tomorrow and in the future we're constantly providing capabilities that can help meet your needs to work with open source on azure you can run multiple apps and increase workload performance and security all while optimizing your it costs over half of all the cores our customers run on azure are running linux and we are heavily invested in enhancing the performance reliability security and resiliency you can build on your terms using your favorite tools favorite languages and your favorite third-party integration and lastly we want you to focus on building your application and not your infrastructure azure provides you with enterprise grade managed open source software databases analytics and ml services to bring ai to your application azure and basis three core principles collaboration security and flexibility azure is collaborative our goal is to empower developers to innovate and help them achieve developer velocity through providing best-in-class tools and environments built for collaboration azure is secure nothing is more important to us than trust particularly when you're talking about the cloud we spend a billion dollars a year on cyber security with 3 500 cyber security engineers to protect your data and business assets and lastly azure is flexible we know developers and we know that they have applications in a variety of locations on premises in the cloud or the intelligent edge azure gives you the portability to move your apps without changing a line of code the open source ecosystem includes solutions for many vendors and communities we strive to make azure a great platform for our partners to jointly build and support their solutions as managed services for our customers today we create a seamless experience across operations identity security and billing to simplify life for our users let's hear directly from one of our partners on their experience partnering with azure and what we built together we are pleased to have elastic join us today osman the stage is yours thanks for tuning in we're happy to share that microsoft has been a great partner working with elastic to help us provide the elastic stack natively in the azure console it's extremely easy to get started simply search for elasticsearch in the azure search bar then create a resource just like you would any other azure service from here type in a name pick a region where you want elastic deployed to you can change a few other settings like enable automatic logs collection so data is sent directly into your cluster and then hit create once your cluster has spun up you'll be able to view the details directly in azure from here you can easily launch elastic to view azure log data that you just ingested you can also make changes directly into the elastic cloud console as well here you can make more advanced changes like enable auto scaling or just resize your cluster if you need to back in azure you can click into monitored resources to see a list of all the services that are currently being monitored and under configuration you can see some more advanced level settings that you can change so if you'd like to resize your cluster enable monitoring or even enable some traffic filters like private link or setup and ipv you can do that here as well so that's a quick demo of the elasticon azure native integration for more information please check out azure.com
elastic or be sure to check out the partner showcase thank you thank you osman we're very excited to announce the new elastico azure native integration to allow customers using elastic services on azure to access integrated billing full technical support and azure portal integration i'm now going to turn it over to donovan brown to walk you through your cloud native journey using open source on azure thanks stormy every day more developers are innovating faster using cloud native development azure eliminates the friction of modernizing and migrating your applications to the cloud by providing flexible infrastructure azure is a platform that provides efficiency scalability portability and built-in monitoring to detect problems before they impact customers this allows developers to focus on delivering value using agile and devops best practices at the center of many microservice architectures is kubernetes here at microsoft we combine our development tools with azure to create a highly efficient inner loop for cloud native development everything a developer needs from source code management editors and ides infrastructures code container registries and ci cd tool chains all designed to work together this empowers us as developers to move code from our dev machines to production with a simple commit we are dedicated to supporting the vibrant ecosystem around kubernetes we work closely with open source communities such as cncf to share our knowledge of working with enterprises and to contribute to the future of kubernetes thanks to dapper writing microservices does not have to be hard dapper is an open source project that takes care of the heavy lifting you need to save state retrieve a secret or call another service just make an http or grpc request and dapper takes care of the rest dapper offers building blocks to help developers overcome common challenges when building distributed applications including service to service invocation secret and state management as well as input and output bindings not a single line of your code has to change to move from your dev machine to the cloud or edge dapper also offers observability and security features and conforms to standards to help you follow best practices if you have ever wanted to run a piece of code in response to an event and scale without having to worry about infrastructure azure functions are for you azure functions enable event driven serverless architecture with an on-demand consumption based cost model azure functions also offer increased agility improved resilience and scalability collaboration security and devops best practices are essential to a team's success github has defined the way developers collaborate and is used by over 60 million developers github enables teams to take an end-to-end view of the security from source code to deployed solution and this entire process can be automated using github actions what i'd like to do now is show what is possible when we combine azure and our developer tools i'm going to take a java app built using the new microsoft build of openjdk and deploy it to aks using github actions it uses dapper to fetch tweets from twitter and scores them with azure cognitive services then the tweets are stored in azure storage and published to azure service bus before being shown to users dapper makes this all possible with simple http request let's go one of the cool things about dapper is you can use it with any language even languages that haven't been invented yet if your language of choice can make an http or grpc request you can use dapper however if you prefer using an sdk they are available for many languages i decided to write this application in java using the microsoft build of openjdk that is a drop-in replacement for any open jdk you might be already using we also ported the open jdk to windows on arm and the new macbook m1 devices as a developer of this application task number one is figuring out how to get tweets from twitter thanks to dapper i did not have to learn the twitter apis or install any sdks i just defined a twitter binding with a query and when a tweet is sent that matches our service gets called i simply wrote a post method and dapper delivered the tweets on lines 39 through 44 i am setting the urls to use the dapper components http request to these urls will allow me to call another service store state and publish events when i'm running locally i will be using redis for my state store and pub sub however when i deploy to azure i will be using azure storage and azure service bus but notice there is no need for redis or azure specific code because i'm using dapper here is an example of a service to service invocation i am using the standard http client class to issue a post request to dapper dapper will find the other service on the network and call the desired method for me the service on the other end just defines a method that accepts a post request to the same route services don't even have to know they're being dapperized to reap the benefits services run with dapper will get observability and tracing for free storing state or placing a message on a topic can all be accomplished by making an http request to the correct endpoint let's run the application locally i have already started the viewer and processor services now i'm going to run the provider using dapper so it starts getting tweets with dapper streaming logs from all three services i feel like a tv show hacker switching to my browser confirms that the third service is processing the tweets from the dapper pub sub component with my local testing complete it's time to move the code to azure because i'm using dapper i don't have to change a single line of my code all i have to do is update the component configurations to point to azure resources the same is true if i move my code to another cloud or to the edge i'm going to deploy my entire solution including my infrastructure using github actions my infrastructure will be deployed using bicep which is a language that allows me to implement infrastructure as code infrastructure as code is a devops best practice where you codify your target infrastructure and automate its deployment using github actions i can also codify my entire workflow using yaml and target mac os windows and linux this is a github actions workflow that builds docker images deploys the infrastructure and installs the application using a helm chart into aks because this workflow can be triggered every time a developer commits code to a repo we can implement continuous integration or ci ci is the practice upon every commit the code is downloaded compiled tested and packaged for deployment here in github picking a run will take us to the summary notice we build three docker images and deploy infrastructure at the same time drastically reducing the time to execute the workflow you can review the individual steps and logs by selecting a job for instance here is a step that used the azure cli to deploy our infrastructure using bicep examining the deployment job illustrates how flexible github actions can be there is an entire marketplace with thousands of actions that you can use to build your workflows if you don't find what you need you can always create your own or you run a custom script for example in the get ip step i was able to query the ip addresses of the services exposed by aks using a powershell script clicking the first address will take you to the viewer now running in azure reading tweets from azure service bus instead of redis this is one example of how easy it is to move your code from your dev machine to the cloud without changing a single line of code the second address allows us to review trace information automatically provided by dapper using zipkin here is a trace of a tweet entering the app from the twitter input binding being scored by the sentiment service saved to the state store and making its way to the viewer this level of trace information is invaluable for troubleshooting microservices and is provided automatically when you dapperize an application writing microservices no longer has to be scary combining the power of dapper github actions and azure enables anyone to build and run microservices at scale today we're announcing the general availability of the microsoft build of openjdk for java 11. this is a new long-term support distribution that is free open source and can be deployed anywhere you might be wondering why first because we run over 500 000 jvms here at microsoft we use java for linkedin minecraft and many other internal systems including managing azure infrastructure secondly we have many customers using java having our own build of open jdk allows us to quickly respond to the needs of our customers and internal teams this also enables us to participate and contribute back to the java community to learn more visit microsoft.com openjdk now it's time for sarah novotny to reveal how you can unlock innovation faster by using our open services and applications thanks so much donovan i'm sarah novotny and i can't wait to show you how to use the latest ai in your applications to both enhance and innovate faster i'll give you a quick teaser though microsoft's here to guide you on your entire ai journey we are empowering every developer and every organization to harness the full potential of ai azure ml allows you to build ai on your own terms in the framework of your choice you can build train and deploy ml models using open source python machine learning libraries and platforms you can manage and track end and machine learning life cycles using ml flow remove performance bottlenecks using pi torch profiler and deploy it anywhere using the onyx runtime onyx runtime is a high performance inference training engine compatible with your favorite ml frameworks and supports accelerated training of transformer models making them 45 percent faster we want to ensure that organizations can build their ml models with the full confidence knowing that they own and control their data on a platform which adheres to some of the industry's strictest policies and privacy standards and has most comprehensive compliance portfolio of any cloud provider our ml toolkits are fully open source and allow you to understand your model and build for fairness protect data privacy and confidentiality and to control and govern everything through every step of the ml process we're putting responsible ml principles into practice and enabling data scientists and developers to innovate responsibly ai is not nearly as valuable without data and that's why we have open data sets we want to empower you to innovate faster by focusing on the work unique to your mission and differentiation these public data sets are accessed via an api and include public domain data for weather census holidays public safety things like that when you need to add your own data sets to the mix you can store it in our fully managed open source databases we give you the best of both worlds all the benefits of true open source communities coupled with the enterprise ready features such as scale high availability security and global reach with our fully managed azure databases you can focus on building your mission critical applications and let us manage your databases let's see a demo of this ai in action we'll be using pi torch profiler a native pi torch performance debugging tool to efficiently analyze and troubleshoot large scale deep learning model performance to maximize the hardware usage of expensive computational resources we will then show you how to optimize and deploy anywhere using onyx runtime both of these open source projects are made possible due to the partnership and contributions of both facebook and microsoft over to you elena thank you sarah let's see how python's profiler open source project developed in collaboration with facebook could help us collect data analyze it and find bottlenecks profiler comes pre-built into python itself starting from version 181 it's very easy to use we just need to add few lines of code in our notebook import profiler and set profile accession parameters we could run notebook and immediately switch to tensorboard tensorboard is fully integrated now with as code we could launch it right there as part of python extension and as we see visualization from our run we could immediately switch to python profiler which is one of tensorboard plugins and we would see overview window which provides high level aggregated data based on activity we could see here camera invocation mem copies data loader and so on and we could also see data aggregated by our model training step in this particular resonant 50 example data loader seems to take a lot of time significant time comparing to our activities in each and every step and that presents potential bottleneck as we could see profiler was able to recognize that problem and provide us actionable recommendation in this case it offers me a solution of increasing number of processes that actually load data into my model so let's follow recommendation and increase number of workers to four once i run the updated script i can collect data i see the data loader is no longer taking significant amount of time across my steps and that's even a decreased average step time for each of my model training so indeed this performance recommendation made by profiler helped us eliminate performance bottlenecks in our program let's take a look at more advanced views of python profile phase view it captures and shows activities across both cpu and gpu devices in fact python profile is first tool in the market that shows both pi torch and hardware activities in the single pane of glass and provides easy ability to create between them first thing we could notice that every step in our training is now depicted as separate highlighted block which makes it easy to navigate and zoom in in each particular step let's take a closer view as we zoom in we see gpu activities presented on the timeline and we could see that for some time gpu was idle here which is not good low gpu utilization pattern is one of the things that we need to look out and try to optimize as gpu is very expensive let's take a look at the first gpu invocation here that's memory copy invocation and we could see once we click on it dependencies and what was happening prior to it we see that this gpu kernel was invoked by pytorch operator 10 2 which copied tensor from cpu to gpu but what was blocking this operation attend 2 was waiting for single process data loader which was invoked before that and which was actually loading the data and while it was loading the data we see that gpu was actually idle so this view shows us how to easily create activities and helps us to resolve and see chain of operation and the resolve of the bottleneck space view is very helpful for detailed analysis in addition to trace we have couple more advanced views gpu kernel which shows the most expensive gpu invocations and the operator view which focuses on pie touch operators in this particular case we see that convolution backward in the segregated view was the most expensive by torch operation we could see also table and sorted by duration or number of calls and see tensor's input shapes for each pie touch operator lately we also had the ability to see a call stack of all code lines that led to expensive python invocation once selected profiler would take us to the actual line of code that led to this invocation which is very helpful for development to take a look at the source code and jump back to the data and all of it in the same vs code view to summarize we have seen how easy it is to use new python profiler users can easily enable profiler with a few lines of code and immediately switch to analysis and a resolution of bottlenecks and it all seamlessly integrated in vs code now i would like to share additional exciting options for model acceleration that was recently contributed to pytorch community onyx runtime training model onyx runtime is open source project that is designed to accelerate machine learning across a wide range of frameworks operating systems and hardware platforms onyx runtime now accelerates both inference and training in your family models it becomes especially important for large scale complex models such as popular hiking phase transformers and lp models it could take sometimes hours and even days to train here i would like to show profiling results from training popular t5 large model while average step time for training using regular python framework takes about 1.2 seconds same training steps using onyx runtime model takes down to 360 milliseconds that's a significant improvement it would save both time and money for our data scientists our goal was also to simplify developer experience to use linux runtime model all we need is to add just one line of code wrapping pytorch model with ort model no other additional changes are required into original training script which is extremely important for complex models like t5 large what i am showing here so how does it work our t model overrides pi torch model and generates forward and backward optimized computational graphs that run the training in the highly performant onyx runtime it optimizes with graphs created from pytorch model using number of techniques such as fusing sub graphs into single operators performing memory optimizations and kernel optimizations and all this power of acceleration is gained with one line of code that users need to add to their scripts and they would see immediately gains and get the savings we have demonstrated two projects pie touch profiler and onyx runtime model that enhance your developer experience using pytorch on azure sara will now share some exciting news with you thank you so much elena we just saw a demo of two of our investments in the pytorch ecosystem pi torch profiler and the onyx runtime microsoft and facebook remain active in the pytorch community and ecosystem we're thrilled to announce as well pi torch enterprise microsoft will provide two years of support for specially designated versions of pi torch starting with pytorch 1.8.1 while we're delivering a reliable enterprise experience we will also be upstreaming all of the bug fixes to the lts pi torch distribution that way everyone in the pytorch community can benefit azure is empowering every developer and is your trusted partner to accelerate your application development all you need to bring is your imagination so what are you going to build today out more about open source on azure at aka dot ms oss build 2021 and check out our blog for this session in order to get a deep dive into the cloud native applications and speeding up your innovation cycle with open source on azure don't forget to check out our open source at microsoft page that's at aka.ms.oss
there you can learn more about the projects that we're contributing to and the foundations that we support thank you and enjoy the rest of build you
2021-05-31