An introduction to the Azure OpenAI Service | BRK216

[Music] hi my name is dom divacaroni i'm group product manager of azure ai i'm an indian male in my 40s and i'm wearing a blue sweatshirt welcome to the session and thank you for joining us during this talk we'll look at how open ai and microsoft have been leading the advancement of ai to new frontiers and the differences these advancements are having for customers i'll reveal the next chapter of this journey to help enterprises leverage these transformational technologies at scale let's get started we've made great strides in ai over the past few years with advancements in virtually every domain automating and enhancing our everyday experiences for some tasks ai has surpassed human capability in the ability to see hear and understand but we still have a long way to go to develop ai that is truly intelligent and can use common sense reasoning to draw conclusions at microsoft it is our pursuit to continually advance ai to create intelligence capable of augmenting human ingenuity and democratize it so it empowers every person and organization on the planet to achieve more microsoft partnered with openai in 2019 to enable their vision of creating artificial general intelligence that benefits all humanity openai has been on the forefront of several major ai breakthroughs powered by azure ai's supercomputing platform openai launched gpg3 model last year and has been innovating at an incredible pace with models like clip and dali that can generate language about images and images from language and more recently open ai codex a code generation model that powers github copilot microsoft has leveraged these models within the power platform and power bi to make data and tools more accessible to customers last year openai launched the openai api to provide developers with an api to apply the ai models they develop this brand new platform opened up a whole world of new possibilities for developers not only does the api give you access to amazing generative capabilities of gpd3 and codex the api is designed in a way that is simple to use and flexible enough to make machine learning teams more productive with the open ai api in market for just over a year we've seen thousands of developers and companies use the capabilities to build new experiences and delight their customers enterprises of all sizes are building differentiate experience with the ability to comprehend summarize and extract information from context very quickly major brands like twitter reddit disney and several other new startups are driving value from the api in a very meaningful way duolingo is on a mission to develop the best education platform to make it universally available they serve millions of users worldwide and make language learning fun and effective duolingo's platform uses gpd3 to provide grammar corrections to improve language writing skills the openai api enables the agility to bring their innovations to market quickly jarvis is a another great example of a company that's transforming the creative process of how we write by using gpd3 jarvis makes it fast and easy to create content for blog social media websites and more it is used by tens of thousands of content marketers at companies including airbnb autodesk and ibm we're still in the early stages of learning the potential of gpd3 and we're excited to see the progress and value enterprises have derived from these models now as enterprises scale their use with business critical use cases they've asked for a platform that helps them meet the compliance regulatory requirements and support them with the performance and reliability for production scale enterprises taking their gpt-based apps to production increasingly need dedicated throughput and latency consistency for their workloads they also require isolation security and compliance guarantees that satisfy various regulatory standards all the fundamentals that have made the cloud the new normal for enterprises to meet these customer needs we are now launching azure openai service a new azure service that offers the openai api on azure with azure's enterprise grade capabilities the azure openai service lets users experiment with tailor and deploy open ai models at scale the service offers gpd3 and codex models with the openai api customization capabilities and tools to help customers have control over the content they generate and deploy these capabilities leverage azure's global cloud infrastructure meaning that it can support your toughest production workloads requirements around scale regional deployments security compliance and reliability the azure ai portfolio includes a wide range of cognitive services which provides specific apis that leverage world-class ai models to deliver capabilities that surpass human abilities for computer vision speech and language tasks like machine translation the azure open ai service joins these industry-leading azure cognitive services adding the amazing gpd3 and codex models to powerful foundation models that azure offers azure open ai expands what customers can do on azure i'm excited to show you how azure open ai service brings together the power of open ai with the power of azure to help developers and content creators provide engaging experiences the service is powered by a family of models with different capabilities and attributes the base series is a set of gpt3 models that can understand and generate natural language the service offers four base gpt3 models called davinci curie babbage and atta with different levels of capabilities suitable for different tasks the codec series is a set of models that can understand and generate code translating natural language to code and code to language davinci is the most capable model and can perform any tasks the other models can and often with less instruction for applications requiring a lot more understanding of the content like summarization for a specific audience and creative content generation davinci produces the best results on the other end of the spectrum is ada with the fastest of the gpt3 models and can perform tasks like parsing text address correction and tasks that don't require too much nuance the service makes it easy to experiment with these models and find the right balance between capability and latency performance for your use case now let's talk a bit about what these models can do gpt3 models can be applied to virtually any natural language task and enables a vast and expanding set of use cases for customers here are some of them gpd3 accelerates human creativity with ideas generated near instantaneously for businesses this means more effective product descriptions ads headlines and faster more effective customer correspondence for example microsoft's dynamic 365 marketing platform aims to reduce the time it takes for customers to write effective marketing content the app uses gpd3 combined with past context for producing tailored results to help users publish effective content quickly gpd3 is adept at understanding the nuances in text extracting meaning and summarizing information in the right tone for the right audience this helps companies like zebriam help businesses troubleshoot problems faster their solution parses logs correlates anomalies recognizes patterns to find the root cause it then uses gpd3 to summarize the root cause reports in plain english to help stakeholders make decisions quickly another example is ross intelligence which offers a legal research platform that uses gpd3 to better search through legal text synthesize law so that legal professionals can provide sound and timely advice to their clients gpt3 is helping transform the customer service industry with rapid access to information so agents can deliver a better customer experience whether it's retrieving information to resolve issues quickly extracting answers for a technical audience or summarizing insights from customer service conversations gpd3 helps companies like sapling build software products that help enterprises deliver a better customer experience for their customers companies like yabble are helping customers with accurate insights in these times of too much signal their solution applies gpd3 to summarize data into actionable insights users can iteratively ask questions of their data and derive insights aggregated from various sources whether it be customer support feedback and reviews using gpd3 yabul has built an insights tool that is helping customers respond to changing market demands with agility here is an example interaction with a tool extracting insights from over a thousand relevant pieces of customer feedback now let's talk about openai codecs the codex models are descendants of the base gpd3 models and are trained on billions of lines of source code in public github repositories to produce working code from natural language which means that you can issue commands in english to any piece of software with an api codex models are the most capable in python and proficient in over a dozen languages including javascript go ruby and sql just like with gpd3 you can tailor codecs to a variety of tasks including code generation tasks to see just how easily codecs can make unfamiliar libraries accessible to everyone let's turn to ryan to show us an example of using codex to build a bottom minecraft with the mindflayer library over to you ryan i'm using codex the code generating version of gpt3 to write code that will power a non-player character in minecraft thus far i have a prompt that is composed of a basic javascript comment explaining that the following is going to be a series of bot commands and the code needed to accomplish them using a specific library now i'll start by giving a command here go forward and i'll preface this by saying that the model has never actually seen the library so even though it generated code that looks reasonable this isn't code that that would actually execute in the mindflare library so let's give it the code it should have written bot.set control state forward to true and now i'll ask it to go backwards and sure enough and now produce the correct code using the spot.set spot.setcontrolstate function and it correctly guess that it should pass back as a parameter instead of forward to move in a different direction now let's see if by giving it a more complex example telling it to jump a few times we can generalize to more complex behavior so now i'll say move now move left a bit and it's produced several lines of code that actually correctly guess how to move left for some amount of time it saw from our example using the settimeout method that this is how we handle something temporal it you know i'd interpret it a few times as a thousand milliseconds and here it interpreted a bit as a thousand milliseconds as well i'll show one more example i'm going to say now right for longer and sure enough it produced the correct code here and we can see that it interpreted for longer as 2000 milliseconds this also demonstrates that we can carry context throughout our interaction with the npc uh to know what to do based on context from earlier in the conversation now i'll move over to minecraft to show this in action i'm now in minecraft and accompanied by my non-player character named boopbot we've continued to build on top of that prompt and given several more examples of the kind of code it can generate to enable new behaviors i'll start by showing some of the behaviors we showed in the last demo i'll say move left a bit you can see it generated that same code i'll say now write for longer and we're echoing the code down here in the chat and you can see that it now interpreted for longer as 2000 milliseconds instead of the 1000 milliseconds and correctly moved right i'll say jump a bit and it jumps for one second and now i'll say what directions have you moved codex is capable of generating not just code but also natural language and in this case it responded with a dialog i've moved left right and jumped so codex can generate code natural language and in this case it's even generating natural language about the code that it generated now we've enabled several complex behaviors just by prompt engineering i'll show one of those i'm going to ask it how do i make a crafting table it's a very important resource in minecraft and it was able to reason over some structured data that was in its prompt to answer you need four oak logs to make a crafting table now i'll say now get those for me and it's generated some more complex code here several lines of code to go mine those four blocks come over to the player and say i got your oak logs so just by generating prompts just by showing examples of how we can use an api or a library we're able to generate novel code that handles new behaviors that we never actually showed the model thanks ryan this is a truly impressive demonstration of the art of possible in gaming but it's not hard to imagine what this means for companies with codex enterprises can make their apis and tools more accessible to more users without a steep learning curve it makes developers more productive accelerating software development openai codex empowers computers to better understand people's intent which can empower everyone to do more with computers which is fascinating if you think about what it unlocks for the future this summer github launched github co-pilot which is an ai pair programmer that helps users write code faster with less work github copilot draws context from comments and code and suggests individual lines and whole functions instantly github copilot is powered by codex models running on azure open ai service hundreds of thousands of developers are using copilot today it is making programmers more effective and widening the aperture of who can be a programmer now let's look at how the service works the github copilot service talks to your vs code editor the editor sends context from your current file to the codex model running on the azure open ai service which responds with text that is likely to follow the content this is then displayed as suggestions to the editor and users can accept or tweak these suggestions these suggestions are remarkably accurate and they're getting smarter all the time let's look at some more examples of how microsoft is using azure open ai to innovate on behalf of our customers microsoft's power bi platform is making data analytics accessible for everyone with the ability to generate dax expressions with natural language dax is the expression language used in power bi in excel to define complex calculations customers can now use natural language to describe what they're trying to accomplish and have power bi automatically create dax expressions for them making sophisticated business logic accessible to everyone without having to become a dax expert the power platform enables app development without having to write any code except when you need to write formulas and power fx now with azure open ai users can express what they want to accomplish in natural language and power platform generates these formulas automatically saving time and skipping the learning curve we believe we're in the very early stages of innovation with these models with many more remarkable capabilities to come accelerating this innovation requires a new kind of platform that enables all users of all skill levels to apply their creativity and scale ideas from concept to production the azure open ai service enables this innovation with the openai api ai is advancing rapidly and we want to make sure that it's evenly distributed our shared vision with openai is to make these powerful models widely accessible to enterprises to deploy safely and securely democratizing these models is a powerful ai software platform the openai api that makes this technology incredibly accessible unlocking creativity with easy experimentation the openai api lets all builders from non-developers to ml experts go from building a compelling prototype in five minutes to scaling customized versions to production level loads the api's simple text in text out interface allows users to apply to virtually any language task or code task by showing it just a few examples of what you'd like it to do imagine if anyone in your company regardless of their programming experience had the ability to bring their creativity to life with a proof of concept in a few minutes illustrating the art of what's possible before you have to make a costly decision to fund a project it unlocks innovation and ideas for enterprises in a very meaningful way once you have your proof of concept developed the api makes it easy for developers to customize the model accuracy for their scenario this deeper customization which traditionally requires machine learning expertise and a steep learning curve is now made accessible to all developers without taking away any of the control that machine learning teams desire the azure open ai service helps you develop these concepts quickly and provides you with the tools to scale your apps in production in this example the user simply states the objective of the task which is to summarize game commentary provides the commentary texts and asks the question what are the highlights of the game so far the model generates a response with a summary this approach is called zero shot and which tends to be accurate about half the time but is effective for prototyping ideas very quickly you typically need a few more examples of the kind of summary you're looking for to make the generated text more relevant and reliable so the next step on the iteration journey is to employ the future approach by providing some curated examples illustrating the kind of detail you want to see in the summary once you develop a working proof of concept with this approach developers can apply additional customization commonly referred to as fine tuning to achieve higher accuracy and low latency performance the api makes it easy to submit a fine tuning job with prompt and completion examples and then deploy the customized model for testing and production the api is designed to be both simple for anyone to use but also flexible to make it for machine learning teams to be more productive the openai api and models provide tremendous capabilities for handling new customer scenarios and help businesses achieve more but to apply these scenarios safely enterprises also need the capabilities and tools to ensure that the models only generate content that is suitable for their application and to prevent misuse we've designed the azure openai api with built-in features to empower customers to responsibly apply this technology in a wide range of applications with content filters customers can use to customize the tone and topic of the content that the models produce we've also added safety features to ensure that everyone is using the system for its intended purpose finally we will be releasing guidance and implementation practices to help customers design their applications while keeping safety front and center to talk about the importance and our approach to the responsible ai here's sarah byrd hi sarah hi dom it's great to be here sarah help us understand what responsibility is and why it's important response play eyes big topic but in short it's about ensuring the technology we build has a positive impact in the world and intentionally designing with that impact in mind this is particularly important for foundation models like gpt3 where many people build on top of the same model so potential issues can be amplified across all of those applications but it works the other way as well if we get it right many applications can benefit from that you bring up a really good point let's get into some specifics how do we develop our approach for responsible ai on azure open ai with any new technology we start with a process that we call an impact assessment where we assess the benefits of the technology and the potential risk for azure openai in particular we were really focused on three potential challenges first gpt3 is trained on data from the internet and as we've all seen sometimes it sounds exactly like that which may not be appropriate for every application we didn't want each customer that deployed the technology to have to solve this problem on their own so we've developed customizable content filters which give customers the power to control what the model says second gpt3 is a powerful model that can be used for many things when our customers build an application they have a particular use in mind but it's possible that someone might try to break out of that use to leverage the technology for their own purposes we built in safety features and worked with customers to implement additional mitigations in their application to ensure that the technology is only used for its intended purpose finally gpt3 is a tool to aid humans but for users to leverage this tool productively they need to understand how it works and how to control it there's a lot that a customer can do in the design of their application to empower users we're working with customers to develop best practices for design and implementation including ui design patterns that we will release with the service overall the goal of these features is to empower our customers to build the best application they can on top of azure open ai and bring the benefits of this technology to many people around the world it's exciting to hear and we have a lot of hard work ahead of us to help our customers adopt these technologies so where do we go from here what's coming in the future we're still in the early development of this type of technology so we expect to see a lot more innovation coming on the responsible ai front we're partnering with our customers during the preview period to learn and enhance our existing safety features and guidelines we're also working with microsoft research and ether to develop new tools to test generative language applications finally openai is the leader in responsible ai as well and we'll be partnering with them to improve the core technology itself so stay tuned lots more exciting things to come well thanks a lot for joining us sarah we have a lot of hard work ahead of us to ensure we help our customers deploy these technologies responsibly and at scale thanks for having me dom azure openai provides powerful capabilities and a delightful experience delivered to support your toughest production requirements for scale regional deployments security compliance and reliability we're excited to launch the azure open ai service a service that truly empowers our customers with powerful ai capabilities and the agility that are key to unlocking the next generation of ai-powered apps the service is available initially by invitation to customers that have well-defined use cases in mind that represent responsible uses of ai technology collaborations with these early customers will help us make sure the responsible ai safeguards are working in practice so we can scale adoption more broadly please visit azure.com to learn more and

thank you you

2021-11-06 16:06

Other news