Full Keynote: Satya Nadella at Microsoft Build 2024
I think that our industry has to have a common vision. It was a time that connected us to incredible things. My name for this vision is, information at your fingertips. And three decades later, we find ourselves in a new era. One where access to information becomes access to expertise. From the farm, to the lab, from the boardroom, to the classroom, this new generation of AI is for everyone, everywhere.
Now, anyone can save time with a personal assistant. With GitHub Copilot, I’m saving about 50% of time. And that's time that I can use to do other innovative things. It allows me to find out the condition of my ponds faster. Anyone can access a personal tutor to learn new skills.
We got to learn about banking: How to apply for a loan, how to save money. We learned so much. I think this technology has the potential to completely reimagine the way every single student learns in the world. This is a new way to analyze with a personal coach. We're going to be able to have not only productivity gains, but insights served to us, near real-time.
Generative AI can learn from the data to help improve the farmer productivity. AI is unlocking creativity for us all. Descriptions are so detailed, in my imagination I can paint the artwork. Now teachers are free to create lesson plans according to our needs.
With expertise at your fingertips. You can build, what matters. Welcome to the age of AI transformation. Good morning. Good morning. It's fantastic to be back here at Microsoft Build.
Welcome to everyone here and joining us on the web. You know, developer conferences are always most exciting, most fun when there's these fundamental changes that you can sense in the air. You know, I've marked all my adult life by coming to PDCs and Builds for the last three decades. I still remember, you know, distinctly the first time Win32 was discussed, I guess it was ‘91, .NET, Azure, right? These are moments that I marked my life with.
And it just feels like we're yet again at a moment like that. It's just that the scale, the scope is so much deeper, so much broader this time around, right? Every layer of this tech stack is changing, you know, from everything from the power draw and the cooling layer of the data center to the NPUs at the Edge are being shaped by these new workloads, right? These distributed, synchronous, data parallel workloads are reshaping every layer of the tech stack. But if you think about even going all the way back to the beginning of modern computing, say, 70 years ago there have been two real dreams we've had. First is can computers understand us instead of us having to understand computers? And second, in a world where we have this ever increasing information of people, places and things, right? So, as you digitize more artifacts from people, places and things and you have more information, can computers help us reason, plan and act more effectively on all that information? Those are the two dreams that we've had for the last 70-plus years. And here we are. I think that we have real breakthroughs on both fronts.
The core underlying force, one of the questions I always ask myself is like, “Okay, this is great. This is like maybe the golden age of systems. What's really driving it?” I always come back to these scaling laws, just like Moore's Law, you know, helped drive the information revolution. The scaling laws of DNNs are really, along with the model architecture, interesting ways to use data, generate data, are really driving this intelligence revolution. You could say Moore's Law was probably, you know, more stable in the sense that it was scaling at maybe 15 months, 18 months.
We now have these things that are scaling every six months or doubling every six months. You know, what we have, though, with the effect of these scaling laws is a new natural user interface that's multimodal. That means supports text, speech, images, video as input and output. We have memory that retains important context, recalls both our personal knowledge and data across our apps and devices. We have new reasoning and planning capabilities that helps us understand very complex context and complete complex tasks.
While reducing the cognitive load on us. But what stands out for me as I look back at this past year is how you all as developers have taken all of these capabilities and applied them, quite frankly, to change the world around us. I’ll always remember this moment in January 2023 when I met a rural Indian farmer who was able to reason over some government farm subsidies that he had heard about on television using GPT-3.5 and his voice. It was remarkable right? For me, it just brought home the power of all of this because a frontier model developed in the West Coast of the United States just a few months earlier was used by a developer in India to directly improve the life of a rural Indian farmer.
The rate of diffusion is unlike anything I've seen in my professional career, and it's just increasing. In fact, earlier this month I was in Southeast Asia. I was in Thailand where I met a developer and I was having a great roundtable and he was talking to me about how he's using Phi-3 and GPT-4 and he was using Phi-3 to just optimize all of the things that he was doing with RAG. I mean, this is crazy I mean, this is unbelievable. It had just launched a few weeks earlier and I was there in Thailand, in Bangkok, listening to a developer talk about this technology as a real expert on it.
So it's just great to see the democratization force that we love to talk about but to witness it is just been something. And this is, quite frankly, the impact of why we are in this industry. And it's what gives us, I would say that deep meaning in our work. So I want to start, though, with a very big thank you to every one of you who is really going about bringing about this impact to the world.
Thank you all so very much. You know, when I think about what progress we've made even since last time we were here at Build, we built really three platforms. The first is Microsoft Copilot, which is your everyday AI companion. It puts knowledge and expertise at your fingertips, helps you act on it. And we built the Copilot stack so that you can build your AI applications and solutions and experiences. And just yesterday, we introduced a new category of Copilot+PCs, the fastest AI-first PCs ever built.
All three of these things are exciting platforms but I want to start with Copilot+ PCs. You know, we're exposing AI as a first-class namespace for Windows. This week we are introducing the Windows Copilot Runtime to make Windows the best platform for you to be able to build your AI applications. Yeah. You know what Win32 was to graphical user interface, we believe the Windows Copilot Runtime will be for AI.
It starts with our Windows Copilot library, a collection of these ready-to-use local APIs that help you integrate into your new experiences all of the AI capabilities that we shared yesterday. Now, this includes no code integrations for Studio Effects things like creative filters, teleprompter, voice focus, and much more. But of course, if you want to access these models itself, you can directly call them through APIs. We have 40 plus models available out of the box, including Phi-Silica our newest member of our small language family model, which we can specific, which we specifically designed to run locally on your NPUs, on Copilot+ PCs bringing that lightning-fast local inference to the device. You know, the other thing is the Copilot library also makes it easy for you to incorporate RAG inside of your applications on the, on device data.
It gives you the right tools to build a vector store within your app. It enables you to do that semantic search that you saw with Recall. But now you can, in your own application, construct these prompts using local data for RAG applications. Now, I’m so thrilled to announce as well today that we will be natively supporting PyTorch and new WebNN framework through Windows DirectML. Native PyTorch support means thousands of OSS models will just walk out of the box on Windows, making it easy for you to get started. In fact, with WebNN, web developers finally have a web-native machine learning framework that gives them direct access to both GPUs and NPUs in fact, last night I was playing with it, turning it on in Edge and seeing the WebNN sample code running.
It's just so cool to see it you know, now use even the NPUs. Both PyTorch and WebNN are available in Developer Preview today, let's take a look. And these are just one of the many announcements today. We're introducing more than 50-plus new products and partnerships to create a new opportunity for you. We’ve always been a platform company and our goal is to build the most complete end-to-end stack from infrastructure, to data, to tooling to the application extensibility so that you can apply the power of this technology to build your own applications.
And so today I want to highlight our top news for this event across every layer of this Copilot stack. So let's dive right in with infrastructure. You know, we have the most complete scalable AI infrastructure that meets your needs in this AI era. We're building Azure as the world's computer. We have the most comprehensive global infrastructure with more than 60-plus datacenter regions, more than any other cloud provider. Over the past year, we have expanded our datacenter regions and AI capacity from Japan to Mexico, from Spain to Wisconsin.
We're making our best-in-class AI infrastructure available everywhere and we're doing this with a focus on delivering on cloud services sustainability. In fact, we're on track to meet our goal to have our data centers powered by 100% renewable energy by next year. Yeah. You know, we’re optimizing power and efficiency across every layer of the stack from the data center to the network. Our latest data center designs are purpose built for these AI workloads so that we can effectively and responsibly use every megawatt of power to drive down the cost of AI and the power draw. And we are incorporating advanced data center cooling techniques to fit the thermal profile of the workloads and match it to the environment in the location where it operates.
And the silicon layer, we are dynamically able to map workloads to the best accelerated AI hardware so that we have the best performance. And our custom IO hardware and server designs allow us to provide dramatically faster networking, remote storage and local storage throughput. You know, this end-to-end approach is really helping us get to the unprecedented scale.
In fact, last November we announced the most powerful AI supercomputer in the cloud for training. Using just actually a very small fraction of our cloud infrastructure. And over the past six months we've added 30 times that supercomputing power to Azure. Yeah, it's crazy to see the scale. And of course we're not just scaling training our fleets, we’re scaling our inference fleet around the world, quadrupling the number of countries where Azure AI services are available today and it's great to see that. At the heart of our AI infrastructure are the world's most advanced AI accelerators, right? We offer the most complete selection of AI accelerators, including from NVIDIA and AMD, as well as our own Azure Maia, all dynamically optimized for the workloads.
That means whether you're using Microsoft Copilot or building your own Copilot apps, we ensure that you get the best accelerator performance at the best cost. For example, you know, you see this in what has happened with GPT-4, right? It's 12x cheaper and 6x faster since it launched. And that's, you know, the type of progress you can continue to see, how, you know, you continue to see the progress as we evolve the system architecture. It all starts, though, with this very deep, deep partnership with NVIDIA, which spans the entirety of the Copilot stack across both all of their hardware innovation as well as their system software innovation.
Together, we offer Azure confidential compute on GPUs to really help you protect sensitive data around the AI models end to end. We're bringing in fact the latest H200s to Azure later this year, and will be among the first cloud providers to offer NVIDIA's Blackwell GPUs in B100 as well as GB200 configurations. And we are continuing to work with them to train and optimize both large language models like GPT-4o, as well as small language models like the Phi-3 family.
Now beyond the hardware, we are bringing NVIDIA’s key enterprise platform offerings to our cloud, like the Omniverse Cloud and DGX Cloud to Azure with deep integration with even the broader Microsoft Cloud. For example, NVIDIA recently announced that their DGX Cloud integrates natively with Microsoft Fabric. That means you can train those models using DGX Cloud with the full access through Fabric data. And Omniverse APIs will be available first on Azure for developers to build their industrial AI solutions. We're also working with NVIDIA’s NIM industry specific developer services and making them fantastic on Azure. So, a lot of exciting work with NVIDIA.
Now, coming to AMD, I am really excited to share that we are the first cloud to deliver general availability of VMs based on AMD’s MI300X AI accelerator. It's a big milestone for both AMD and Microsoft. We've been working at it for a while and it's great to see that today as we speak, it offers the best price performance on GPT-4 inference. And we'll continue to move forward with Azure Maia. In fact, our first clusters are live and soon if you're using Copilot or one of the Azure OpenAI services, some of your prompts will be served using Maia hardware.
Now beyond AI, our end to end systems optimization also makes cloud-native apps and the development of cloud-native apps better, right? Six months ago is when we announced our first general purpose ARM-based compute processor Microsoft Cobalt. And today I am really excited to announce the public preview of Cobalt-based Vms. You know, Cobalt is being used for video processing and permissions management in Microsoft 365, helping power billion of conversations on services like Microsoft Teams already. And we are delivering that same ARM-based performance and efficiencies to many customers, in fact, including Elastic, Mongo, Siemens, Snowflake and Teradata. In our most recent benchmark data and tests, our Cobalt 100 VMs delivered up to 40 percent better performance than any other generally available ARM-based VMs. So we are very very excited about Cobalt getting into the market.
Now let's move up the stack to the foundation models. With Azure AI, we offer the broadest selection of frontier and open source models, including LLMs and SLMs, so you can choose the model that makes the most sense for your unique needs and your application needs. In fact, more than 50,000 organizations use Azure AI today. Yeah.
It's great momentum and it all starts though, with our most strategic and most important partnership with OpenAI. Just last week, OpenAI announced GPT-4o, for all their latest multimodal model, which was trained on Azure. It's an absolute breakthrough.
It has text, audio, image and video as input and output. It can respond and just have a humanlike conversation that's fast and fluid. It can even be interrupted mid-sentence. GPT-4o is also the top performing model on benchmarks across a variety of modalities and it's always going to get you to the best answer. It has state-of-the-art performance at understanding the information you actually provide in your prompt because that's really what matters.
What OpenAI demoed last week, I mean, that was just pretty magical stuff. And yesterday we showed how Copilot will leverage GPT-4o. In the video I'm just going to play, notice that you can share your screen or session as a prompt in Copilot and have Copilot help you in whatever it is that you're doing.
Let's roll the video. Hey Copilot, how's it going? Hey, I'm doing great. And it looks like you're about to dive into some Minecraft. Ready to build, explore and maybe dodge a few mobs? Yeah, I'm trying to figure out how to make a sword.
My son plays this game all the time and I have no idea what I'm doing. No worries. I'm here to help you impress your son. I see you've got a crafting table in front of you, which is a great start. To craft a sword you'll need some materials.
Can you open your inventory? Just press “E” on your keyboard. Yeah. Do I have what I need? Let's see. You've got some sticks. Perfect for the swords handle.
But it looks like you're missing the material for the blade. You can use wood, stone, iron, gold or diamond. Let's start by gathering some wood or stone.
Shall we head out and find some resources? Yeah, that sounds great. Oh my god, what's that? Oh no, it's a Zombie! Run! You need to get away from it fast. Either sprint away or quickly build a tower of blocks straight up to get out of its reach.
If you can find a shelter or dig into the side of a hill, do it now. Just make sure it can't reach you. Alright, I think I might be safe now. That was a close one. Great job finding shelter. It's really cool to see that.
You know, the magic here is that Copilot is your personal assistant that's helping you with any task, helping you stay, quite frankly, in the flow in the moment, whether it's a game or a document or a line of code. And as OpenAI innovates, our promise is that we will bring all that innovation to Azure too, in fact, the same day that OpenAI announced GPT-4o, we made the model available for testing on Azure OpenAI Service and today I'm excited to say that it’s generally available on Azure AI. You know, what this means though is that now we can have these groundbreaking apps that all of you can build using this capability. One of the coolest things is that now any app, any website, can essentially be turned into a full multi-modal, full duplex conversational canvas.
Let's take a look. Alright, let me ask the agent here. Hey, Seth. How's it going? What can I help you with today? I am fantastic. I'm sorry, I'm in such a hurry. I'm going to go on an overnight camping hike, where we're camping - it's going to be dark, cold.
Should I use these shoes? It's the mountains. It's probably going to be cold. Hey, no worries. Let me take a look. Ah, those are the TrekStar hiking sandals.
While they're fantastic for summer hikes and keeping your feet cool, they might not be the best choice for cold - Okay, I'm sorry to interrupt you. Pick the best thing that I should use, and can you just add it to my cart? Just one product. No worries. Let's go with the TrekReady hiking boots. They're durable, provide great ankle support, and most importantly, they'll keep your feet warm in the cold mountains.
I'll add them to your cart now. Ready for checkout? I see that, amazing. Thank you very much. I hope to have a good time camping.
You’re welcome, Seth. Enjoy camping and stay warm in those boots. Good luck and have fun in the mountains.
While Seth was, you know, Seth was in a hurry, but there you go. GPT-4o kept up with him. It's incredible.
I really want to thank the OpenAI team for their partnership and and really the responsible approach to innovation, helping our industry move forward. Sam will be here in fact, joining Kevin in a little bit to talk a lot more about what's coming because that's the exciting stuff, how do you all sample what comes next. We also, I brought, we're bringing lots and lots of other models as well from Cohere and Databricks and Deci, Meta, Mistral, Snowflake, all through Azure AI. We want to support the broadest set of models from every country, every language. I'm excited to announce, in fact, we're bringing models from Cohere, G42, NTT DATA, Nixtla, as well as many more, as models of services, because that's the way you can easily get to managed AI models.
And we all love open source, too. In fact, two years ago at Build, we were the first to partner with Hugging Face, making it simple for you to access the leading open source library with state-of-the-art language models via Azure AI. And today I'm really excited to announce that we're expanding our partnership, bringing more models from Hugging Face with text generation inference, with text embedding inference directly into Azure AI Studio. And, and we're not stopping there.
We are adding not just large language models, but we are also leading the small language revolution. So small language model revolution, you know, our Phi-3 family of SLMs are the most capable and most cost effective. They outperform models of the same size or the next size up even across a variety of language reasoning, coding, as well as math benchmarks. If you think about it by performance to parameter count ratio, it's truly best in class. And today we're adding new models to the Phi-3 family to add even more flexibility across that quality cost curve. We're introducing Phi-3 Vision, a 4.2 billion parameter
multimodal model with language and vision capabilities. It can be used to reason our real-world images so generate insights and answer questions about images. As you can see right here. Yeah. And we're also making a 7 billion parameter Phi-3 small in a 14 billion parameter Phi-3 medium models available. With Phi, you can build apps that span the web, your Android, iOS, Windows and the Edge. They can take advantage of local hardware when available and fall back on the cloud.
We're not simplifying really all of what we as developers have to do to support multiple platforms using one AI model. Now, it's just awesome to see how many developers are already using Phi0-3 to, you know, do incredible things. From Amity Solutions, the Thai company that I mentioned earlier, the ITC, which is been built a copilot for Indian farmers to ask questions about their crops. Epic in health care which is now using Phi to summarize complex patient histories more quickly and efficiently. And out of the very, very cool use cases in education. Today, I'm very thrilled to announce a new partnership with Khan Academy.
We'll be working together to use Phi-3 to make math tutoring more accessible. And I'm also excited to share that they'll be making Khanmigo their AI assistant free to all US teachers. Let's roll the video. I felt like I was in a place in my teaching career where I felt like I was kind of losing my sparkle. And I would just feel really defeated when I looked out on the classroom and I would see students that just didn't look engaged. Teachers have an incredibly hard job and what we think we can do is leverage technology to take some of the stuff off of their plate, to really actually humanize the classroom.
By some miracle, we became a Khanmigo pilot school. With new advances in generative AI, we launched Khanmigo. The point is to be that personalized tutor for every student and to be a teaching assistant for every teacher. I started to build these more robust lessons and I started to see my students engage.
We're working with Microsoft on these Phi models that are specifically tuned for math tutoring. If we can make a small language model like Phi, work really well in that use case, then we would like to, kind of, shift the traffic to Phi in those particular scenarios. Using a small language model, the cost is a lot lower.
We're really excited that Khanmigo, and especially in the partnership with Microsoft, being able to give these teacher tools for free, to U.S. teachers is going to make a dramatic impact on U.S. education. I think we're going to make them the innovators, the questioners, isn't that really just why you wake up every morning? Right? Because that's our future, our next generation. And to me, that's everything.
You know, I’m super excited to see the impact this all will have and what Khan Academy will do. And Sal is going to, in fact, join Kevin soon to share more. And I'm really thankful for Teachers like Melissa and everything that they do. Thank you very much. You know, of course, it's about more than just models. It's about the tools you need to build these experiences.
With Azure AI Studio we provide an end-to-end tooling solution to develop and safeguard the copilot apps you build. We also provide tooling and guidance to evaluate your AI models and applications for performance and quality, which is one of the most important tasks as you can imagine with all of these models. And I'm excited to announce that Azure AI Studio now is generally available. It's an end to end development environment to build, train, and fine tune AI models – and do so responsibly. It includes built-in support.
For what is perhaps the most important feature, which is, in this age of AI, which is AI Safety. Azure AI Studio includes the state of the art safety tooling. You know, to everything from detecting hallucinations in model outputs, risk and safety monitoring. It helps understand which inputs and outputs are triggering content filters. Prompt shields, by the way,
to detect and block these prompt injection attacks. And so today we are adding new capabilities, including custom categories, so that you can create these unique filters for prompts and completions with rapid deployment options, which I think is super important as you deploy these models into the real world. Even when an emerging threat is, you know, appears. Beyond Azure AI Studio, we recognize that there are advanced applications where you need much more customization of these models for very specific use cases. And today I'm really excited to announce that Azure AI custom models will come, giving you the ability to train a custom model that's unique to your domain, to your data, that's perhaps proprietary.
That's same builders and data scientists who’ve been working with Open AI, brought all the Phi advances to you, will work with all of you to be able to build out these custom models. The output will be domain specific. It will be multitask and multimodal, best in class as defined by benchmarks, including perhaps even specific language proficiency that may be required.
Now, let's just roll up the stack to data. Ultimately, in order to train fine-tune, ground your models, you need your data to be in its best shape. And to do so, we are building out the full data estate right from operational stores to analytics in Azure. We’ve also added AI capabilities to all of our operational stores, whether it's Cosmos DB or SQL, or PostgreSQL. At the core though, of the Intelligent Data Platform. Is Microsoft Fabric.
We now have over 11,000 customers, including leaders in every industry who’re using Fabric. It's fantastic to see the progress. With Fabric, you get everything you need in a single integrated SAS platform. It's deeply integrated at its most fundamental level with compute and storage being unified. Your experience is unified, governance is unified, and more importantly, the business model is unified. And what's also great about Fabric is that it works with data anywhere, right? Not just on Azure, but it can be on AWS or on GCP or even in your on-premise data center.
And today we are taking the next step. We're introducing Real-Time Intelligence in Fabric. Customers today have more and more of this real-time data coming from your IoT systems, your telemetry systems. In fact, cloud applications themselves are generating lots of data, but with Fabric, anyone can unlock actionable insights across all of your data estate.
Let's take a look. Introducing real-time intelligence in Microsoft Fabric, an end-to-end solution empowering you to get instant actionable insights on streaming data. At its heart lies a central place to discover, manage, and consume event data across your entire organization with a rich governed experience. Get started quickly by bringing in data from Microsoft sources and across clouds with a variety of out-of-the-box connectors.
Route the relevant data to the right destination in Fabric using a simple drag-and-drop experience. Explore insights on petabytes of streaming data with just a few clicks. Elevate your analysis by harnessing the intelligence of Copilot in Microsoft Fabric using simple natural language.
Make efficient business decisions in the moment, with real-time actionable insights and respond to changing landscapes proactively. Allow users to monitor the data they care about, detect changing patterns, and set alerts or actions that drive business value. All your data, all your teams, all in one place.
This is Microsoft Fabric. And, we're making it even easier to design, build and interoperate with Fabric with your own applications, right? And in fact, we're building out a new app platform with Fabric Workload Development Kit so that people like ESRI, for example, having, you know, who have integrated their spatial analytics with Fabric so that customers can generate insights from their own location data using ESRI’s rich tools and libraries, right on Fabric, right. This is just exciting to see As the first time, you know, where the analytics stack is really a first-class app platform as well. And beyond Fabric, we are integrating the power of AI across the entirety of the data stack. There's no question that RAG is core to any AI-powered application, especially in the enterprise today. And Azure AI Search makes it possible to run RAG at any scale, delivering very highly accurate responses using the state of the art retrieval systems.
In fact, ChatGPT supports for GPTs, their Assistants API, are all powered by Azure AI Search today. And with built-in OneLake integration. Azure AI Search will automatically index your unstructured data too. And it's also integrated into Azure AI Studio to support bringing your own embedding model, for example.
And so it's pretty incredible to see Azure Search grow over the last year into that very core developing service. Now let's go up through developer tools. Nearly 50 years after our founding as a developer tools company, here, we are once again redefining software development, right? GitHub Copilot was the first, I would say, hit product of this generative AI age. And it's the most widely adopted AI developer tool, 1.8 million subs across 50,000 organizations, are using it. And GitHub Copilot, we're empowering every developer on the planet to be able to access programing languages and programing knowledge in their own native language.
Think about that. Any person can start programing, whether it's in Hindi or Brazilian Portuguese, and then bring back the joy of coding to their native language. And with Copilot Workspace, staying in your flow has never been easier. We are an order of magnitude closer to a world where any person can go from idea to code in an instant. You start with an issue, it creates a spec based on its deep understanding of your codebase.
It then creates a plan which you can execute to generate the code across the full repo that is multiple files. At every point in this process – from the issue, to spec, to plan, to code, you are in control, you can edit it. And that's really what is fundamentally a new way of building software.
And we are looking forward to making it much more broadly available in the coming months. And today, we are taking one more big leap forward. You know, we are bridging the broader developer tools and services ecosystem with Copilot for the first time. We are really thrilled to be announcing GitHub Copilot Extensions. Now you can customize GitHub Copilot with capabilities from third-party services, whether it's Docker, Sentry, and many, many more. And of course we have a new extension for Azure too: GitHub Copilot for Azure.
You can instantly deploy to Azure to get information about your Azure resources just using natural language. And what Copilot did for coding. We are now doing for infra and ops.
To show you all this in action here is Neha from our GitHub team. Neha, take it away. Thanks Satya.
GitHub Copilot gives you suggestions in your favorite editor like here where I'm writing unit tests. Copilot is great, at meeting you where you're at regardless of the language you're most comfortable with. So, let's ask for something simple, like how to write a prime number test in Java? But, let's converse in Spanish using my voice. How to check if the given number, is a prime number in java? Look at that. Thank you, Copilot. Copilot is great at turning natural language into code and back again.
But, what about beyond the code with the new GitHub Copilot Extensions, you can now bring the context from your connected systems to you. So, now I can ask Azure, where my app is deployed. I could ask what my available Azure resources are or I could diagnose issues with my environment. And this isn't just for Azure. As Satya announced, any developer can now create Extensions for GitHub Copilot, and that includes any tool in your stack. Include your in-house tools, keeping you in the flow across your entire day.
Actually, 75% of a developer's day is spent outside of coding: gathering requirements, writing specifications, and creating plans. Let's show how GitHub Copilot can help with that. Live, on stage, for the first time.
So typically, my day starts by looking at GitHub issues. Looks like we want to support a rich text input for our product description. Let's open Workspace and get some help with that. Copilot interprets the intent of the issue to see what's required. And it then looks across the entire codebase, and it proposes what changes should be made. This specification is fully editable, and the whole process is iterative, but actually, this looks pretty good.
Copilot can now help us build a plan on how to implement this change. All right, that's a great start, but we must not forget about our documentation. So let's edit the plan, and have Copilot update our readme. And then we can even get Copilot’s help in starting to implement the code for us.
Now, this was just a simple example, but in a large enterprise codebase, there are tens of thousands of files, and dozens of stakeholders involved. And that means meetings. So many meetings. Workspace helps you focus on what you need to change.
And by the way, as a developer, I'm always in control. I can see exactly what changes Copilot is proposing and I can even get a live preview. All right. Let's test out the input. All right.
This looks great. So, I can go back and I can edit my code, in VS Code, or I can submit these changes as a pull request to share with my team. GitHub Copilot, Copilot Extensions, and Copilot Workspace help you stay focused on solving problems and keeping you in the flow. Back to you, Satya. Thank you so much, Neha! I tell you GitHub, Copilot and everything that that ecosystem is doing is just bringing back a lot of fun and a lot of joy back to coding.
And really the thing about staying in that flow, is I think what we all have dreamt for, and dreamt about, and it's coming back. That brings us to the very top of the stack. Microsoft Copilot.
We built Copilot so that you have the ability to tap into the world’s knowledge as well as the knowledge inside of your organization and act on it. Now, Copilot has had a remarkable impact. It's democratizing expertise across organizations. It's having a real cascading effect.
Right. In fact, it reminds me like of the very beginning of the PC era where work, the work artifact and the workflow. We're all changing. And it's just, you know, really having broad enterprise business processes impact.
It's lowering, I always say this, it’s lowering both the floor and raising the ceiling at the same time, for anything any one of us can do. Since no two business processes are the same with Copilot Studio, you now can extend Copilot to be able to customize it, for your business processes and workflows. Today we're introducing Copilot connectors in Copilot Studio, so you can ground Copilot with data from across the Graph, from Power Platform. Fabric, Dataverse, as well you now have all the third-party connectors for SaaS applications from Adobe, Atlassian, ServiceNow, Snowflake and many, many more.
This makes the process of grounding Copilot in first and third-party line of business data. Just a wizard-like experience enabling you to quickly incorporate your own organizational knowledge in data. We're also extending Copilot beyond a personal assistant to become a team assistant. I'm thrilled today to announce Team Copilot. You'll be able to invoke a Team Copilot wherever you collaborate in Teams, right? It can be in Teams, it can be in Loop, it can be Planner and many, many other places. I mean, think about it, right? It can be your meeting facilitator, when you're in Teams, creating agendas, tracking time, taking notes for you.
Or a collaborator writing chats, surfacing the most important information, tracking action items, addressing unresolved issues. And it can even be your project manager, ensuring that every project that you're working on as a team is running smoothly. These capabilities will all come to you all and be available in preview later this year.
And we're not stopping there. With Copilot Studio, anyone can build copilots that have agent capabilities and work on your behalf, and independently, and proactively orchestrate tasks for you. Simply provide your Copilot a job description or choose from one of our pre-made templates and equip it with the necessary knowledge and actions, and Copilot will work in the background and act asynchronously for you. That's I think, one of the key things that's going to really change in the next year where you're going to have Copilots plus agents with this async behavior. You can delegate authority to Copilots to automate long running business processes. Copilot can even ask for help when it encounters situations that he does not know much about and it can’t handle.
And to show you all of this, let's roll the video. Redefine business processes with Copilot Studio. Create copilots that act as agents working independently for you. Simply describe what you want your copilot to do. Easily configure your copilot with the details it needs like instructions, triggers, knowledge and actions. Quickly test your copilot before you deploy, and seamlessly publish across multiple channels.
Watch it use memory for context, reason over user input, and manage long running tasks. Copilot can learn from feedback to improve, and you're always in control. Put copilot to work for you. with Copilot Studio. You know all around this stack is perhaps one of the most important things that we at Microsoft are doing, which is wrapping it with robust security.
You know, security underlies our approach with Copilot, Copilot+PCs, Copilot Stack. We're committed to our Secure Future Initiative. You can see, you will see us make rapid progress across each of the six pillars of SFI, and the core design principles, right? Which is secure by design, secure by default and secure operations.
You'll hear about throughout this conference. In fact, a lot more in Scott's keynote tomorrow, how it underlies everything that we build and everything that we do. So, coming to the close, I want to sort of, you know, there are many announcements that you will hear about at Build, but I want to go back to I think the core of what I think why we chose to be in this industry and why we come to work every day as developers, which is the mission ultimately of empowering every person and every organization. At the end of the day, it's not about innovation, that is only useful for a few. It's about really being able to empower that everyone, And it comes down to you all as developers and builders of this new world.
For us, it's never, never about celebrating tech for tech's sake. It's about celebrating what we can do with technology to create magical experiences that make a real difference in our countries, in our companies, in our communities. Already, this new generation of AI is having an incredible impact, thanks to all of you, the passion you bring and the hard work you put in. And I want to leave you with this one unbelievable example of how you are all building a more accessible world, which means a lot to me using our platform and tools. Thank you all so very much.
Enjoy the rest of Build. Audio description is something that enables me to be able to watch a program or a film and get as much out of it as everybody else who is sighted. A white car drives down a road. Hands on a steering wheel. I see art as a collective good, I think everyone should be able to have access to art.
Audio description really helps me get the full experience. A portrait of a group of 17th century civic guardsmen in Amsterdam. The challenge, though, is that there are limited amounts of audio descriptions being incorporated across media and entertainment. Tech and AI have the potential to bring the blind and low vision community into the fold. So at WPP, we really care passionately about opening up access to content to people in the way that they want to consume it. The tool that I've made is an application that allows you to upload videos, and on the other end, with GPT-4 with Vision and Azure AI services you get your video back with spoken narrations over the top.
Kitchen scene with cat and Hellmann's mayonnaise. This makes audio descriptions cheaper and faster. Our goal is to be able to offer this product as a service for all of our advertisement campaigns.
There are so many artworks in the Rijksmuseum, there are almost a million. To describe ourselves, it would have taken hundreds of years. With AI, we can do this in a matter of hours. The subject is a male, with a reddish beard and mustache, visible brushstrokes that add texture and mood.
The first time I heard audio descriptions it just brought me delight. It was this opportunity of “Oh my gosh, I'm seen.” Through the power of AI we’re able to do things only dreamt about until recently. When we strengthen our access to culture, we strengthen the culture itself, connecting our shared humanity.
2024-05-25 18:59
