Developer Keynote: You can just build things

[Music] heat heat [Music] heat hey heat hey heat [Music] [Music] hey hey [Music] hey heat heat heat up [Music] [Applause] yeah yeah [Music] hey hey hey [Music] hey hey [Music] hey hey hey hey [Music] [Music] [Music] technology is undergoing a seismic shift a shift that requires change at all levels optimizing operations modernizing apps and data training and supporting your people a shift that requires rewiring your business for new levels of agility productivity and innovation let's take this journey together visit our booth to experience how we can help your business [Music] we're sitting ducks waiting for them to infiltrate our defenses millions of tries and they're in uh-oh this one's in the code [Music] [Music] heat [Music] heat [Music] uh hey heat [Music] heat hey hey [Music] [Music] [Music] hey heat heat [Music] heat heat heat [Music] heat hey hey hey [Music] [Music] [Music] hey hey hey hey hey hey [Music] [Music] [Music] [Music] [Music] heat heat [Applause] [Music] i always say start with the business value define the business value and recognize the leverage that you need to unlock that growth for the business it's really about having a digital base camp giving us that innovation and speed that we can build from going forward [Music] what we're going to need for our companies to achieve the next wave of growth is to do something completely different the future doesn't wait how do we transform sustainably scale intelligently navigate uncertainty the path forward isn't always clear but together with Deoid and Google Cloud turn your biggest challenges into breakthrough moments mitigate against climate impacts using geospatial data protect against cyber threats with confidence and reshape the future of business with generative AI let's shape what's next together [Music] [Music] heat heat [Music] [Music] [Music] [Applause] [Music] yeah hey hey [Music] thank you so much for having us we are the meeting tree this is creative technologist feel free to look us up on LinkedIn we hope you have a wonderful day [Music] heat heat [Music] hey goat hey hey hey [Music] heat heat developers [Music] [Music] please welcome to the stage Brad [Music] [Applause] Calder all right hi everyone and welcome to the developer keynote all right what we've been waiting for i'm excited to talk to you today about how Google Cloud is transforming software development we're innovating in three key areas first we're locking your ability to build the next generation of Agentic applications we're empowering you to build agents that are AI powered services these agents can plan reason learn collaborate and act to achieve goals on behalf of users or in concert with other agents now these agents can be incorporated to be a key part of your application and seamlessly integrated with the rest of your cloud stack and at Next we're empowering you to build and deploy these agents with our new framework and products so we're very excited to introduce our agent development kit to help you build your agents our agent engine to help you run your agents and agent space to help users interact with your agents now together these allow you to create applications where agents and users can work together to achieve a common goal second we are helping you be the most productive software engineers with our code assist and cloud assist agents these agents allow you to accelerate development and streamline your cloud operations across your entire software development life cycle then finally all of this is powered by our Gemini models allowing you to leverage Gemini's massive context window and multimodal support this allows seamless integration of video images and voice with your data creating deeply immersive real-time insights and experiences and in today's keynote we're very excited to demonstrate how you can benefit from all of these innovations and to do that I have two very special guests coming to the stage please welcome Richard Schroeder and Stephanie Wong all right thank you Brad it is great to be here again shout out to all of you here in Las Vegas everyone on the live stream watching and the hundreds of people backstage who have done heroic work making this event happen so thank you all of them so what kind of Google things do we have in store today Stephanie yeah well we have heard from so many devs that they want to see the best ways Google Cloud can help them build great software so that's what we're going to showcase today from getting started to scaling and ultimately a sneak peek at the future with some of our newest creations awesome this whole thing is a bunch of demos so we're going to get started on this so set us up Steph what are we about to see first well we are going to show developers how they can build things including agents with Google Cloud and Gemini it starts with the models in fact Gemini still offers developers one of the longest context windows and birectional multimodal capabilities and you can all try out our latest models including Gemini 2.5 Pro in AI Studio so let's see this in action all right to kick things off I would like to welcome to the demo desk Paige Bailey and Logan Kilpatrick [Applause] [Music] hey everyone in the next few minutes we're going to showcase how you can build AI enabled applications with Gemini so where does building with AI start today well for Paige and I it often starts with a simple question can the model actually help solve this problem that we have in our head yep and that's exactly where Google AI Studio comes in it's a place where you can rapidly prototype anything but Logan do you want to tell us what's new before we dive in like that UI refresh from yesterday yeah I think the question is always what's not new in AI Studio so we just shipped a brand new UI which has been awesome we have all the latest Gemini experimental models we have Google search as a tool which works aentically um and even new model capabilities like native image editing and generation which we'll see in a second um but Paige I think you had an idea for an AI app that you potentially wanted to try and see if we could build absolutely today we're going to be multitasking i have been wanting to remodel my kitchen for the longest time and I bet Jim and I can help but uh you know I'm an engineer not a general contractor and I don't even know where to start you know there's all sorts of things from regulations and fixtures electrician work you know I I really feel like AI could help here yeah I think I can so let's use Gemini to break this problem down we'll have Gemini sort of generate a super detailed starter prompt for us we'll have it put together a renovation plan and last we'll see if we can use Gemini to actually visualize what that remodel might look like awesome three steps let's head over to Google AI Studio and get started awesome so you can see here a very very detailed prompt that Gemini has created only from the short description that I've given the model um you can see it's including everything from my current kitchen dimensions to things like budget project scope countertops cabinetry all sorts of details that I would have never even considered that can help us get started with this renovation project you can also see models here off to the right and when you hover over them you can get additional insights into um things like latency what the model might be really great at and also really nice details like costs awesome Paige so let's take this prompt that Gemini actually generated for us we'll copy it we'll jump over into another tab which has a bunch of pictures of your kitchen a picture of the floor plan a bunch of other unstructured data and we'll send that prompt off to Gemini and have it sort of kickstart this renovation for us awesome so I'm running that now um a bunch of things are happening behind the scenes this is a thinking model which means that we get 65,000 tokens of output which are great for long form analyses and detailed docs we've got our super long prompt that's giving uh that's giving a lot of really nice insights and then also this mysterious thinking box so Logan break down what's happening behind the scenes as Gemini is doing all this work yeah so we're using a reasoning model which means that before the model actually gives us this final renovation plan it's trying to think about all the different things that we might need to take into account and as you mentioned before Paige you and I have never done a kitchen renovation before so this is a great example of why reasoning is so powerful um and it'll hopefully come up with a bunch of stuff that you or I never would have been able to awesome so in this thinking section we're seeing an information gathering strategy it's pulling in things like Seattle costs and local regulations it's giving me pros and cons about the specific materials assumptions about the layout just based on this floor plan that we sketched out um and this is still just the thinking box we haven't even gotten to the main model output yet this is amazing um so if we keep scrolling we can start seeing um the major output but you know Logan um Oh wow it's even done a little sketch of the kitchen this is me that's awesome yeah so so uh you know budget is everything so how can I make sure that this uh renovation project would actually fit in my budget yeah great question and so we're using grounding with Google search which means for all the information which the model might not have you know recently trained on like things like up-to-date material costs even like local regulations and codes which can always be changing the model can pull that information in and actually make this not only like a theoretical renovation plan a super practical one that's ground in reality amazing and it looks like Gemini is still cooking you know I would really love to be able to visualize some of these things that it's describing and you mentioned that Gemini might be able to help with that too right yeah i don't think an AI renovation use case would be complete if we couldn't actually see what that renovation is going to look like so let's jump over to your third tab uh and we have a picture of the kitchen and I think this was another super detailed prompt that Gemini actually generated for you yes gemini was able to cook up a really really nice prompt very detailed with all of the different features that I would want to use to reimagine my kitchen nice yeah and so let's hit run again and hopefully we'll see a result in just a few seconds yeah and we're using Gemini 2.0 Flash so hopefully this actually literally only takes a few seconds amazing so I love this um it's been able to take my layout uh you can still see that door off to the left that we had off to the off to the left in the original picture um it's given me a new stove this beautiful Peter green backsplash which is a new word that I learned um and also uh it looks super nice and airy but I don't see a lot of lighting um I'm going to ask for Gemini to add tube globe lights and that's something that it should be able to do as well right yeah this is actually a really great example in practice if you saw that first prompt super deep detailed really verbose uh for the second prompt we can also just ask it to very explicitly change hopefully just one thing which is going to be these lights up in the top yeah I'm also curious to see what kinds it picks it's one of the fun parts of playing with this model i love that that's awesome yeah look at that the beautiful globes amazing and uh this is excellent i I really am excited now to kind of bundle this up send it over to a general contractor and get started yeah I love this and really this is the power of Gemini all coming together from understanding videos to native image generation to grounding real information with Google search these are the kinds of things that you can only build with Gemini absolutely and it all starts in AI Studio from here we can take this prototype we can go ahead and grab an API key and we can scale this out into a full application using the power of Google Cloud and Vert.Ex AI so thank you so much for being my renovation buddy Logan yeah this is a lot of fun Paige yeah next time we're doing your kitchen i hope so [Applause] nice all right that was great feel like that original kitchen has seen some things i don't know it might have makes me want to still DIY my own project though maybe yeah amazing yeah i mean I might legitimately start here with my next home improvement project that was a pretty rad use case and not a way I would have thought of using AI myself okay so what you're saying is you're not just going to keep watching old reruns of this old house i'd have to watch it the first time to see a rerun well Paige can you tell me more about what you use Gemini's long context window for absolutely so long context windows are a gamecher when you really know how to use them um in this example we saw you know some things like photos and images and you know a few different sketches um but with long context you're able to pull in full videos or even full code bases to use in your projects and I use it a lot uh especially when um kind of pulling things in to be able to generate really really detailed um code generation in my favorite IDEs that's great yeah i mean you showed how easy it is to get started which I love but at the same time the more you learn about the features the better the whole experience gets as team keeps building some great things here i'm really excited about some of the things you unlock with native image generation tell me more there yeah you can change floor plans you can change different spaces you know I loved it for my condo renovation but it really gets magical when you can imagine all of the different kinds of projects that you can pull in um I especially loved that we were able to pull in search as a tool and search grounding um and when you're able to add all of these agents into your projects they're great for automating these multi-step processes yeah it's amazing i do love that multimodal reasoning that long context is such a big deal but Steph I am more of a software engineer than a construction engineer and I'm kind of psyched about the coding and Gemini 25 Pro amazing at coding yeah i mean I think we're all kind of happy that you're not a construction engineer i don't know if you'd be standing here the world is a safer place yeah well we're seeing so many developers swarm to this model because of the quality of the coding responses so Logan I got to ask how did this happen yeah it's a great question i know we didn't show it in this demo but all four of us though have been spending a bunch of our time playing around with the new coding capabilities i I think it's it's honestly a story of all these unique Gemini functionalities coming together it's long context it's natively multimodal it's you know bringing in native search i think you can do a bunch of stuff for code use use cases that weren't possible plus innovation in pre-training post- training reasoning so much stuff well thank you so much Logan and Paige thank you great all right here we go it's a good start i love that we're building for builders in this whole show i could not have planned this any better 10 out of 10 no notes but what's next we actually use Gemini here to help us with how we might restyle our place but what could an agent do here how do we do something that would help a business scale this type of thing well agents can automate processes unlock entirely new use cases and improve the way we do the things that we already do every day yeah the million-dollar question is what in the world is an agent how are you defining that yeah we should probably start there well an agent is a service that talks to an AI model to perform a goal-based operation using the tools and contexts that it has and agents can help us accomplish many goals some agents will automate tasks that we need to scale and some agents will help us solve complex problems either autonomously or by asking for human help when needed i see so building an agent is a lot like building a service but it has some of those unique traits you mentioned that push an agent beyond just a classic cron job or a web service no yep that's exactly right got it in this next demo we'll create an agent to help contractors by handling tasks like verifying building codes and looking up permits as input it'll use Paige's ideas and any supporting documents like floor plans the agent's goal is to generate a PDF proposal we can share with Paige to hopefully win her business that sounds like a giant timesaver what is actually the best way to build an agent well you start with Vert.ex AI of course you do vert.ex AI is our endto-end platform for

building and managing AI applications and agents as well as model training and deployment all right since all I apparently have is questions for you now that we are using Vertex AI to build an agent how do we actually get started you're all a great question sir you're lucky I'm here all I got agent development kit it's new open-source model agnostic and supports model context protocol great we're making it easier for developers to build agents let's see how please welcome to the demo stage Dr fran Hkelman [Applause] [Music] thank you Richard and Stephanie i'm here to show you all the goodness of the agent development kit ADK that we just released to build an agent in ADK with Gemini and Vert.xai we need three things we need an instruction we need tools and we need a model the instruction defines the agents goal the tools enable the agent to perform actions beyond a plain LLM vitional function and API calls and the model handles all LLM task and is responsible for calling the tools in ADK we support the model context protocol my favorite protocol mcp creates a standardized structure and format for all the information an LLM needs to process a data request this allows our agent to use tools for retrieval augmented generation also known as rack adk is a Python SDK let's write some code here I am in cloud shell editor using ADK the first thing I need to do before we define an agent is to connect to Gemini in Vertex AI this is very easy to do with the N file i can either configure my Google Cloud project or I can easily access Vertex AI using an API key okay with that established let's look at our agent code remember I said an agent needs instruction tools and a model we define our agents goal using an instruction it's the foundation of any agent we describe in natural language what the agent will do i use Gemini and Vert.ex AI to help create this instruction it's quite detailed because we need to cover a lot of edge cases for this agent the instruction focuses on taking a customer request and creating a PDF proposal and we set the instruction right here in our agent constructor next let's explore tools we're going to add this analyze building codes tool it allows our agent to perform rack by accessing our own private data set for local building codes the analyze building codes function defines the tools arguments and return value and the agent is going to decide on its own when it needs to call this function so it needs to understand what the function does and we use this dock string here using natural language to describe what the tool is doing and this is how the agent knows what the D function does and when to call it the function takes a building feature like plumbing or windows it searches our rack database for local codes and then it returns a summary clear dark strings are vital they guide the AI model in selecting the right tool to perform rack we need to retrieve information from outside the agent for this I used the model context protocol server from Google's MCP toolbox for databases which we contributed to open source and I deployed it to this endpoint here the analyze building codes function uses rack to ground the agents responses in data from our database with this we've defined a tool for our agent to accomplish its goal okay since we're building an AI agent we need an AI model from Vert.ex AI adk is model agnostic so I could use models like Llama or Claude but I'm using Gemini 2.5 and the model is a key part of the agent because the model handles the execution of the tools we defined and it produces the output we're looking for a proposal for pages remodel in essence building an agent in ADK boils down to just three things instructions tools and a model okay time to test the agent let's open the terminal adk comes with a deaf UI for local development you call it by running ADK web okay open localhost let's select our agent the defi handles multimodal inputs like images for us so I'm uploading pages ideas and her floor plan please create a proposal we will integrate this UI with Gemini code assist and its supported idees later this year okay gave us an answer wow look at this it created this whole professional PDF what a massive timesaver agent defit is public today i want you all to build something with agents on Gemini that makes your life easier thanks so much [Applause] great job Fran awesome stuff yeah that was terrific i love seeing that yeah I mean it's it's really wild to see how easily Fran could take some of the fun ideas from Paige and Logan's demo and make an enterprise class agent from them i mean it's not very hard to see how this could be useful for lots of other tasks and businesses anything that requires just a lot of reasoning no oh there's so many things we can build with agents and they really unlock a whole new level of automation i thought agent development kit looked pretty cool right you all like that i think that's pretty cool yeah there we go a round of applause for agent development kit was there anything unique though that you had to think about differently when building with it i know you hit on those three things that make up an agent but was there anything otherwise you had to think about i mean at the heart it's just code prompt engineering is really important so be really explicit with your dog strings and your instructions that's good advice for everybody all right thank you very much Fran terrific job all right Richard well let's shift a little bit to talk about how developers are building even more with our products and frameworks yeah yeah yeah let's talk about one of my personal favorites or so I'm told Vert.Ex AI agent engine which was recently made generally available it was an agent engine makes it easy for you to deploy and run agents built on any agent framework it simplifies the deployment process so that you can focus on your agents code and provides you enterprisegrade security controls production grade monitoring and logging and even evaluation and quality frameworks and Richard what else do we have for agent discovery and sharing you want more it's all agents to make life easier for devs we actually have something pretty awesome called Agent Space you've been hearing about that this week it's our hub for agents across a company as you saw in the demo in the main keynote yesterday you can build these no code agents directly in agent space and you can register agents built with the agent development kit and make them available here that's good we can also use this to share agents that are built by developers and can be surfaced to all the employees within their own company or to all companies that use agent space from the cloud marketplace and some of the best part is that this supports thirdparty models and agents and provides a common surface for access control across all of your users so you might have all heard about teams of agents working together these combination of agents are able to take on complex tasks with both high agency and high automation as we're about to see next so let's see some cool new things we have for building multi- aent systems in Vertex AI managing agent systems at scale and some of our newest tech to help with debugging and fixing our agent services let's make some noise and welcome to the stage Dr aberami Sukumaran [Music] thanks Stephanie thanks Richard hi [Music] everybody complex processes require more than one agent so we're going to create a system of multiple specialized agents and go through the process of deploying and debugging a multi- aent system now we've already got the construction proposal agent that generates client proposals next we'll add two more agents one for permits and compliance and another for ordering and delivering materials both built with Vert.ex AI

using agent development kit with Gemini we'll need these three agents to work seamlessly together as a single agent that can handle the end-to-end process let's check out the code to see how we use ADK that is agent development kit to build and orchestrate a multi- aent system we start out by defining our root agent so this can call one or more sub aents based on the goal first we'll start with our instructions this is very similar to how we do with a single agent but since this is a multi- aent system we'll give additional instructions that define how these additional sub aents will be used and routed to all in natural language and define any additional instructions or dependencies that are required between these sub aents right here next we'll need to declare these agents as sub aents this is similar to how we declare tools here you see three sub aents highlighted proposal permits and ordering agent all right when my agent is ready I can deploy it directly from ADK to Vert.ex AI agent engine it is a fully managed agent runtime that supports many agent frameworks including ADK here we have the code snippet that allows me to deploy my multi- aent system to run on agent engine once I execute this in a few minutes our agent will be deployed and available to call and share across apps and users the call to create returns an endpoint which looks something like this perfect now that our agent is deployed and running let's test it out in a place where I want to make it available to other users which is you heard it agent space here I am at the agent space console where we can see all of the agents available to me in the left pane right here I'm going to go to renovation agent which I just deployed so to start off let me start uploading the proposal document that was generated in the previous demo and I'll start interacting with the multi- aent system i'll ask it to kick off permits process the renovation agent will route our request to the right sub agent based on the instructions we have specified great it's already created a checklist of permits now let's say I want to ask it something related to ordering i'll ask it to check order status for some of the materials that I've previously ordered this should call oh snap all right looks like my renovation agent has a bug in it well since I've deployed my multi- aent system to agent engine and since it uses tools from this Google cloud project I'm going to go to cloud logging to see if I have any information that helps me debug let's go over to cloud logging logs explorer there it is yeah that is why I have a bunch of red all right that's a long wall of exception texts i see investigate i click that and create investigation this is a new feature in cloud assist called cloud investigations which helps diagnose problems with infrastructure and even issues in your code once the investigation is complete I'll click view investigation and the report is open i have seven relevant observations in the report and a hypothesis section let's see what the observations are all right it has identified that a posgress SQL column not found and that's status and it's also identified the column name that is status and also it takes me to the source code to debug it and it has a direct link to make a Gemini suggested code edit and make the fix i can navigate over so in the hypothesis section I also have the overview of the problem i have recommended fixes along with a direct link to Gemini suggested code edit which will directly allow me to make the fix let's navigate over here we are as you can see right here it has the comparison and a sidebyside view of the current buggy version of the code which has the wrong column referenced and on the right hand side I have the fix to it that is Gemini suggested edit and the right column that I was supposed to reference was order status i have incorrectly referenced it as status i like the fix i'm going to accept code suggestion and save and redeploy all right in about a couple of minutes it'll be deployed how cool is it that I saved so much of my debugging time with cloud investigations [Applause] now that I fixed our error once it's deployed in about a minute I could go and retest the agent and share it out to users without having to make any change in the multi- aent system itself so building sharing and even debugging complex agent systems got so easy with agent development kit vertex AI and agent space along with cloud assist investigations right so excited to see what you'll all build thank you [Applause] wow that was uh amazing tech influencers are just like us their code doesn't work either that's amazing i mean it's easy to see how the scale and number of agents can really grow right bringing together agents seems like it could be kind of complex but you actually made it look pretty easy so what did you learn using agent development kit that can help all devs build multi- aent systems right i personally have three key takeaways from building this multi- aent system with Vert.ex AI first it's important to have a proper understanding and reasoning uh for the specialization of each of the sub sub aents second uh do you know like what is the root agent how are you going to connect and route between these and make sense of all the responses and u gather the results and um finally I learned I could do multiple types of agent routing handle context state and do a lot more with agent development kit especially when I have to do complex flow control well you absolutely had a great example Obby of a local agents working together yeah it was great and you know to make it easier to connect any agents together we actually just launched that agentto agent protocol just like MCP standardizes how agents connect to and use tools a toa makes it easier to discover and connect agents especially when those agents are from different ecosystems or vendors it's all open source we're building this with over 50 other companies so you can build agents on any of those platforms and those agents can communicate and collaborate to help you achieve the goal that's really cool i showed how we can have a bunch of agents built on Vert.ex AI work

together to accomplish a goal but in the future I know we'll want to use agents that are built with different frameworks and different vendors and I'm really thrilled and excited that Google is actually working with so many others to create a standard for exactly that yeah now I know we're obsessed with agents but you did sneak something in there about cloud investigations and why do the developers in this room actually care about this what does this really help us do it's hard enough to build systems that orchestrate complex agents and services developers shouldn't have to sit around debugging and all these multiple dependencies getting to the logs going through the code and all of this can take a lot of time and resources that developers typically don't have so that's where cloud investigations can save the day it helps identify and fix code across the services of a multi- aent system so we can quickly get an app back up and running all right well everyone give it up for Abby thank you amazing that was so great so so far we've shown you lots of ways to just build things with Google and AI but we also do love the ecosystem and devs loves using what's new and exciting and frankly what's the most useful for a specific job it's the nature of how we build modern software gemini is actually so powerful in Google Cloud and our tools next we're going to show you how to take advantage of Gemini from your other favorite IDEs and tools all right wait hold on hold on Richard yeah let's not forget the other way we're enabling developer choice what else did I forget vert.ex AI model garden hello going to the shed that's terrible it's enabled over 200 models to date to be deployable for developers worldwide you can choose what models work best for each task got it with that let us see how building with Google AI gives you unprecedented choice in both IDE and the models please welcome Debbie Cabrera to the stage [Applause] thanks Richard how's everybody doing can I get a woo wow okay that was great you know what else is great google is empowering developers to use the tools and models that you like best to build amazing apps first we'll showcase how you can use Gemini in your IDE of choice and then how you can bring your model of choice to Google Cloud for your apps let's start with our IDE experiences and pretend that we're building a new service that'll be used for our agents to help with budgets we'll use three popular ideides that are enabled to use Gemini first up have you heard of Vibe coding people are doing a lot of it and more with Windserve a new and popular IDE with a streamlined experience that focuses on agentic coding here we can chat with Gemini 2.5 Pro to start creating code powered by AI you can ask something like build me a Java micros service app with sample services for budgeting the model will then iterate with you a lot over multiple steps it might prompt you for commands to run in the terminal and even create some files on your behalf it's really simple to go from zero to template and continue building from there in Windsurf huh looks like I'll go to my backup machine because Gemini got a little tired over here i want to show you what it built to get there I'm going to take the escalator and I'll take the elevator up okay looks like it had a little bit more energy on this side there you go you can see how great it is okay now let's hop over to cursor to see how you can start making some specific services using Gemini as the model of choice for coding cursor excels in codebased analysis and reasoning so keeping with our budgets example there is a lot of code here but this is just a budget controller Java file with an existing controller let's say we'd like to add some input validation so we know that everything looks good before we go ahead and process it using cursor's inline option with Gemini as a selected model we can ask to add input validation and hit submit and there we go you can see the highlighted text that was automatically generated by Gemini looks like it's added a few lines in case there's a bad request and all of this is done right within the existing file recognizing any differences needed and even adding changes based on the type of entity that you need the validation for now since we're already working in Java let's see what we can do in Intelligj and the Jetrains family of idees in Intelligj we're connected to Gemini via GitHub Copilot we're using the same codebase i have my code on the left and on the right I've given Copilot some context for different files for my budget controllers we can go ahead and ask in the chat to add unit tests and I speak Spanish so I'm actually going to ask in Spanish let's see oh give it a second oh Gemini okay looks like it's proposed a new file and maybe some modifications to a current file I have and once this is completed I can choose what I would like to accept and input into my code and the most exciting part is that all of these IDEs now support the Gemini 2.5 family of models out of the box and these are just three tools Windserve Cursor and Intelligj and C-Pilot but we're enabling devs to use Gemini wherever it suits you best even in tools like Visual Studio Code Tab 9 Cognition and Ader okay now let's talk about running models now my favorite model is Gemini of course yours too right right yeah but there are so many great models out there and some are particularly well suited for specific tasks or use cases so behold Vertex AI's Model Garden you can connect to some of the most popular models or even bring your own from registries like HuggingFace model Garden supports the latest and greatest models across creators including Llama Gemma 3 Anthropic and Mestral so let's check out the Llama 3.3 model from Meta it's offered as a service via Model Garden here we can test the model to see what the response looks like by asking something like "What capabilities can you offer for designing renovation budgets?" And there we go there are the results we can also use the results we get and compare with other models that we may be considering and we can do this without having to spin up our own infrastructure or pre-allocate expensive GPUs or if you decide to deploy this you can deploy to your own endpoint or a cluster so let's implement a model from model garden in our application code here we're in Visual Studio Code and in this example we've been using Claude and it's great but our apps run on Google Cloud and the model's running somewhere else we want to run our model colllocated with our app to minimize latency and get the benefit of a simpler security surface so with two small code changes we can use the same cloud 3.7 model from model garden we'll use anthropics library and all we need to do is change how we initialize the model here we have the project ID and region instead of API key and then change the model's actual name and that's it then we're done and we're set up to use the model garden version instead and we can use other models via the Vert.Ex AI SDK

to do something similar we're striving to meet developers where you are your team can build great apps using Gemini as your IDE of choice or you can use Vertex AI model garden to call your model of choice no matter what you use we're excited to see what you come up with gracias [Applause] that was pretty great those demos were moe i don't I don't speak Spanish i don't I I think we can talk um that's something give it up for Debbie that was great um yeah wild stuff two things are true of developers we are very particular about our tools but we're also excited to try new things i'm excited by the fact that we've taken such an open approach here and you can use Gemini from so many different tools so developers can keep working in the tools that they love yeah so I don't know about you Richard but personally I've been playing around with Cursor and it's great but I love that we're making Gemini available for devs wherever they are and for getting access to the latest models in Vert.Ex AI Model Garden yeah and I'm excited to share our latest Gemini model 2.5 Pro is now available in Gemini Code Assist for individuals you can use it right now not right now wait till after but we've brought even more Gemini to our own developer surfaces android Studio now is supported with Gemini Code Assist it's in preview this adds Android specific AI capabilities to Gemini code assist and Gemini in Firebase provides complete AI assistance in the brand new Firebase Studio so we know AI is transforming the way that we build software but the impact of AI extends far beyond the lines of code that's right it is changing how we experience the world around us like playing or even watching your favorite sport for me that is Major League Baseball go Padres's okay wait hold on so you're saying that AI can finally teach you how to throw a knuckle ball i'm hopeless 0% chance nothing can help me but it can revolutionize the way that fans analyze player performance and the very best part anyone who is passionate enough to change the game with AI can do that with Google Cloud well that's why we partner with MLB who's using Google Cloud to process 25 million data points per game to compute previously inconceivable stats using Gemini and Vertex AI MLB can compare live plays with its massive Statcast data set in real time and find fresh insights we wanted to see what the community could build with this data and our AI so together we created the Google Cloud MLB hackathon yeah we had tons of submissions but we are thrilled to introduce the grand prize winner here on stage who will show you how he used Gemini and Vert.Ex AI to actually analyze analyze baseball pitcher performance so everyone give a really warm welcome to our winner Jake Debatista [Applause] [Music] hey everyone i've always been impressed by how good Mo's motion capture and analytics are it seems like every detail of every play can now be measured using high-speed cameras but not everyone has access to that see back in college I was a shot putter and I used to spend hours watching videos of myself throwing and searching for errors in my technique there had to be a better way to analyze a throw without the need for high-speed cameras to capture biomechanical data so I built a fully customizable prompt generator for analyzing a pitch it only took me one week to create this using Gemini API and Google Cloud let me show you how I did it since every pitcher has different preferences and skill levels Gemini would need to be able to adapt its analysis depending on the pitch let's start with an experienced pitcher with lots of historical data i'll analyze a video of one of the greatest left-handed baseball pitchers of all time Clayton Kershaw i'll analyze his pitching mechanics to take a look at the last pitch he threw in the seventh inning of a nearperfect game this will show how far off his ideal performance he was at that point in the game cool so we start by pre-processing the video using Open CV a computer vision library and then storing it in Google Cloud from there selections are made such as pitch type and game state to pull in MLB data including Kershaw's profile into our prompt each selection here helps to generate a unique set of system instructions and prompt details tailored to the pitch being analyzed finally this is all sent to the Gemini API the model indicates Kershaw was throwing his signature curveball with nearly no deviation from his ideal but this isn't just for the to pros this is a tool for all of us that's why I added an amateur mode which uses parameters for less experienced players our co-host Richard loves baseball and threw a pitch a few weeks ago let's see how he did i'm scared are you sure you want to show this to thousands of people Richard no not at all all right lining up and it's not that bad well you're no curse Shaw i've already generated the results in another tab let's go over there and see how we did here we go oh okay the major leagues for you Richard i know i'm better at this job i think so yeah maybe don't quit your day job hey so you have some potential look at what it says he has to tighten up his arm a little bit and add some leg drive to maximize his power here Gemini uses a different prompting framework for less skilled pitchers after a little prompt engineering on my end it's able to quickly adapt from a professional model like Kershaw to an amateur like Richard in a single click altering the grading rubric accordingly and what's cool about Gemini is with simple prompts it can process multiple frames simultaneously allowing me to analyze the entire motion not just individual snapshots giving a much richer understanding of how each part of the pitch contributes to the final outcome this essentially worked out of the box meaning I didn't need to implement a custom model or build overly complex data sets as an athlete and front-end developer I was shocked that I could build a computer vision application using Gemini by myself in just one week this was built with pitching prospects in mind so coaches and athletes at any level could help players without the need for high resolution cameras now it's possible for anyone to have a personalized coach right in their pocket thank you awesome all right give it up for Jake congrats again on your hackathon win we had a ton of submissions that stood out if I knew he was going to talk like that he wouldn't have won um Stephanie what is the takeaway other than I probably need to work a little bit on my pitching performance i'm I'm glad you got that out of it could you have imagined though using generative AI for this type of analysis a few years ago i I love that this is valuable anywhere that visually analyzing a process makes sense i mean you think about quality control in manufacturing process optimization on some large production line even troubleshooting parts inside a generator that's amazing capabilities there yeah so the list is pretty endless and this is something that you just couldn't do well at scale before AI yeah i'm still thinking about sports and AI use cases it's a problem but even within Google we have that AI basketball coach right here at Next have you tried that well I tried it yesterday here we go where it actually coached me on elbow position height and knee bend so basically everything you're roasting my pitching when that's what we're looking at good okay fine i deserve that i deserve that uh the recent work we did with our CEO with CEO Jeremy Bloom and his team at the Winner X Games to build an AI commentator and score analyzer that was awesome yeah it was so that work added another dimension to what Jake talked about which is essentially that Gemini can see and help make sense of information that isn't immediately apparent to the human eye in just eight weeks a small team of cloud developers built an AI commentator that could add an entirely new element to the fan experience for men's Super Pipe ai scoring and analysis for the first time ever during a liveaction sports event that's so cool cool and correct me if I'm wrong i think Gemini accurately predicted the top three winners based on just analyzing their practice runs yeah exactly right and XG Games is super interested in this idea of AI as an unbiasing mechanism pairing AI insights with human expertise to help remove bias from competitions these things can watch longer than we can we might miss something for a second but the AI doesn't which is great yeah it's true i also love this idea that Jake touched on of AI as a coach like how can you use vast amounts of nuance data to improve performance i feel like we're going to see a lot of momentum in this space next year no doubt it's great that AI is making things faster less expensive but I'm actually really excited about the ability to do entirely new things that weren't possible before where is this all going what should we all be preparing for so far we've seen what's next what's after next well let's take a look at two areas where our industry is changing even further we're giving you interfaces that are more powerful responsible agents that are more helpful and a cloud platform that helps you orchestrate it all nowhere else you get all that yep ai is showing in more places not just your IDE and notebook right think of agents as teammates that partner with us in all of the different places that we work we're we'll show you how an agent powered by Google AI partners with you to do deep data analysis using BigQuery Vert.Ex AI and

Collab this will make getting insights and answers from your enterprise data dramatically easier and will make you a hero to your business stakeholders so please give it up for Jeff Nelson here we go Jeff [Applause] thank you Stephanie who else here loves data all right all right me too i've worked with data for over a decade and know firsthand just how hard it is to turn that raw data into something useful today I'll show how our new data science agent helps us turn that raw data into a data app this app will help sales managers access personalized forecasts at our construction company i'll start in a BigQuery notebook that's powered by Collab to build our forecast to do so I'm going to check out my product sales table with a little bit of SQL code that I pasted in from the clipboard we can see the table here and if you'll notice there's a DF beside the input df stands for data frame and that's BigQuery loading its results into a Python dataf frame python allows us to use libraries like big frames to execute my code over tables of any size so I'll use it to paste in a little bit more Python to drop a few columns we don't need for our analysis and I'll write one more cell to aggregate total sales by fields like order date and customer state and this just helps gives us some metrics for the rest of our forecast this tabular view here is great but seeing the data is even better so when I switch over to the chart view I can start to spot interesting patterns or potential issues before going deeper next I'll use this data to forecast sales with the Gemini data science agent that's built into this notebook all I need to do is click ask agent and I'll input a prompt to generate a sales forecast from our input table from here on out all code is generated and executed by the Gemini data science agent this is a collaborative process so the data science agent acts kind of like my peer we can go back and forth in simple natural language in this chat box and you'll also notice that the agent uses Spark for feature engineering this is only possible because of our new serverless Spark engine in BigQuery so switching between SQL Spark and Python is now easy and allows developers like you and me to use the right tool for the job that's something I think we could all get used to so here the Spark code reads from our input table applies some light transformations and writes to a transform table next to build the forecast our agent uses a new Google Foundation model called Times FM it's meant for forecasting and it's accessible directly in BigQuery unlike traditional models this one's pre-trained on massive time series data sets so I can get forecasts simply by inputting my data let's take a quick look at the output i'll exit out of the data science agent and we can see for every product ID and date combination we now have a forecasted value as well as lower and upper bounds for a 95% confidence interval but our chart view helps us see this even better and this is the kind of thing I want to share with my sales managers but for that I'm going to need a data app so that all of my assets are packaged up and easily sharable but here's a secret i don't really like building apps so watch this right with this notebook I can click create data app select the cells I'm interested in publishing in this case it'll just be the visualization i can click next and I'll leave the default inputs here when I click create BigQuery packages everything up and gives me a link that's external that means this forecast is now a data app that's accessible to everyone so sales managers can now directly access this app they can input their own parameters and get personalized forecasts without needing to know any data science at all now that's powerful so I'm psyched about this too so we easily transformed raw data into a fully deployed forecasting data app made possible by this data science agent but wait there's more we're also launching specialized agents for data engineers data analysts and business users in preview and the data science agent you just saw integrated in BigQuery notebooks is coming soon you can also get started in Collab today so scan the QR code on the screen and start building i can't wait to see what you create thank you [Applause] that was amazing did you Did you hear him say he does not like building apps congratulate Jeff on his first and last time on the main stage keynote congratulations what's going on there um but that was a ton to see in a short amount of time that was super duper impressive yeah I mean I I know the data science agent is really taking off based on the reaction here it's true and uh I'm excited to see our next set of agents for data launching this year yeah I like this concept of agents as teammates it's not just about using a single call to a model and that's it it's more than that as devs we're more empowered with agents to do ongoing and complex analyses with all of our data yeah well Richard what else do we have for everyone today we got one more big demo we want to give you a sneak peek into what we really see as the future of software development we've got some amazing features coming later this year that will make building software so much faster and easier by using a software engineering agent what else to further boost productivity to show us how all this works get excited for my friend Scott Densore [Music] [Applause] hey everyone I want to walk you through an early view of how you'll build software using agents designed specifically to help developers this functionality is coming to you later this year gemini Code Assist provides an extra pair of coding hands to help you create applications and remove repetitive and mundane tasks so you can focus on the fun stuff this is the Gemini Kodasis combon board we're moving beyond editors to a new way to develop software with agents the combon board will let you orchestrate agents to help you in all aspects of the software development life cycle it is includes something we're calling a backpack that holds all your engineering context style guides security policies formatting preferences even your previous feedback so Code Assist can be a more effective coding partner all right let's start here we've got a technical design doc for a Java migration a task I think we'd all like to hand off so we'll make a comment and assign the migration to code assist directly from our Google doc when it has questions those will get added to the comment thread where we can respond directly this new task will show up on the combine board so we can keep track of its progress google Docs isn't the only place you might want to start a new engineering task we can ask code assist or send so code assist a message asking it to fix this continuous integration build failure directly from our team chat room or you could assign it a bug directly from your bug tracker in fact why even assign bugs manually when we can just ask codeits to do it it can keep an eye on our repo to perform bug triage do root cause analysis and fix issues automatically we can ask it to do code reviews on all incoming pull requests leaving great actionable feedback much better than I get from Richard back nobody puts baby in a backpack out of here and of course you'll see all those assigned tasks bugs and each new issue as they arrive right in the combine board while we're here let's start a new project we'll take a product requirement document for an earthquake monitoring web app and ask codeist to produce a pro prototype okay that looks good we're going to fast forward through this a bit and then we're going to preview that web app right from within the conbon board you can see how this now becomes our new development loop i can tell Kotus the changes I want and let it take another pass and repeat that until I'm happy with what I see or I could decide decide I want to just dive into the code right in my IDE so you know it's been a few weeks since I started this project i think Richard been doing some check-ins and he's likes to do a lot of talking but not a lot of coding or playing baseball so let's see what he's done oh richard of course he left me some to-dos so I'm going to hand this off to code assist and then I'm going to look into his code access oh don't do me like that oh you just wait i think that was at build failure it was you earlier wasn't it probably probably all right so designed to tackle all kinds of developer tasks code assist and comb on board will extend your reach and allow you to do more as a developer and turn even more of your ideas into reality i invite all of you to check out the app dev spotlight right after the keynote where Ryan Jab will show you some of these great new features in much more detail and tell you how to sign up for the preview thank you there you go give it up for my uh former friend Scott that's uh like do I need to get in between you two so rough well that really felt like the future of software development right i love the idea that it will let me spend the vast majority of my development time focused on solving interesting and important problems and less time on the boring stuff like writing tests and CRUD code you're doing a lot of CRUD code right now i won't be yeah i mean building a prototype or MVP like we're showing here

2025-04-13 19:07

Other news