How to build a culture of experimentation: optimize for speed at scale | AWS Events
- Hi everyone. Thank you for being here. I've been in your shoes and you're all having this wonderful experience at re:Invent, seeing all of these new things and you're thinking, how on earth am I going to integrate all of this thing in my environment? So I was chief technology and innovation officer at NASA Jet Proportion Laboratory. I saw the cloud coming and was the first to use it as an enterprise.
Learned a lot of lessons and I'm going to share those lessons and hopefully it'll help you. And you're going to look at this and you say, "Well that sounds too simple." Well, the point is, it needs to be simple. If it's too over complicated, you never do it. So have you heard about the culture of innovation that AWS uses and Amazon uses? It's very powerful, but it's difficult to turn a big ship.
So this is a subset that we call culture of experimentation that you can implement in any of your business units. We did it at the Jet Proportion Laboratory in NASA and it worked wonders. So what do we hear from executives this year? I'm part of the enterprise strategists. We talked to about 2000 executives and I've talked over a hundred myself this year. What we're hearing is the priority, the number one priority is the need for speed. How do I go to speed to profitability, speed to market, speed to compliance, speed to change my culture.
And speed can be scary. Our road is not usually the Audubon where you can just floor your Ferrari. Instead, it's a very curvy road.
So what do you do? You slow down? No, because then you're not meeting the need for speed. So instead, what do you do? You add guardrails. And I went too far. So you add guardrails and those guardrails... It actually helps you a lot.
I'm going to go through an example of where we added guardrails for NASA. It's about compliance, it's about security, but it helps you move much, much, much faster. So we're going to attempt to answer these questions today. First, why now? What's different? Well, the need for speed obviously, but there's more to it. And how do you build this culture of experimentation in your enterprise as quickly as possible? And then I'll show you some examples.
And then I was our internal futurist at NASA kind of taking on that role. So I'll show you what the future trends are. And the purpose is, if you can experiment, you can experiment on those trends.
So if you build it, they will come. That's the idea. How many people have been at multiple re:Invents? Raise your hands. Great. Most of you, this is my 13th. I was part of the very first one. And the best part about re:Invent is sharing and learning lessons from each other.
So I'll share mine and hopefully you are networking and learning lots of lessons. So what's different today? How do we need to prepare? Did you realize there's something called a disruption change index and it's showing that change is happening 50 times greater than just a greater pace than just a few years ago. So what does it mean in practice? It means if you do what we used to do, which is to, "I'm going to have a new data strategy, gimme $20 million and I'll have it done in four years."
I mean in four years, I'm not going to be around. The need for speed is much, much faster. I'm curious in here, who is in IT? Raise your hand, raise them proudly.
Okay, good. So I'm going to have a key takeaway here. I was NASA's first CTO for IT at NASA's Jet Proportion Laboratory. And what we're seeing across the world today is the same thing we saw then. IT wants to do it right. The business units wants to move fast, it creates this friction.
So I'll talk about that. So what we're seeing is that experimenting is the key. So Thomas Edison famously learned that 1000 ways not to make a light bulb, but he made a light bulb and it was a great demo. He lit up a street.
Fantastic. Can't do anything with it. What really matters is the use case. So once the use case was there, the infrastructure, the electric lights, the electric motors, you could now move the work to the workers. So the infrastructure is key and the use case is key.
The other thing that happened during there is industrial revolution hurts us today. If you are a company that was successful and you grew large and have been around for a while, you have a challenge. The challenge is the culture. There's silos and how do you innovate within those silos. So that's what we're about to talk about.
It really comes down to risk, personal risk and organizational risk. And how do you make that smaller? Well, don't treat everything as a one-way door. Do you know what a one-way door is? All right, forget one-way doors. Let's talk about two-way doors.
A one-way door is where only the CEO or the executive can make a decision. So it means that we're too slow to make decisions. If we can make everything into a two-way door, which means you can walk through that door, you can come right back and you add no risk. And I'll talk about how to minimize that to make it really, really simple. But that is the key.
Then if the risk is minor, the cost is minor, and you can now experiment with speed. So now if this will help us, how do we build and thrive in this culture of innovation? Well I think Andy Jassy, did you see Andy Jassy on stage yesterday? He's so awesome. Well he was awesome as well when he said this, which is really to experiment, to try lots of experiments, not where you know the outcome because then it's not an experiment, but eliminate or minimize the collateral damage. So that's one piece. The other piece you need for a culture of experimentation and to be successful is that everybody knows your mission. I'll give you an anecdote.
Hopefully you recognize this gentleman, John F. Kennedy. And of course I came from NASA, so he's one of my heroes. He went to NASA in 1962 and he said famously, "We're going to put a man on the moon.
Not because it's easy but because it's hard." Then he's a friendly guy. So he talked to the janitor and he said, "What do you do for NASA?" The janitor could have said, "I make it clean for the engineers so they can do better work." He did not. He said, "I'm helping to put a man on the moon." That's the type of mission we need all of us to know.
And as executives, really in the end you are decision makers and your storytellers telling different stories so that everybody will understand the mission. So if we were to really simplify this, and again I'm going to keep it simple, optimize for speed. Anybody heard of ChatGPT? When it came out, it went sky high. The companies that were differentiated for AI, they were able to pivot and their multiples went sky high. The ones that couldn't, they were kind of left behind. So differentiate, build for speed, and differentiate for the things that matter, the things that are important to our future.
It's quite simple. Decide if it's a one-way door or two-way door and try to make it a two-way door. If it is a two-way door decision where you add no risk, then let the people closest to the problem make the decision and you will see parallel innovation across your enterprise. And then the other thing, we had this big problem. I see it in large enterprises. We're trying to be perfect right at the shoot.
That doesn't work. What it happens is analysis paralysis and you don't get started. So instead, assume that it's not going to be perfect and then iterate. The other thing is "little I" innovation.
If you find... Because one of the problems that we hear all the time is, "How do I innovate while I keep the lights on?" Everybody's busy. How do we get the time? What you can do is if you can find a process, even if it saves 10% of time. Now what if it saves 10% of time for the whole company? Now everybody's got more time to innovate. So the key here is to find things to automate.
And you've heard generative AI once or twice. It's a great way of doing that. Now I'm going to wake you up.
See how awake you are. On three tell me, what does COE stand for? 1, 2, 3. I heard Center of Excellence. That's what it used to be.
What you really need is you need a Center of Excellence, but a small one. I built it for cloud. What happens is you build a Center of Excellence, they think they're excellent and everybody else hates them. And they build their own Center of Excellence. So not only do you have shadow IT, you have shadow COEs.
But you do need a COE. Just a very different. The Center of Engagement, where all of these innovations are coming at you. How do you take advantage of it? How do you benefit? Well be scrappy, be quick and easy. Reduce the risk.
So the membership is cross-functional. You really want at least an IT person, a business person, and cybersecurity for making sure that we're not leaking information. The outcome is business cases that you can try.
How often? Lunch and Learns are great, they are really a good way to spread this. But again, I democratized at JPL, NASA, machine learning, data visualization, internet of things and on and on and on. How did we do it? Say who's interested in this? By the way, do you have the people in your organization that knows internet of things? I ran into it at JPL they said, nope, don't have it. So I said, okay, is anybody doing home automation? Oh yeah, that's easy. Well that became the internet of things group and they did a lot of things.
So you probably have the people perhaps not the skills, but it's an easy thing to solve with a Center of Engagement. So when you think about who to promote, promote the people who can get other people to participate in these new technologies. But it's all about the business case. How do you get time back? The projects that are just focusing on technology, kill 'em. Refocus those people on somebody who's doing a business case that works with a business and now you get a lot of time back. So have you heard about working backwards? Anybody? Okay, it's a great concept. I love this one.
It means that you put the, here's what success looks like. Write a press release. Fast forward a couple of years as if it had happened and it focuses on the problem. Then write frequently asked questions.
What will people ask? What about internally? And it works wonderfully. I'll give you an example of all things compliance in a second. The other thing we learned is are you in love with the solution? Maybe if you have indigestion. So don't fall in love with the problem, with the solution.
Instead fall in love with the problem because with iteration the solution will change. One of the anecdotes here is from Jet Proportion Laboratory. I wanted to see how spacecraft was built because once it's in space, Matt Damon isn't always there to save us.
So how do you know that? So I went to Google Glass, bought 10 of them, came up with 10 use cases, and now you were hands free, you could annotate and it was a great training experience too. Then they shut down. So then we said, okay, we have our problems, they're fantastic. New solution, HoloLens. And now we actually build spacecraft using augmented reality.
You can see if your hand will fit through, et cetera. So falling in love with the problem because it'll persist and don't they make the mistake I made, which is falling in love with my own solution because it's not a good idea. The other thing to do is tap into your employee's ideas. Have you tried this? I talked to a friend who has a son who plays water polo and she said, this is great, we could do this. This would feed like two of them. But the idea is to be scrappy. Try it out.
Now I'm going to give some very simple recipe and you're going to think it's too simple, but it works. First, the words you use are super important. So define what an experiment is.
When I came in as NASA's first CTO, I grew up at JPL, I went into industry, I came back, I thought everybody would absolutely love me. They didn't because I was now IT. So I said, well this is great, we should just pilot it. And everybody goes, ah, we can't do that. I said, why not? So then I remembered what my philosophy processor had said, if you find intelligent people acting stupid, you are doing something wrong.
You're using the same word for different things or different words for the same thing. So I said, okay, when I say pilot, what do you mean? What do you think? Oh, it's pre-production. We've done all the compliance work. We're about to go live. Okay, what if we just do a proof of concept or an experiment? Great, that's easy.
So thinking about how people interpret your words, the story is really key. Then the other one is fast, speed. So you break it down into small use cases, problems, and then give them two weeks and only two weeks to solve it. Those three people I mentioned, security, IT and the business person. And let them create a demo. They write a one pager, what success looks like.
They create a demo. And then what you do is have them record it. Very scrappy. So typically the business person is showing the demo, the IT person is using their smartphone to videotape it, about a one to two minute video annotated.
Then you store it on the website. So when people come and ask later, didn't you do something about internet of things such an AI? Yeah, here it is. Here's the one pager and here is the link to the website.
And the people who did it become heroes because they now get recognition, which is the key. Now do you measure return on investment on these rapid experiments? No because it has the connotation of business case. And if the whole thing is only two weeks, you just kill the momentum. Instead measure return on attention. Now what on earth is that? It's the other key here.
Make up words, make up anecdotes, so that people don't have their previous opinions on it. So return on attention was and is, if you get attention on this demo from the end user community, that's great. Positive attention. Negative is also good because you get attention. If you get positive attention from two user communities, now you will get the investment and you will absolutely measure return on investment. So that's...And you can easily annotate it.
If I got, the way I did it, if you get return on attention, very positive and they want it, that's an eight. Then I said, okay, you like it, you pick up the funding because I don't have anymore. Then if they do, then it's a 10. And then what happens is it's difficult in our culture to bring in new skills and have them integrate. So when you have these different business units working together, they bond. And so now how do you measure that? One way to do it is to see the movement of people from IT into the business unit and vice versa.
And that's a really positive thing because now the whole company will benefit. Now working backwards, what we're seeing from across the world and across history is that new technology is easy to adopt for individuals and small companies, even big companies, but the regulators are slow. So what do you do? Work backwards from the regulators.
One of the things that I came up with, because we had the business need was something called GovCloud. Is anybody use AWS GovCloud in here? A few, thank you. And so I went to the regulators and said, turn out there was seven of them, "Amazon is building this thing. We want to go live as soon as it's available. What do you need me to prove?" Then we built it, went to them, they signed off.
And now we had a package for when the regulators came. By the way, who's in a regulated industry here? Okay, about 15% of you are right. The rest of you are wrong.
We're all regulated, whether it is from a public sector or from our board of directors or the employees that are trying to join us. We have to be compliant. And that's a problem. But that's one way of doing it.
Go and talk to the regulators, whether they're cybersecurity, you risk people, external, and say, "What will it take for you to be live?" Join us in this experiment. Then I mentioned shadow IT. If somebody in the business unit is doing something really good, that's a good thing. Instead of stamping it out, shine the spotlight on it and say, "Can you do it for the whole company?" And now the magic happens, which is the story will spread about, look at this, we built this. Now the whole business can benefit and you will get more and more and more ideas.
So the storytelling is really key and those videos help. So let me give just a few examples of this. Anybody in here old enough to take pictures with old analog pictures? What was the real problem in the end when you pulled it all up? It was that we didn't take enough pictures, it was too risky.
The risk was too big. So anybody interested in soccer? I am hugely. Have you ever seen a bicycle kick? It's the most amazing thing. If you have, you're very fortunate. If not, you see one here.
How did the person take that picture? Well, they just held down the shutter and took thousands of pictures. Now we have electronic pictures. And then they picked the best one and made money. Actually they didn't. Machine learning picked it.
So it's just a use case is so much more powerful now. One of these experimental use cases is Axfood. It's Sweden's biggest grocer.
And anybody in the grocery business in here? The margins are razor thin. So how do you innovate? Well, what they saw is they used generative AI and they were able now to try this thing. And if you see something strange, this is actually milk cartons in Sweden that went out live. This ad campaign was created completely from generative AI. Do you see anything strange on it? You ever see a cow with five legs? So if we were normal marketing, we'd say, "Well that's stupid, get rid of it." But work backwards.
Who's the customer of milk? Who is it? Kids, exactly. They loved it. So usually when you change design, the sales go down for a while before they came up, they didn't on this at all. This is brand new. In fact, not only did it go up, but all the executives across the company said, "This is good." So they're at the beginning of this hockey stick because they have now shown business value of generative AI.
And Bjorn Blomqvist who did it, is now chief AI strategy officer. So it helps. Now here's Jet Proportional Laboratory. Has anybody been there? Good, good. It's a feast for space geeks like myself. It's a fantastic thing.
So if you have a chance, go see it. But it's fun, it's exciting. But it's also an 86-year-old enterprise that was successful, that grew large, that built silos. So I came in to try to figure out how we can use technology. I saw cloud computing as being one of those interesting trends.
We didn't really know what it was, but it looked interesting. Well that led to a lot of new things. The things up in the right hand corner are the things that we invented and AWS built. So you have a very strong voice in this room, very strong, much stronger than you think. The key is to look at the things that are new that on these announcements, and if it's something that you're interested in, be vocal, be fast, try it, be critical. And it's a way where you can influence what's built and how it's built.
So what we found, I mentioned guardrails. Once we came up with GovCloud, people were not afraid of ITAR anymore, International Traffic in Arms Regulations, that was one regulation. So we saw the level of innovation will go sky high because everything was a two-way door.
You could try, it didn't work, no big deal. You didn't have to move a bunch of things out of the data center. The grumpy guy in the middle there, anybody recognized him. Scott Kelly, he's actually on the space station training on the instruments on the space station using augmented reality. So there's all kinds of things you can do, including we realized, so now thinking big, we were going to run out of data center space, just two missions would be a hundred times more than everything NASA had collected before.
So we had no choice but to close all of our data centers and move the processing to the data. And it turned out to be an AWS cloud. The unexpected benefit is now that this data from these satellites is helping to save lives because it predicts floods and droughts in Africa, Australia, and other places.
So once it's in the cloud, you have these unexpected benefits. Anybody know what this is? Anybody? It is looks exactly like Curiosity. It's actually Perseverance, but same idea. It's a rover on Mars. And on the first re:Invent we keynoted putting the first rover on Mars. So that was fun.
But what's interesting about this one, it's great, it's looking for signs of life, but it had an experiment with it. So here's what happens in a big enterprise. The people who did this, it's a two and a half billion dollars rover mission, didn't want this because it would add risk. So we had to prove that it doesn't add risk and it worked and it has flown on Mars.
And now we know we, the next rover is going to be a rover and a helicopter in one. So these experiments really, really pay off and it got a lot of attention because you can now fly over the hills or fly down in craters and communicate the rovers, communicate with the helicopter. So the next exciting thing on space, and I'll leave it is Europa. Europa is a moon of Jupiter and it's the most likely place in our solar system to house life. More water than on Earth, underneath ice sheets.
So we've launched already, NASA has launched a spacecraft and it's going to land in 2030. I'm sorry, it's going to orbit in 2030, looking for landing place. Really exciting, completely built in the cloud. And in order to schedule these big 70 meter to hundred 10 feet antennas, we use quantum computing.
So experimenting, it worked. But we didn't buy a quantum computer, we just used AWS Braket. And now our people know how to use quantum computing. So that's the key here.
So now I'm going to finish with a few of the trends. What's interesting here is the trends you see at home, the technology you use, people bring it to work, especially you executives. People see it to be interesting and now it affects the market and it goes back and forth.
So technology trends are pointless unless people adopt them. So here's what we're seeing from the big picture. On the IT side, the technology side, cloud and generative AI is they go together.
You really can't do generative AI without cloud. But it's a machine learning evolution. It's a technology revolution, but it's also a journey of discovery and that's why that whole experimentation comes in. It's going to increase the amount of data we use. So for you, who raised your hands for IT, how on earth are you going to store all this data? How are you going to make sense out of it? That's one of those think big items to think about.
How do you know if you have a good data strategy? It's actually easy. You take one of these experiments all the way through and then you see, did I know who to ask? Did I have the data I needed? Did I know to ask? If not, let's fix the data strategy for that thin slice. By the way, the same thing for generative AI it's responsible or ethical. So you can build it piece by piece instead of waiting four years. Then the other thing that's really exciting is, I mentioned Internet of Things. It's under-hyped, it pays immediate dividend.
And what we're seeing is the sensors from across the world collect a lot of data. It goes up in the cloud, use whatever machine learning of the day. It's not always going to be generative AI. It's going to be whatever comes next, squeeze it down and then push it out to the sensors and now they can run independently. Nobody knows how to do this yet.
So this is a huge opportunity for business. Now once you've done that, you actually have a digital twin. Digital twin, the word was created in the 60s, but it's still highly relevant.
Now more than ever because once you have this digital twin, you can now have a conversation with it. How would you have a conversation with this complex physical environment in a digital twin? Generative AI. Just ask it questions and you will get answers and you can dive deep. Then what you can do, my father always said, "Son, measure twice, cut once." Well now you can measure a hundred thousand times and deploy once by inserting faults into the digital twin. It's really exciting.
And then it's going to help us deal with these major business challenges. Sustainability, it's probably the number one. We're going to all have to have geopolitical changes. We all have to figure out how to deal with it. It's usually having a backup plan.
If somebody is going to invade another country, they will cut the fiber, use satellites to keep the internet going, et cetera. Supply chain, have a backup. Factories got to move them to somewhere that's not going to flood. So it's thinking big about those things. The rise of the new space economy, it will go from 350 billion today to 1.8 trillion in six years. How does it do that? Because there's a lot of startups working on this and more commercial than government will fund it.
Then put in your key business trends here, thinking big about what could happen. It's really an exciting journey. So I'm going to just very quickly do go into one mega trend here, which is generative AI.
Very quickly because Tom Godden did an awesome talk about generative AI. Did you hear him speak? If you didn't, seek him out, it was awesome. But what we're seeing here is AI is going to reshape our world. So don't fall in love with the solution today. Fall in love with the problem. What is it that we can do? And it's adding empathy.
I'll tell a very quick story here, but it's also about these agents that you hear about. You can automate your entire enterprise without spending four years and allocating a huge group of people. You can get a first draft by using agents as you've heard about.
So the key here is synthetic data. Anybody use the Amazon One? Anybody at all? If you go to football game and you stand in line and waiting for your food and all of a sudden you hear all this cheering and you know you missed that awesome touchdown. Here, you just put your hand and it's now paid and you go back to your seat.
Same thing as Just Walk Out technology. How did they train it? It's actually much, much more effective than, or impressive than, the iPhone or Android that can unlock with your face because it's comparing to one face on the phone and it's you. This is comparing you to millions of people. So they injected synthetic data and this is your opportunity of using more data.
Take your hand, it took a hand, it added wedding rings, bandaids, and much more. And it tested it millions and millions of time with machine learning. And then it's now 99.9999% accurate. It's pretty amazing. And it's an API that you can insert.
So it's the building blocks that Matt Garman talked about in action. Now this one is interesting. What do you think was the average lifespan in 1900? Anybody yell it out if you're brave. 50. Awesome.
It was 47. What was it in the year 2000? 90, I like your thinking. It was 77. What is it going to be in 2100? I have no idea. But what we do know if we're living longer and AI is not this impersonal thing, it's something that's going to help us, be with us the entire journey. I sat next to an elderly man on a plane and he had this friend, Daisy, and great conversation.
I read it. He had medical problems, Daisy gave ideas. Turned out it was generative AI. So he was lonely and now he had a companion that could help him medically. That's where it's going. It's our lifelong companion.
I have grandkids, my daughters raise them. They have cameras that look at them, sound, white sound, et cetera, et cetera. So this AI is with us and it's not this cold impersonal thing.
Now, as leaders, who do you go to to solve technical problems? To your teenagers. So we all need to become technology teenagers. Try it out. Generative AI, try PartyRock and see what you can do.
You will definitely, you can create a generative AI application. You'll impress your peers. Probably not your teenagers, but that's all right. So what do you do as leaders? You future leaders, they imagine what can be done. Then they explore it with all these experiments, then they create it.
Get rid of the things that didn't pay value so you can focus on the things that did. So decisiveness is really key. So really what we're seeing is the whole thing could be summarized in six words, which maybe is what you wish I had done. Think big.
What can possibly happen, that's on you. You've got to just think about what could you do. Then... start small. That's the experiments.
And the ones that pay dividends, you can now scale fast. And the power here is if you put it in cloud native, it'll scale up and down automatically and the risk is minimal. So finally, if you do one thing, do what I did at NASA.
It worked wonders, the challenge between IT and the business. I rebranded IT. I'm seeing other executives do it, to be innovating together.
It's still IT, but a very different IT. It's where you are forcing people to work together and they share the stories, they share the celebration, and you now create this bond and the whole company can move forward. So thank you very, very much for your attention.
2024-12-23 22:08
