All. Right hello everyone welcome. To. Next thanks for coming out my, name is Bret McGowen I'm a developer advocate on the Google cloud platform focusing. On all things serverless, I'm super. Excited to be joined today by John Hurt's CTO of smart parking you're, gonna hear a little bit. About smart, parking later they are doing some really interesting industrial-grade. IOT, and serverless around. Smart cities and parking that I think you'll find super, super interesting. So. If you want to follow me on Twitter I'm at at Brett mcg. And you can ask me all of your Google cloud server. List questions. But. As, we get started I want to talk a little bit about what is the philosophy or what are the principles, of server list there's no real there's no official, sort, of community definition, of what it means to be service but, these are the three tenets that I like to think of so, serverless means that there's invisible, infrastructure, you're, running. Your web server or you're running your compute code or some query and kind, of all the plumbing, underneath it is abstracted. Away from you so things like networking and, operating, system updates. Attaching. Hard drives, patching. Dependent. Libraries these, are all sort of taken care of you so you just worry about your, code, scaling. It's, part of infrastructure you scale up but also you scale down so, the idea is if, you're, not actually running, a query or doing. Some compute you don't pay anything so. This, kind of encompasses. A lot of pieces. Of server lists and generally, they're considered. To be two sort of types of server lists there's, what a lot of people think of compute, so, these are typically, these little functions, these functions, as a service, that, respond, when some event happens in the cloud write a file gets uploaded and a, function runs so these. Compute. Tools. In a server list perspective, are abstracting. Away the infrastructure. That's running your applications, so the actual, plumbing. Underneath it right the the virtual machines the operating system and so forth but there's a whole other category of service these so-called backend as a service and the idea with back-end is a service it's not abstracting. The infrastructure. Of your application, but actual component. Of your application, so things, that make logical cohesive. Sense in your application things like authentication things. Like databases, so, the classic example that I always use is, email, so, if you really, wanted to set up your your home your own cluster. Of SMTP. And pop servers and work, with ISPs, to get whitelisted, so you're all your email doesn't get message marked, as spam like sure you can do that but sending email is probably not the most important, thing that you're doing as.
A Project or as an as an organization, so sign, up for something like SendGrid or mailgun and just outsource it right you just consume it as an api using them the data and they take care of all of it and there's a whole suite, of these. Kinds, of back-end as a service so things. Like databases, texting. Payment, and so forth and, not, these aren't all necessarily Google, Cloud ones but we actually have a whole bunch so I'm not gonna dive into all of these but this is just a slide to give you a sense that we, have these databases machine, learning and data, processing, server list tools so these are all things that, spring. To life when they're needed they do some execution, and then they go to sleep when they're done and they scale up and down very, tightly with your actual usage. So. Let's dig into a little bit about so, the topic of this is building service systems so putting. All of these pieces together. So some things to think about as you're building a service system so, we're gonna play serverless pros and cons so. A probe server list it your. Resources scale from 0 to potentially. Plan its scale and that's awesome that's a huge Pro but. A con is that you scale from 0 to plan. Its scale and why, is that a problem is because if you are touching systems that don't scale up you can crush them so if, you have some poor little my sequel database that, your, say cloud functions are talking to and they get all of these requests, and they scale up nicely and they absorb all of this traffic that's awesome, but then they slam, this database that could. Be a big problem right so it's both a pro and a con so. One. Thing you can do which is sort of the purpose of this talk you could just go all in with serverless get a surrealist database, that scales up with you that's nice not, always practical. Especially. Have some sort of legacy, or dependent, systems so. Another thing you can do is has some kind of cue and make, sure that you're not exceeding, the, quota, or the connection limit to your database or whatever your dependent system is and. Then finally you can maybe throttle. Your server list computer so you could say it. For some tools you can say don't don't. Have more than X number of instances or don't have a number of X min X, amount of connections you can let the your server, list provider take care of that for you, all. Right let's place again some, server list pros and cons pro. Resources. Scale from zero to planet scale cons. Resources. Scale from zero to planet, scale are you sensing a pattern here so why, is it a problem this time well. We talked about crushing your dependent systems but, what about crushing your wallet, so.
The Idea is. Actually. If you get a ton, of traffic it'll, absorb it quite nicely, but. You pay for it and think about so. So there's a couple of solutions here one, and this is my favorite get, a planet scale wallet and pony, up just, pay for it I love, this because my, salary comes from Google cloud platform so, I would love for you all to have huge. Huge. Bills so I can go cavorting, about the world but. That's not an actual solution right we don't really, want to do that. So. One, thing you probably want to do is be really, careful and be really aware of what is happening with, billing, in your application, so, we have these programmatic, bad budget, notifications. That you can set sort of budget thresholds. As your. Project. Is hits, them so something like 20% eighty percent fifty percent or whatever and. It actually using. Server lists you can publish to publish in. That does some some. Action so here's what the UI looks like you, can say alright, my. Budget for the month is 20 bucks and when I hit half of that send, some message to pub/sub and, serverless. Lee your cloud function will spin up and you, can talk to say whatever. Appropriate. Good. Google cloud platform API, to shut down some virtual machines it's audio your database or just make someone aware that, I hey something. Is going on maybe it's maybe, it's good maybe you actually just genuinely are getting a lot of traffic maybe. Someone is trying to ddos you there's a whole lot of reasons why this might, happen so it's, really important, to not, find out at the end of the month that, this awesome serverless system you built gracefully. Scaled up to infinity, and that's how many dollars are, waiting. In the other end of that bill. So. There's, a couple of considerations putting, together these systems I'm gonna kind, of at a high level touch, on some of these products and then we'll talk about how to put them all together so. Server lists for apps and compute, so, here are kind of the three broad, categories I would say so, we've got App Engine standard which is. Very. Service it, has been for a long time in the sense that it scales up and scales, down to zero so the resources go to sleep when you're not actually using them and you can run your whole app think of like your your web app or your entire web app will scale up and scale down it's, really nice but. Sort of the the column to that is your whole app scales up and down based, on whatever the highest traffic piece of your app is so, you have some piece that never gets used or very rarely gets used it's, gonna it's gonna it's going to scale your whole application up, and down based on the highest use, resource, cloud, functions much more granular so, if a function gets executed once, a month it, will literally get executed, once a month and if it runs for 300. Milliseconds, you pay for 300. Milliseconds, so really tightly granular. Units. Of compute and sort of where. This comes into play we. Would talk about building these service, systems and stitching, things together as, cloud functions respond to events in the cloud so file upload events messaging events on pub/sub and we'll take a look at those later, so these are the things that naturally. Lend themselves to gluing, different pieces of a cloud together, and. Then finally we have firebase, which is a whole suite of tools including a database particularly. If you're building mobile apps web. Apps or you're doing. Embedded. Devices and say. Yeah. Do C++. SDK if you're doing embedded devices or. Unity, if you're doing games you have SDKs, for that so here's kind of a visual representation of how cloud functions, can kind of glue these pieces together so you've got these triggering, events file upload and message that pops up an HTTP, request comes, in and it activates this cloud function and it automatically, we take care of running it and scaling.
It In response to all these events and thanks. To things like the, client, libraries. Your. Cloud function is automatically, authenticated, by default, whole rest of your Google services so things like, talking. To bigquery or, doing, machine learning really. Nice integration you don't have to Slough around surface, accounts or worry about API, keys or anything like that it's it's really nicely, automatically. Authenticated. So. Firebase which I started to mention. Has. A cross-platform. Suite of tools it does a ton of stuff it has things like virtual Android, devices. For testing in the cloud it has, things like analytics. In your mobile app it, has things like showing ads doing push notifications, a whole bunch of tools for doing mobile and web development, and. How. These two played together specifically. Cloud functions in firebase is a. Lot of people you can just you can build a fully featured app just. Using, firebase. So something like fire stork because your client actually talks directly to the database which is kind of crazy for a lot of people but it actually works and it works really well and we'll see an, example of a customer who's doing that in a little bit but. Today, we're coming along we're talking to a cloud fire story does this real-time syncing, so it's talking to other to other, clients it's really really nice but we wanted to kind of extend, beyond. The, capabilities, of fire store I'm gonna supercharge, it with Google cloud so, that's where cloud functions kind of sits in the middle here and again, it's automatically, authenticated, to Google cloud so you can have it say watch a fire store database for. Some record that's getting inserted some, changes happening some delete and then automatically, it. Will run and then it can extend into GCP to do things like Big Data machine, learning talk, to pub/sub to integrate with third-party systems. So. Let's take a quick. Example. Of a demo we, can switch to the first demo laptop we're. Gonna do this exact here we're gonna have a cloud function that's listening to a database that takes some action so. To show, you what. Firebase. Looks like. I. Will, show you all of the codes that I wrote that. Runs on the server, which. Is none, everything. I wrote for this demo actually runs in the client the client all talks directly to firebase. So, if you want to see the Java Script it's just a few. Lines I'm gonna say listen and firebase to the root message. Node whenever that value changes, go. To title here and then change the text hopefully. That shows up. So, that that's it that's it is literally a dot HTML, file there's no back at all backing at all so. I can come. In here and actually. Let's pull up a couple of other browsers. I've got Chrome. Firefox and, opera anyone, still using opera oh, this. Yeah it's always 102, Wow guess, it's a big room keep. The faith my friends. So, here's my here's my database it's called raccoon house it's a long story tell, you later I just, love raccoons actually that's the whole story so. It's, listening to this message node right so whenever this value changes it's gonna automatically, change over here in these three other applications, so I'm gonna say, hello. Google. Cloud, next, and, hit. Enter so just. To explain what happened this over here on the right that's firebase that's running in the cloud and then over here my app on the left is my app with. Literally just JavaScript. But. Like I said called, functions bridges the gap between firebase. And Google cloud so let's apply some machine, learning I. Pre. Deployed a cloud function that's gonna watch not, for message but for. Message. Too and it's. Gonna take the value that I put in there and apply some machine. Learning translation. So I'm, gonna say hello all of, my friends, and, hit. Enter and you should see again, it. Automatically, updated. It but then it also translated. It to Spanish hola, todos amigos but, of course we have a lot more than Spanish as part of the Google translation API so, maybe we'll translate into Japanese French, Danish Greek and. Actually. We can do way more than five right so let's. Translate a whole bunch of these. Because. We had a cloud function watching, this note here and.
Then Adding it automatically, to the database so, with just a few lines of code we, can just get these results from the client all. Right and, go back to the slides. Alright. So that was computing, so that was functions. That. Was firebase and it was App Engine so. I'm not gonna go into too much detail on some of these but. Just to kind of know what, when we talk about data I like to think of it in three categories there's, data that your application, runs on. So think about you have an e-commerce site and you're, doing things like holding. A shopping cart for a user you're. Handling. Inventory, these are like active reads and writes to your application. So. We have a couple of things there fire store which. Is a. No sequel database I'll talk a little bit more about that in a second. We have BigTable, which is not technically. Serverless in the sense that it does not scale to zero it scales down to still. Very expensive. But. It is powerful and john will talk a little bit about how smart parking uses it in their architecture. But. Then you have reporting, and bigquery which you've probably heard a lot about so to our server this reporting tool so you can query petabytes. Or terabytes of data using standard sequel but you don't run your app on bigquery, right it's just a reporting, database you dump all of your all of your data into it and then you can have your data scientists, churn through it your, statistician Xand, really do interesting reports and visualizations but you only run your app on it and. Then data flow if you need to enrich or transform your data in real time as it, comes in you, you can actually sometimes get away without using cloud functions for a kind, of a lightweight way to do it but if you need to do real data transformation, then dataflow is the way to go. So. Just to dig, in just a little bit into firestore, it's. So no sequel database it's serverless, so again one of the tenants of server list is you only pay for what you use so, firestore you literally pay per operation, on the database so if you're not doing writes or deletes, reads. You're not actually interacting, with the database actively, it, doesn't cost you anything right other than sort of the storage that's already in there which, is a really interesting model for a database you. You. Don't have this like persistent, cluster, that you just have to keep hot and keep, replicating that's. All handled behind, the scenes and you only pay when you are actively, interacting, with it as. Offline support, for even for web so, we can kind of cache these, kind.
Of Intelligence, around fetching. Actually from the database or fetching from a local copy so, the super super cool I could do a whole talk on firestore but there are tons of them at next so definitely check those out, and. Then yeah serverless pricing. All. Right and then last, I'm going to touch into server list machine learning learning and then we're gonna see some examples of how people have put all of these together so, on. The spectrum of machine learning from, a generic, task, like I just want to know if this is a dog or not so the we have a whole suite of machine, learning API so like a vision API speech, API and, these are trained for just general purpose, problems, so, in, this, case I want to know I. Don't. Necessarily want to know is this a dog I want to know is this miss potato this is my sister's dog. When. I first put the slide on it just said potato and I was like that seems like the worst machine, learning, model. Ever there's like potato, no. That's her name her name is potato so Miss potato. So, on one end of these pre-trained, api's and the other is you have these specialized. Pincer. Flow, machine. Learning models that your data scientists, have gone through in britain we do have a service, tool called macleod machine learning engine that will scale up your tensorflow, to, do training based on your huge data sets and how these kind, of automatically, choose some parameters to see what would turn best on and then it can also scale up and down on prediction, so once you've actually trained but you still actually have to write machine learning code as, you saw today though. We actually have something that sits in the middle made a couple of announcements around that for. Auto ml so, if you did if you missed the announcement or had not heard of auto ml the. Idea here is you take a bunch of images so a bunch of images of potato. Not of potatoes. And. Then you give it to ml engine and it will learn to recognize for. Example one. Particular, dog one, particular cat or whatever makes sense for your business usage, and then. You can just talk to it via client, library or an HTTP endpoint. All. Right so now let's talk a little bit about how do we put all of these people pieces together what, are people doing with that so. On. A pragmatic, perspective like how do you actually connect these so, one of the things I think is interesting serverless. Architectures, tend to be event-driven, architectures. Which, means in. A in, a trillion traditional model I have my monolithic. App and I want to take some action so I I actively, make some call to some other end point but then you kind of run with these whole issues well what if that endpoint is down now I've sort of queue up that request, what if there's latency, I also have to have a discussion a lot of times like what what does this aim point look like what kind of data format. Do you need it in well. A lot of times you can just use events, as your API so, we as a cloud provider have. Standardized, like eight a file, gets uploaded and here's what that looks like right here is. The field that you will get here. Is for example a timestamp of when the file is modified here's, the path of where it is in cloud storage and that can serve as your API in a lot of cases so, I don't actually even have to have, you a conversation with someone who's, working, on that file because they know what that's gonna look like it's been predefined, as a service, event. By. For example Google. Cloud. And. I've actually built a whole bunch of apps where literally, I was, just hey I'm gonna give you a file and that was the last conversation we had because, it was all standardized, but, you. Will have plenty of cases, where you want to do more complex, interaction, you actually need to have data that goes along with that file so, their. Cloud pub/sub is your friend and I'm not gonna go too deep, into cloud pub/sub but I think it often gets overlooked, as a piece of Google cloud because, it's incredibly, powerful and incredibly flexible, and, if you don't know what pub/sub is. It's. Essentially. An asynchronous messaging, system so you, talk to it over HTTP, or G RPC, and you can just give it arbitrary, let's say JSON messages whatever format you want it's really. Really fast and scales, really really, high, but, as this name implies pub/sub, if you're not familiar with it sort of as a model of message, delivery it.
Lets You do things like have. Multiple, subscribers, creating. Subscriptions, to a topic so. What that means in practice, is if I, am say a cloud function and I'm doing some computation, on the piece of data and I wonder sort of hand it off to the next chain. In the in. The architecture, I, could. Just call it directly but. Then say what if I have other other things I want to swap in either in the middle there or I want them to also respond, to that event so now I kind of have to make this thing aware, of what is happening next in the architecture, if that makes sense what you want to do is just push it to pub/sub it's alright for like have, at it and then whoever wants to subscribe to it and it could be called cloud functions there's a nice integration, with cloud functions and pub/sub whoever, wants to subscribe, to that topic and get that data can pick it up it could be something at Google Cloud it could be something off premise it speaks HTTP, and G RPC so it, integrates really nicely with a lot of third-party systems, so. Let's take a look at what this looks like oh here's the subscribers. Three. Different subscribers can all respond, to the same message because, there all subscribe, to the same topic so here's, what this might look like in practice so, again we've got our app we're. Just talking to cloud firestore we're happy as can be but we need to take some kind of action so, say. I've added a user, a new employee comes, in at my company so, I insert them into the firestore, database. Under the users. Document. So. Clouds of cloud, functions, recognizes. That picks it up and says, aha I know. Some. Other things want to take some and we have a new employee there's a lot of things that have to happen other than they just get inserted in the database right so, the. Cloud function just doesn't do anything it just publishes, a JSON message and says hey here's the user, ID here's whatever relevant, made it metadata, goes to pub/sub and, we, have a couple of other systems, that are listening to pub/sub so we have maybe a. Third. Party SRA another. Function on google cloud that, handles. The badging and so the badging system is just nice. Integration cloud functions, you just say hey listen to this topic whenever, a message comes in automatically, run but, then maybe we have some other system some IT help desk system that. Doesn't, run on Google cloud but, it does speak HTTP, so, we could actually have it listen to that same pub subtopic so with one message we, can fan out to as many systems as we need and it turns out to be really really flexible and, and, dovetails, really nicely with this event based system, so, a. Couple of real-world architectures, where we're seeing people put all of these service tools to use, so. There's one medical. Consulting, by a company called cloud, ER. So, let me describe it sort of very very briefly and then we'll see it in action actually. So. The idea is, the. You. Know you go to the doctor you have a conversation, they. Talk to you for a while and then, sometimes they'll take notes as they type sometimes.
They Wait till the end and then they type up the notes but there's a lot of tendency - there's a lot of works there right so what. The folks at cloud or did is they come up with a system where it. Basically will transcribe, the doctors conversation. With you and, using. The cloud speech API, sort, of using cloud speech API it will transcribe, it put, it into text so you can do processing, you, can use, the data loss prevention API DLP. API to actually redact, personal. Or sensitive information because. Maybe you don't necessarily want, to store. Personal information as a part of this record like as a part of this transcription. Maybe that lives in a different database because it's sensitive so the DLP API can actually extract that can sort of star, it out can. Handle images the, DLP API, then puts it in cloud storage which. Has a trigger. For the natural language API to do processing and, you. Know do things like what was a sentiment, were than the nouns and the verbs and, once you get to the natural language API there's, a whole world of things that open up so you could for example, imagine. A situation where, you, have a two, microphones, one that's near the patient, one that's near the doctor and you could actually do sort of some mapping you, could say oh I saw, the the the doctor acts asked, this question at this time stamp and the, patient responded at this time so you can do some correlation, you could start to develop some, really, really powerful data. Processing, in mining techniques to, improve healthcare, and. So we were actually, fortunate. Enough to have you. Know do you want to come up so. This is. The. Can, we go back to the demo one laptop, so. Clowder is a is a company based in the Netherlands so, this is a Dutch. App and since. I do not speak Dutch. I'm. Gonna have some help you. Know let's, see can you say what's. What's, a really serious, medical. Situation, I have. A I, have an itchy knee. You. Know I don't know what that is in Dutch I. Will. Not repeat that. Not. Because it was offensive because they don't speak Dutch so you can see now it's taking the audio file uploading. It to cloud storage, where, again it's going to go to that chain of using. The speech API to inscribe, it DLP, API to, redact. Personal information and then it's gonna do some analysis, and so we have here results so. If we click that it actually as oh I don't speak Dutch but, oh we missed that we're missing the transcript.
Yes. So, I don't speak Dutch but I'm guessing that first one knee is knee. And. Then what is joke. Hitching. All right so it actually was able to identify so locations, are where in the body was the issue and then a particular symptom and then we pull up the other one so we can see so. This is a previous one so they actually do the transcription I'm going to blame the Wi-Fi because when a demo, goes around onstage it's, probably the Wi-Fi. Cool. Thank you yep. Yes. All. Right if we go back to the slides I think, speaking Duchess applause worthy because I do not speak it at all I'm, sadly, monolingual. So. Here's another example of. Production. Sort of serverless, architecture. That we've seen is a company called guesswork and they. Solve an interesting problem so. They saw the problem, of being. A recommendation, engine for. Users. Who have never used a site before in a, market where ecommerce is often. New or, the site itself is rather is new so how do you make recommendations. When. You have no history of purchase, purchases. Either in sort of your store at all or for, a particular user specifically, so, it's interesting. Complicated. Problem and they use a mix of machine learning and service super. Cool so part. Of what they do is. They do it on images. So, they, will look at products across, countries. And across, continents, to see, okay. Maybe, I don't know. What this is called in this language or instead of actually translating, the the metadata around it like well let's just take a picture of it right a bowl is here a bowl, is a bowl in China is a bowl in South Africa, is a bowl in Texas and, so by, using the vision API and, some custom machine learning they can analyze these images and so they actually can aggregate data across huge. Amounts, of existing. Ecommerce. Data sets to, actually make intelligent recommendations. When, they really have no actual. History. Or basis to do it on and. They do it also real ously so. The. Kind of classic example Black Friday they, went from from, one user essentially, to 20 million in one day and, thanks his power is sort of that scaling, up and scaling down of service all of these components scaled up with them so. The flow is essentially, the, users on their mobile app and. They're. Talking to firestore and so this is interesting because they actually replaced, they. Were using App Engine for their web. Back-end, and. Data store for their data storage and they replaced both, of those they got rid of App Engine and datastore, and use it just. And use just firestore instead and that's because your client talks directly to fire store and it turns out a lot of what the App Engine app was doing was literally just taking data from the cut client putting, it in the database getting. Data from the database giving.
It To the client and in. A firebase fire store model when your client can talk to, directly, to the database and don't worry it's protected by a bunch of security, rules in this kind of whole process around that but, it actually eliminated. One of the layers entirely of the architecture, they were able to consolidate it. Entirely. Into fire store so, anyway the mobile app talks to fire store and, those database records, key. Off a couple of cloud functions, one, of which goes. To the vision API takes, a look at okay, what is this users looking at okay, I'm gonna extract some labels and then go and try and find some matching, and. Then also pump. Some of that data into bigquery so, they can do some reporting later and. Then the course going. Back to the theme of back-end. As a service is there some service that is out there that does what you want to do don't, reinvent the wheel so they use send great to send an email directly. To the users to make those recommendations. And. One of the things that they liked about cloud, functions so not just that it's scaled but because. They are called functions are designed to be sort, of single purpose little. Dedicated. Functions there's, a bunch of them strung together so, they could actually swap, those functions out and. Do. Like. Refactoring. Their code or, they can experiment with new things do some a be testing for example doing using, pub/sub and say okay well we have this we, have a message coming into pub/sub and we have this function that's listening to it now well, let's actually spin up some other function that also listens to it and kind of do some a bee testing and parallel and you never had to take the system down at all it's all happening, live so, super interesting and super cool architecture. Using, all serverless tools. All. Right so that is all that I have as far as those for another. Even. More interesting, use case I. Am, excited to welcome all the way from Auckland. New Zealand I was, fortunate enough to visit these folks a few weeks ago when I was down in New Zealand the, really, really interesting stuff that they're doing so, please welcome John herd CTO of smart, parking. Thank. You Brett it's, a real privilege, for. Me actually some of my colleagues to, be here and just, tell you and share with you a little bit of our journey, into. A purely. Serverless, system. Which has started. To replace our legacy, platform. First. Of all I'd like to just tell. You about smart, parking as a company you probably don't, know who we are because we're sort. Of one of those little startups, that started. Up down, on us in the south of the. World, the. Technology is developed. In Auckland, New Zealand but. We've, grown and and we now have operations. All around the world we've been going for 15 years. You're. Probably aware you know most startups, when they become you, know people become aware of them you've usually been around for a little while.
We. We, have tens of thousands, of sensors now installed. Around the world and these. Are you. Know dealing with probably, one of the things that you hate the most which is parking, but, our mission, is to actually reinvent, that experience, and part, of the strategy. That we've taken is using new. Service. Technology, as a way to also reinvent, that. So. Just as a, picture, this, sort of outlines, the solution, framework. That we actually have today, and. This is actually in operation in, real world cities. All right no comments, than a moment but, first of all we have a very comprehensive, range. Of solutions that work today that our customers, all around the world are depending, upon day-in, day-out this. Is industrial-grade. IOT. You. Know you'll be familiar with car parking right you know we all do it you, know it's one of the the hassles of our lives the. Interesting thing about parking. Is first of all a car can depart and another one can arrive, in really. Really quickly so within the second, we, have to detect very accurately, whether, a car is arrived or departed, and also, to obviously, know that that, car is present there or or, it's not present. In that parking, space and. The reason that it's really important, is that the. Parking operator tends to issue infringement. Notices to you which could go to court and so, being. Legally, enforceable, really, puts some very strong constraints. On the class of IOT now. The other thing that we do is we put these sensors, into the ground there's. You know diagram, there of a car sitting over a sensor so, we put them on the ground which actually, is really. Bad, for radio communications the, other thing is the sensor is actually running on a battery and it has to last for five, to ten years in the road right. Because we can't run power to them and and, we don't have an you know we can't put big batteries these have to be small batteries so these are battery-powered they're, radio communication, we put these big metal things on top of it which really, sort of creates. Havoc, for, the communications. Of real-time communications. And so these are the things that we've had to solve. But. Most. Excitingly, and this is actually being driven by our customers, is that, this is more than just parking, because as soon as you put those gateways on, to the lamp poles around, the streets of the city or inside, your multi-storey. Car parks, you've, started to build an infrastructure, for doing smart. City functionality. Whether it be monitoring garbage. Cans air, quality, etc. Etcetera, and so this. Is actually where we have, started, to have to head and, so. As I mentioned where we're in cities around the world and that further, off there is actually a Westminster, in London London. Westminster. City has, our sensors all around the streets and, it's actually changed, the way their parking, operates and in the city of London.
So. The. Question why cloud native I quote cloud native because, what, we're actually, doing with, our system, today is there, is no operating system there is no Linux there is no windows we, are not using, relational, databases, for, example as you would traditionally use. And. The reason is that we're, starting, to build very. Very live interactive. Streaming. Systems. And that's actually the requirement. Of our lives for. Example you know if you're driving along the road and and you know your google map. Guidance. You know this the the turn-by-turn, sort. Of took twenty seconds, to sort of tell you when to turn you know you'd be up a tree right, this, is the same for most of the types of solutions and requirements, of our modern world we're, requiring, that instantaneous. That live delivery. Of information, the. Other thing about. Going. Completely service, or cloud native as I call it is that it's allowing, my, engineering. Team to focus on the real benefits. And the real functions, that we're trying to achieve to reinvent, that parking experience for. Example when, you look at your traditional, code and I'll come on to a slide of our our, existing, legacy system and you'll sit under standard and I'm sure many of you are very familiar with the traditional. Ways. Of building platforms, it. Takes, a huge, amount of effort and I would estimate it between 50 to 70 percent of your coding effort is just gluing things together, gluing, an operating system together with your relational database with, the logically, is built with, the web servers with the communications. With. GCP. With Google, cloud platform, going. Completely service. We've, actually be able to eliminate, a lot of that. The. Other things that redefines, a, number of the key things that we've had to deal with historically. For example disaster, recovery when. Data, now has eleven nines availability. In the Google cloud platform it, makes you think differently, about how. You store, and replicate, and manage information manage. Your data sets and by, the way you know most of the cloud operators, Google included, it's. It's a really dog-eat-dog. Cutthroat. Competitive. Market, on the pricing, of storage of data and so now we actually treat date you know the storage of data is really a non-issue. It's actually the processing, of data that there's a most. Important. III. Sort of explain. It that also when. You're trying to do new things just, doing the old what, you know doing things the old way and hoping for a different outcome it's not really the smartest thing to do and so we took a strategic, decision of doing.
The. Technology. Doing it building, our new platform in, a new way with new technologies. Because we wanted different outcomes, and. Fundamentally. I am. Absolutely. Convinced, that what we're going to see over the next three. To five years is that all information, system. Processing. And building, of large-scale platforms. Like ours is, going to be done this way in fact the legacy, OS is like Windows, and Linux and so on we'll slowly start to become less and less significant. For. Our future. Next. Question why Google, well. It's important, to actually think about the, philosophies. And the drives of, the vendor, that you using as to why they're building technology. In a certain way and and. Google has submission, which you all know about you know about organizing. The world's information, now. The world's information is, schema-less, there is no, defined. Static, set class, of data and this is actually important for all of the modern applications, we have today and in. Fact all the technology, demands the the ability. To scale up and scale down and and, to deliver. Global. Scale computing, is fundamental. To what googles, mission is and so that was aligned, with what our requirements, were. As. Brett. Has already outlined google, has been building, and and taking, from their assets, of knowledge this, deep. And broad. Range, of. Service. Building, blocks that we can actually quickly put together and then, finally, machine. Learning, which, is the way we are going to interact and deliver, information in. Much. More, fluid, and natural, and consistent. And continuous, manners. Is through machine learning so Google's. Machine learning, centricity. It was also significant, to us in our decisions, there so. Let me just quickly go through what. We. Call smart cloud which is our platform for, doing this smart, parking solution. But also more importantly expanding. Out into smart cities first, of all we wanted to be able to ingest, and and, and and, send, communications, to anything, and so, having an open, eye o T framework, which is IOT core from, from, Google. It was, a key element in, AB. Construction, of SmartCloud it, actually allowed us also to eliminate all, the burden of developing, that technology, trying. To be aligned with standards of the industry, because Google's going to be doing that for us and it scales up and scales down for the millions and millions of transactions, messages. That we we, deal with today every. Single day. So. City, scale global. Scale processing. Now. The following, points. On this was. About also illuminating, and moving, you know getting different results about what we are trying, to achieve and wanting to achieve was. Taking. The developer, out of delivery. Of dashboards. Of being, able to do big data type data reporting, and analytics and, also, business rules and. So. We built into smart cloud will make our mechanisms for doing, the end user the user of the system to do their own dashboards. To do their own simple. Big, data analytics using. Bigquery and, data studio and then. Finally managed a pis so. This is a sort of application. Framework, and the vision of where we are going and we're already starting to do a variety. Of these things we are certainly doing the data intelligence and analytics we're, deploying broadband. Off the gateways, in the streets so, that public can have public broadband, and so we're doing that in one of the cities right now I. Mentioned. I was going to just show you this is what. You know our legacy. System looked like this is it's, not the DS you know not every bit of the detail there but fundamentally, this was the architecture, of our system and as you can see there are lots of moving parts it, is complex, it's, actually hard to keep it together and keep it stable. Right, now and many of you will actually be familiar with this day and day out right, and and, we or even had the, religion, war of the Linux versus, windows, in the company, well and so, all these were the fundamentals, and this. Is what we've moved to so, we've gone completely to, using IOT core as tough as the bottom. End which in ingests. And aggressors. Messages. From from, devices, and sensors through, our gateways which we call smart spots. Then we process those those messages, coming in using.
Cloud Functions, into. Pub/sub pub/sub is just awesome it's, it's actually the way that we actually, split. Information. Into different pools and we're, delivering this information, into both BigTable. Into bigquery doing. Processing of it you're using, logic, we're using App Engine, obviously to, run various, parts of multi-threaded. Processing. And then, very. Importantly we, we front-end, it with the tools such as data studio for, data, analytics, and and, and a variety of other machine. Learning and such things and so, and then finally we've got a whole portfolio, of api's, which. Connects to both, our own systems. And also, external. Integrations, so I just wanted to very very quickly I know time is running out can we go over to demo, two and. I just wanted to quickly show this is actually. Smart. Cloud actually in action and this, is a city Hamilton. In New Zealand so live, data and if. I tap, on it I can actually see you know there's a car that's been over staying there. And. This is just one of the dashboards, and, by, the way this is all being, front-end, behind. It is firebase, and, firebase, off for. The authentication. And. Yeah. Let. Me show you also here's, another city live. Again, this is Cardiff, on the other side, of the world and doing. Exactly the same things. And. I just wanted to also show you the data studio, data, analytics, reporting, that we have built in this is just incorporated. Right into, the, smart cloud system so this allows data analyst, to go and build their own reports, we, haven't had to go and create no, developer, is required, to go and create these reports, anymore and they're rich with. Capability. That is just, fantastic, so, we're, putting or democratizing. The mechanisms, of doing people. Letting. Our customers, to their job and just. As a little. Additional. Thing that we're starting to research is using, machine. Learning and here, what we have is we're, looking at a small. Range of parking, bays on. A streets in a city and and. This is actually so the actual and then using the predicted. Occupancy. Of those bays and we can tell we can start to forecast, using, machine learning and this is is adapting. And changing as, the, dynamics, of that site or change. Adjusting. And so, this is really exciting because this, is going to deliver us new ways of interacting with the user with. Knowledge about the future, giving them prediction, about where, the likelihood is, of be able to park etc, so, if you're quickly now go back to my slides. So. I just wanted to finally summarize you, know going through in, our sort. Of pathway our, road to, cloud native or service computing, first, of all api's are really important this is actually the way we have defined, the, architecture. The framework, of smart. Cloud the api's, are essentially, the scaffolding. And the blueprints and in, fact if, you're. A programmer, you know in those first. Class you, know computer, science 101 programming. It was input output, and then logic right, AP, eyes are actually the framework of that input/output, it starts to define the framework so that was really important for us the. Other thing was looking. At the data from a streaming, perspective, rather than just putting into a database and then pulling it out we, looked, we needed to actually manage, our data in a constant. Flow stream. You know right from the device in the street through, to the dashboard, on on somebody's. Screen or the app in somebody's hand live in real-time and so. This is where we we actually are pushing data into, big, table, big. Tables you know is it seems that the same technology that Google search works on and what. We're doing is we're creating what we call frames, which, is done parking, event and so when a car arrives a new frame is created, in BigTable. And then as new, information comes, in such as a payment comes in or a, rule, gets. Triggers. Something like an overstatement, though.
That Frame gets updated then when the car departs, that collapses, and meanwhile. While we're doing that we're, pushing those events through pub/sub into, bigquery and, also into other, doors and. What we're developing here is actually a knowledge graph of the. Information, that's going on which is really enabling, us for the knock for AI. And, capabilities and also. Very significant, for us is is it's delivering, a roadmap, of functionality. And capability, to not just ourselves to, respond, to the market needs but also for our customers, as they, use it so, we're able to eliminate a lot of the overheads, and burdens that we used to you used, to have were there a legacy. System and we're focusing, our energy and being, able to allow ourselves and our customer, to our customers, to focus, their efforts on things that really make a difference to them, and. So I truly. Believe that we are actually reinventing. The experience, and it's really, needed obviously for parking, right it sucks, and. So one. Of the things that we're trying to, do is to actually change that experience, in a number of ways and in, fact this is actually the metric. By which I actually make judgments, about if, we're doing something is it going, to reinvent, that experience, and this is a very important point to me. So. Just, a few things about our little journey with Google so far we're. Learning a lot right. It's, not actually, a smooth, highway. With no bumps, on the road we're, having to keep learning but this is actually good because that means that we're pushing boundaries, we're actually doing something that is actually not, just like everybody, else we're actually making. A difference you know if we're reinventing. The. Experience this is what you got to do you've got to actually, experience. Some pain and I. Mentioned. Before focus. On data streaming and data flows using. BigTable. Is still, an art there's, a lot to learn and, I, see some nods right and. It's actually, getting to the last point you know we've had great support engineer. To the engineer with Google and it's been actually quite exciting. When we pose, a question they go gee I don't know how to do that and that's really hard, we. Can't do that or so, on and so that's been a really.
Fantastic. Experience, and. We work with other cloud operators, by the way we're a big consumer, of other cloud and the. Support. We've had eve near to engineer. The Google support team is just awesome best. Of my experience, so far. So. That. Was the end of my comp my presentation. And so. Just, a couple of things we, do have some t-shirts I know we will run out of t-shirts, but if you want a t-shirt at the end of the talk. See. Some of my colleagues at the back of the room we've also got a white paper which. Talks about, how, we do, the flow what we call the, temporal. Parking, event processing. And it's not just about parking, it's about anything is. SmartCloud. Is a very general purpose system. Yeah. XYZ. Time is. I'm. Sitting we just we're, not sure so John will be around there's, an IOT pod there's a service interactive, exhibit, on the first floor that I'll be hanging out and a whole bunch of folks from. This various product teams and engineers across, Google so please step I stopped, by there's a vortex, cannon if you don't know what that is it's gonna be pretty cool and. That's. It will be around for questions thanks. You.
2018-07-28