Hello. And, welcome. To, migrating, a monolithic, application. To micro services. I'm. Your, speaker, Soren, walls I'm a software engineer in Google. Cloud. Your. Feedback is greatly appreciated. The. Surveys, are gonna open they said in 20, minutes ish I guess. Give. Me five stars like and subscribe I'm sure everyone's made that joke. Allow. Me to introduce. Myself. Can. You all hear me by the way. Great. My. Name is soren, and I. Work in an org at Google called cloud, engineering. Productivity, and today, I'm gonna tell you about two projects, I've worked on, in cloud. Low. Heat and glass. Pane. Unfortunately. My, teammate. Sergio who. Is the, I, didn't mean to go to that slide just yet let. Me go back, can. I. Yeah. Okay, my teammate. Sergio is the. Tech lead on low he this, first, project here and he, couldn't be here today he gave this talk with me in San Francisco earlier this year and. I, just want to give him a quick shout, out because, he's. Graciously. Allowed, me to still. Use all of his examples in this presentation, and. As. Examples. In in the talk as you'll see so. Thank You Sergio. Alright. A show of hands who, here has, heard of release. Orchestration. Before. Anyone. Okay. A few people how about CI CD. More. Of you okay that's what I thought all right well lo he is essentially, a release, orchestration. Tool release. Orchestration, for those of you who don't know is essentially, how you can automate the process, of planning. And managing, a software, build through different stages and environments, including. Testing, and deploying releases. To your end-users through, continuous, integration, and or, deployment, which is what CI CDs. Like. Most other tools relating, to see ICD lo he has this concept called a flow a flow is, composed, of three things, triggers. To begin the flow a pipeline. With multiple steps to execute, and finally. Notifications. That fire when, the pipeline finishes. But. What makes Loki a little different from other see ICD tools you might know is that it's primarily designed to handle hybrid. Applications. Which. Are apps that may use different tech stacks different, tools or even completely, different infrastructures. Now. Glass, pane on the other hand is a metrics. Tool I'm sure you're all familiar with, the concept of metrics, it's essentially, an internal, metrics, platform, at Google, that allows directors. And managers, to understand, the health and relative. Performance of, their projects, and their, teams. So. Today. I want, to tell you a story about. How. We took, both Loki and glass pane and converted. Them from, monolithic, apps and split. Them up into micro services. But. First what is a monolithic, app well. Traditionally. This is the way most engineers. Used. To think about applications, being shipped as large, unchanging. Chunks, of code that simply, met the initial requirements, set out by product. Teams and executives, no, more and no less all, the functionality, and business logic is. Compiled. Together in a single binary with, a single data layer this. Has some, advantages, and some disadvantages. Of. Course we decided to build Lohia in glass pane as monolithic. Apps in the first plane at in. The first place why did we do this, well. Monolithic. Apps allow you to experiment quickly. They. Allow you to focus on the business logic of, your application, without having to worry about all the, complex. Interactions, that can be evolved when you start to introduce new binaries. It. Prevented, initial, over-design, of our services, to begin with we just wanted to get something hacky, quick up and running. We. Were also a pretty small team so having, one monolithic app allowed us to pretty. Easily manage. The changes, that we needed to make, also. Having, one binary means. That we only need to do things like debugging. Security, and operations, once. It. Was good while it lasted, don't, get me wrong but, as our project and team size grew we, started to face some new challenges we. Started to feel like we had. Gotten. Ourselves into a mess, we. We started having more product, requirements, as that as the project gained popularity within, Google, we. Began onboarding, new team members and that became difficult because, there was too much cognitive, load every time we brought someone onto the team they had to learn the whole, architecture. Of the entire application, we. Had to start delaying, deployments. In order to sync features across the, different sub teams that we started having.
There. Was also no way to independently. Scale, the, application. The. Different pieces of the application rather, that, needed. For, example one part might need more performance. Than another one but there was no way to do that, it. Also became, pretty, messy, to debug because we, weren't, very good about keeping everything, separate, in the binary which is possible. But we. Weren't very good at it so everything. Just got, really, mangled together, so. We started to feel like we had just built this cage around ourselves. That we were desperate to, escape and. This. Is where micro services came in but. First what. Is a micro service and. Where. Do they come from. Okay. A little. History lesson before we get into the nitty-gritty details before, microservices. Became popular, service-oriented, architecture, was introduced, in the late 90s, the. Separation, of business logic into, different modules was, a big improvement over monolithic. Apps but, it didn't go quite far enough in terms, of decoupling, services, from each other if. Their application. Was like, a loaf of bread. It. Was kind of like they figured out how to slice, the bread but. Then they just still wrapped it all in one big wrapper anyway. Finally. And it, not the best analogy but you get it right okay finally. The industry figured out how to completely. Decoupled, the services, into micro services even, for, large-scale enterprise. Great applications, with, millions, of users these. Tiny services, introduced, a lot, of neat benefits. Here. Are some of those benefits, micro. Services turns, out are pretty simple to understand, these. Small services, have a pretty, simple onboarding, process. For. New team members that might may, want to join your team, it. Also allows you to work on different tech stacks one service might be in Python another might be in Java etc. They. Also guarantee, something called isolated, responsibilities, which I'll get into later it's a part of a broader concept called, domain driven design that, tends to work really nicely with micro services. You. Can also deploy them completely, independently, they're, scalable, and distributable, you. Can reuse business, logic across your application without having to rewrite it or have, complex. Inter dependencies and micro. Services are really fault tolerant, I'll. Show you some patterns later about, how you can handle, situations when, services, go down much. Better than you can with a monolithic, app. All. Right so, if like us these, benefits, sound nice. And. You're. Thinking okay, maybe I do one of my great my monolithic, happened to micro-services, how, do you actually go, about doing that.
Specifically. When we decided to migrate glass, pane and low heat into. Micro services we asked ourselves two questions, one. Was. How, big should the services, even be and two. How. Do, I go about splitting, them up once I've decided on the size a number, of micro services. Well. The, micro, part of micro, services, it turns out is more of a label, than a description, to be honest the size of a micro service can vary which, is the whole idea behind service, granularity. Unfortunately. There's no one-size-fits-all. Approach. It's. Going to depend on, your, tech stack and your infrastructure. Ask. Yourself, on my. Tech stack how, much overhead is associated, with setting, up a new service, a good. Rule of thumb is if it, costs, more to set up and run your service than it does to implement, it that's a sign the service is too, small. As. I previously mentioned a good approach, to use is something called domain, driven design. Domain. Driven design essentially. Means thinking about each piece of your app as a separate. Domain, here. It's important, to, have bounded, contexts. Around. Your domains and to follow the single responsibility, principle where. Each domain, is only responsible, for a single, thing, if, you're. Familiar with object-oriented, design this is kind, of like each, micro service handling, one strong, entity. Per. Service. Along. With its relative, related. Business logic and data layers. But. Beware of an anti-pattern called nano services, this. Is essentially, when your services are too small and in just a moment I'm going to show you an example, of, how we learned this the hard way in glass pane so that you don't make the same mistake. But. First how, did we decide on the number of microservices. Turns. Out glass panes magic, number was seven, ish. By. Our estimation, the original, monolith, had approximately, seven. Different domains, so, we decided to split it up in, seven, ways the, core service, was, our new micro, service that handled user requests, and talked, to the other six services, each handling, their respective, domain. Sounds. Good right. Unfortunately. We took things a little too far when we decided to create a separate, metric, collider service, for every, single metric in glass pane. This. Is a classic, example of, something called nano, services, or services. That are too small, if. Glass pane was going to support hundreds. Of metrics. We, were going to need hundreds, of micro services but not only would we need to build deploy, release. Hundreds. Of micro services to both prod and QA, we'd, also need to set up the api's, and, other, GCP services, to execute the business logic. These. Are supposed. To click. That's. The business logic happening this, was not feasible, for. Our team size and our tech stack so we decided instead to. Go with, this sort of serverless, approach. We'd. Have a single, micro service responsible, for all the metrics this meant that we. Only needed to build deploy, and test. And. Release, to. A few different tape staging. Environments, broaden and QA and maybe dev. But. By using Google Cloud functions, to.
Collect Metrics. Instead, we ensured this service still obeyed the single responsibility principle but. Without all the maintenance and setup that was required for a new, ji-ae instance, for every micro service essentially. Ok. We, talked about deciding, on the size and number of your micro services but how do you actually go about splitting, up the application. If. You have a monolithic, app one, useful, pattern for this is, the. Strangler, pattern, and it. Gets its name from, the. Strangler. Vines of the fig, tree. Which. Essentially. Start. At the top of the tree up. In the leaves and they. Slowly, creep. Down the tree. Until. They've rooted, themselves, in the soil and, the. Tree dies. Think. About your model if the gap like that just, kill. It at the end. If. There's one thing to remember about strangler, vines though, it's. That they start, at the top because, the smaller branches are the easiest to consume. Good. Thing to remember later ok. So let's walk through an example the. Big thing you want to see how do I split up my model if the gap to. Start with we have our monolithic, app it's. A single binary composed, of three, services in one database that, handles all the users requests, so. The. First step in the strangler pattern is to set up something, called a proxy. So. That now when. The user sends a request we. Can begin redirecting. Traffic to. Where. Any. Guesses, you. Can just shout them out. Sorry. Back. That's, actually, right we're. First, of all gonna just redirect. Traffic. Back. To. The monolith right, now it's not even gonna do anything, it's just an extra layer we've added that, eventually. We're, gonna use to redirect traffic to, the micro services but. Right now we don't we don't have anywhere to redirect to yet. Okay. Remember. What I said about strangler, vines. Where. Do they start. Where. It's. Easy. Yes. Okay. At. The top because, it's easy so here the least complicated service, happens to be the inventory, service so that's where we're gonna start. So. First. We create a separate binary with, a separate database to handle inventory, where, we can start reimplemented, the necessary logic, for, the inventory part of our application but, before, we begin implementing, we, should first set up a link between, the proxy and this new binary so we can big begin, to redirect the traffic once. The, new, service is implemented. Then. We, can just remove that, part, of the, monolithic. App, once. The new service has replicated all the old logic, from the inventory part of the monolithic application. Okay. So, the, next easiest service, happens to be the. Shipping, service so like the strangler vines we'll take that one next and. We. Create a new binary hook. It up with the proxy. And. Begin. Reimplemented, the shipping logic from are monolithic app in this new binary perhaps. You copy over some code perhaps, there's some new code perhaps it talks to some new services, you set up a different database etc, then.
We. Remove the shipping part of our model if the gap and repeat. For, the final service the accounting, service drop, a new binary in set. It up with the proxy. Implement. The logic remove. From. Our monolithic, app and, now. The, tree has been entirely consumed, and voila, you're. In microservices. Thank. You. Just. Kidding that's not the end. One. Thing to be. Aware of is that there's. An anti-pattern called. Data-driven migration. What. Is that well you, may be tempted to. Begin. With the data side of, things when you start to migrate you're monolithic, app over rather than the, implementation, details in the business logic this, is actually the, opposite, of what you want to do I won't have time to go into details about data-driven migration, but, you should look up Martin Fowler's. Article. About this it's if. If you're curious why, read. That argh. So. When, we split up lo, hee for, example this ended, up being this process of using the Strangler pattern ended up being pretty straightforward, because in our original monolithic, app we only had, a, few. Modules. That all already. Were sort of separated out into their, respective domains so, each module, just, essentially, became, a new. Micro, service for. The most part there, were a few exceptions which I'll talk about. One. Of those exceptions was, the pipeline, module, so, in the original Python. Implementation of, of loci, the pipeline. Module was responsible, for storing the pipeline configuration. Managing. The state of a, given pipeline execution. Managing. The stage types catalog, and, it. Contained. All the implementation, details that needed, to run. Those actual stages. It. Was sort of like a Frankenstein. Module, so when we when, we were moving. To this architecture we decided to split it up so, instead. When. We implemented microservices, we. Had a separate pipeline, service that, managed the state and configuration, of the pipeline a stage. Type service to contain the catalog, of available, stage types and finally. The GCB executor. Which. Takes, the stages and, runs, them. Alright. Now. That, you know how to split, up your app. Where. Should you start. Well. A good, place to start is with source code and documentation. Raise. Your hand if you. Know what. A design, doc is. Okay. Raise. Your hand if you've ever written a design doc. Fewer. Of you at-bats oh yeah.
Nobody. Ever writes design Doc's but they're important, you, should write them. You. Can use design Docs specifically. To create a strategy, about, how you're gonna migrate what. We did with low Heon glass pen was we essentially created one doc for the final architecture, design and then multiple, Doc's one for each micro-service, to talk about the entities they were responsible, for the related business logic and data layers and so, forth, also. One thing to note is you can establish clear boundaries between them about, the contracts, between your services, it's. Really nice. One. Other thing to consider is do you want to split up your repository. Into, maybe, a repository per, micro service with, Lohia and gasping we decided, to do this namely. Because they. Allow us to establish really, clear dependency, relationship, between the micro services it's, pretty. Obvious in the, dependencies, when, you're depending on another service, if they're in completely, separate repositories. It also. Allows. You to pretty simply. Deploy. Them separately because they're. Separate. Finally. Set. Team, responsibilities. If you're. A, small, team maybe you have one, person who's responsible for several, micro services and if you're a large team maybe you have a whole sub, team dedicated. To a single micro service just. Depends but it's important to set up those responsibilities. So that there's, no orphan, micro services lost along the way. Now. One, thing you might be wondering is, how. Do I share, code, between my. Services, with all of them existing. In separate repositories, potentially, and being deployed as separate binaries well, the way we decided to handle this in glass pain was, by setting up a common. Repository, this. Is essentially, a repository that all the services can depend, on for shared code without, having to depend on each other or without having to copy code, from service to service, it's. Really you can feel free to use this same, strategy. So. You've. Designed your services, you've written design Docs and. You. Made a plan for your source code so. How can, you decide on a tech stack. Microservices. Can be implemented in many different ways you could create separate, app engine instances, or use a customized, compute, engine, or even, use containers, on kubernetes, for. Data heavy apps firebase. Is a good choice - you, can go serverless. Even, with something like google cloud functions, or if you're not ready to move everything to cloud just yet you, can even use tools like Kay, native to, essentially.
Run Micro services on trend, using your own existing, infrastructure. In. Low heat we decided to use gke. Gke. Allowed us to deploy a wide variety, of applications using. Containers. Essentially. If you can container as it Jiki can run it. It. Also gives you nice things like self-healing, services. In the ability to auto scale them as needed. And we. Get declarative. Configuration, out-of-the-box. With. Also, easy, network. Setup and automatic. Upgrades, to the latest kubernetes versions, and. Gke. Also integrates pretty easily with other GCP services. For. The back-end framework, we chose Java and spring in part, because our, new, team once we Don boarded new members was more familiar with Java, spring, is a pretty mature framework, used throughout the industry it's. Open source it. Was built to run in the cloud and it's, as flexible as you want it to be you can configure, things yourself or, let spring configure, things for you and there's tons of documentation. Online to play around with so, it's, easy to get started. Specifically. In Loki we took advantage of these, two technologies, designed to easily integrate spring, with gke the. Spring cloud GCP project, enabled, us to integrate with other, api's, and services on GCP out-of-the-box and spring, cloud kubernetes, allowed, us to easily consume native, kubernetes. Services. Both. Low heat and glass pane also use this, front-end. Framework called polymer, for. Our front-end UI and you, might be wondering why am i talking about a front-end, framework, you, know talk about micro services well palmer, takes advantage of something called web components. Where, each piece of the UI is. Essentially. A separate, component with isolated, responsibility, so, we can think about our front-end in the same way as our back-end. Take. The navigation, navigation drawer. In lo he for example as you, can see each element, is. Actually, composed. Of, its, own small. Independent, reusable. Web components, just like the, services, that we've been talking about I encourage, you to check out the polymer project and the lit project if this, sounds interesting. Anyway. Pardon my digression. Where. Were we. Well. I mentioned some, of the benefits, of micro services, showed. You how to migrate, your app using. The Strangler pattern and I told you how to get started with design Doc's source, control and tips, for choosing a text deck. So. Our, we've done. Not, quite imagine, you might still have some questions like, first. Of all how, should my services, talk to each other, what. About flags, how, do I configure, my services, and with. All these requests, bouncing, around, won't it be harder to debug. How. Can users, and clients, access. My services, if they're all running separately, and what. If a service goes down, is. This better than a monolith in any way, well. First calm down. Geez. I'll. Get to it all right how. Do. I expose my, services, to, my users clients. And. Anything. Else that wants to consume my api's. Let. Me tell you about two patterns, that, you can use to simplify the interactions. Between your, services gateways. And aggregator. Or composite, services, as. The. Name implies, an API, gateway, acts like a door to your application, essentially, the, Gateway decouples, your clients, from, your downstream, services by, ascending, API, requests. To the appropriate, services, without. The user needing to care where they end up you. Can also use a gateway to aggregate, calls, and configure. The optimal, api's for different types of clients, or users, you.
Can Even do things like throttle, requests, to, prevent overhead use retry logic when a request fails and set. Up perimeter security, to. Block users. From, accessing some, services, that you might not want them to but please note perimeter, security is not the only type of security, you should have in, your application, let's. Keep that in mind. In, spring, bhoot turns, out setting up an API gateway, is as simple as adding a single dependency, to your application, now, you're, able to match routes, with api's, add. Predicates. Sand filters to your paths and even. Throttle, your api's with. Rate, limiting. Excuse. Me in, addition. To setting up a gateway you, can also create services, that aggregate, calls to. Other services, this, allows you to simplify, your calls between. Services. And implement, additional logic like retry, handling, or exception, handling. This. Is really nice for when you want your users to be able to perform complex, operations. In a single, request without. Worrying about the implementation details of the downstream services so. For example Loki's flow service is a composite, service, that manages, flows and, when. A loops and when, a user wants to create a flow the. Flow service talks to these other services, to. Create new pipelines, notifications. And triggers so, that the user doesn't have to this. Allows the, flow service to report exceptions, also from the other services, back to the user and to retry. Executing. Pipelines when they fail for, example. But. Speaking of retrying, things when they fail is. It possible, to perhaps, configure. The. Number of retries that, I attempt, without. Redeploying, my service you might be asking what's a good way to configure, my services. After. All the. Services might have a bunch of configuration, applied to them just depending, on the environment that they live in right. Things like. Retries. Feature. Flags. URLs. And ports you, might have pub subtopics and time outs that you want to configure do. I have to reboot, the service every time I, want, to change something, turns. Out no fortunately, spring cloud kubernetes allows us to apply these sorts, of changes to our kubernetes. Configuration. Map which then delegates them, to our already deployed, services, when. We need to you, a configured, variable, in our source code we just used the add value, annotation, in spring and. Whatever change we made to its value just gets automatically, applied. Nice. Since. Glass pane doesn't use spring. Though, we, didn't have this cool configuration. Management out-of-the-box, so we basically decided to implement the same functionality, ourselves, we, created.
A New service called the configuration. Micro service and, essentially. It listens, for changes, to a protobuf, file in the configuration. Repository. And whenever. We make changes, to that file the, configuration, service to text them re-emits. Those changes, via pub/sub to all the services that need to listen to those events and those, services can just pick up those, configuration. Changes using a similar annotation, we created in Java. Alright. Once. We have the. Services running. How. Do, we make sure that we can carry out really complex, operations, in our application, across. All, these different services that we have how do we coordinate, them. For. An example imagine, a user wants, to delete, a, project, in low heat in, response. To this request the project service should delete a project, simple. Right. Well, not quite. Remember. That in low key flows are composed of other things like pipelines. And and and flows and triggers that, shouldn't. Fire once a project is deleted, otherwise these orphaned, flows and triggers, might rise from the dead like zombies. Spooky. And what. If also, maybe environments, need to be updated or pipelines. Need to shut down and what, if in the future a new. Service. Needs, to know when, a project gets deleted what if we're maybe collecting, metrics about when projects get deleted or something, how. Can, we make all of this happen, atomically. So. That. Operations. Don't, affect each other in ways that they shouldn't and, everything. Still happens as you expect even though the logic is spread so, far across your micro service, furthermore. We. Need to be able to roll back remember if something fails along the way. Thankfully. Saga. Patterns, exist, and, they. Are designed to handle exactly. These types of complex transactions, while ensuring that operations, do in fact happen, atomically, let. Me tell you about two of them, orchestration. And, choreography. In. Orchestration. As the, name suggests you, essentially have a single coordinator, or director, service, it maintains, the state of the system it. Tells other services, what to do and if something breaks it makes sure's to handle it there, are downsides, to. This centralized, approach though. You. Should consider, like. Potential, overhead, for, complex operations, because you only have one thing managing, all the state. In. Choreography, on the other hand services, produce events that. Other services, react to there's. No single, service coordinating. The actions all. Services. Are and are responsible, for handling their own state by subscribing, to whatever topics, they need to. Below. He we use both in fact, for, example we use a fan-out approach, to omit events from the project service when a project gets deleted through. Eventual, consistency other, services, listen, for, that event and react accordingly. We. Also use a fan in approach, when a single service is interested, in events for multiple services the. Metric service for example uses this technique to collect metrics. By. Listening, for events from, multiple, services. Handling. Transactions, in any type of application, can be difficult. So. Just, carefully, consider them to make sure things work out smoothly and can be rolled back safely. Ok. We covered a few handy saga patterns for, coordinating, your services, but, as your, application grows how, do you keep track of requests, being sent between your services when you need to debug them for example. Distributed. Tracing, is a, really nifty technique, to do this you. May have heard of a, a tool called Zipkin, it's. A well it's a protocol, actually it's, a protocol spring uses, to, add breadcrumbs. To HTTP. Headers. Automatically. Which, allows you to track requests, as they bounce from service to service, as. An alternative, there are tools like envoy, and Sto, which I'm sure you've heard of around. The, conference that. They create a service, mesh to, essentially. Enhance your HTTP, requests between services on any tech, stack to, provide your application, with distributed, tracing capabilities, out, of the box.
To. Implement distributed, tracing in low heat all we needed to do was add these two dependencies, to our project after, this our micro, services began, pushing, information about, requests, to, stackdriver and we, can even watch them live as they happen so, back in our DCP console we're able to see requests, bouncing, across our services, and simply, by clicking on a specific request, you, can directly, view the the logs, for the service that generated, that request, it, makes troubleshooting, errors, relatively. Easy. Okay. So, I've addressed. Some, of your concerns hopefully, about how to expose, your services, with, gateways. Configure. Them without. Redeploying them. Coordinate. Them with. Saga, patterns but, there, is still a couple major common. Concerns, that I want to address before, I, end this presentation and open up for questions that promos will get there or almost through, namely. You might still be wondering how, do, services, find each other and when. I add a new micro service how, do the others know it even exists. Turns. Out there's, a name for this service. Discovery, or discoverability. For. Services to find each other you typically use something, called a service registry. When. A new service is created it registers, itself, with the service registry to say hey I'm a service. Hello. When. One service, is interested in talking to another service then it just goes to the service registry and asks, it which services, are available and. Thanks. To spring cloud in and, gke. This, is relatively simple here's, an example of how we use service, discovery in low heat to add a new type of trigger the pub/sub, trigger - the trigger service we. Implemented, this new, pub/sub. Trigger we're. Able to just drop it in the cluster. Then. That. New trigger service. Registers. Itself with the service registry in gke. The. Trigger service picks. Up that change from, the service registry that all the services can talk to. Now. Whenever. The trigger service for example needs. To ask what types of triggers are available it, now, can, interrogate, this new service and say hey are you a trigger service and if. It responds with the expected request now. The trigger service is aware that the pub/sub trigger, exists, and, voila. You now have a new, trigger, without having to redeploy anything. Alright. Finally. This. Is a question we, get a lot what. If a service, goes down right. What, should you do if one of your services fails, unlike, monolithic. Apps a major. Benefit, of micro services in having, separate binaries, is that. Your application, can continue. To function, right. Instead, of crashing, when, one piece of it fails, that's, really cool and. If. You've been paying attention, so. Far in my talk. You. May have noticed a pattern, and. You. May be able to guess what I'm about to show. You next. That's. Right more patterns, yay. Okay. Alright. The circuit breaker pattern, and self-healing. Services. The. Circuit breaker pattern is used, to detect failures and allows, your application to react to them. Intelligently. So. This is what the flow service and lo he looked like before we implemented, the circuit breaker pattern, in, this, example the user wants to start a new flow, which, should execute a trigger by talking, to the trigger service, but. What, happens when the trigger service goes down. When. Faced with an, endless, loading, screen the user might start clicking the button furiously, saying, create a new trigger great a new trigger what's happening why aren't my triggers being created well. Currently. There's. No way for the user to know something. Down the line failed, and. They're. Stuck what, do you do well. Let, me, show you how we, can improve the user's experience. By. Implementing, a circuit breaker adding. It to the flow service so, when everything is functioning normally the.
Circuit Breaker does nothing, and it just passes the users request along. To, the trigger source and the users happy but. Again let's, say the trigger service, goes down, for, some reason, now. The, user sends a request the. Circuit breaker may. Retry, a few times and, then. The. Circuit gets opened, when, it realizes hey I've tried a couple times and nothing. Happened we should do something else there. It reverts back to some, other logic or shows the user a useful error like hey the trigger services down come, back later. To. See how this might look in code I'll show you how we implemented, the circuit breaker pattern in glass pane so. The code you see here was actually before the. Circuit breaker was added and as, you can see in that last line of code there this service will just throw an error whenever the metric service is down it's. Not terrible, don't get me wrong but we can do better. To. Improve the, users experience we. Created, our own app method. Retry, annotation. Which, acts like a circuit breaker as you can see this, allows us to configure the number of attempts as well as a fallback method. To. Revert, to a different behavior when it fails. So. When. This, time when the metric service fails we actually go to the low. Memcache, and fetch metrics, that we've already collected in the past so, that we can still show the user some. Useful. Information even, if the metric service is down instead, of just throwing an exception and, best. Of all the, sanitation lives in the common repository, I told you about so all the services can use it without having to rewrite it or depend on each other. Another. Useful thing to have is, self-healing. Services. Wouldn't. It be great if your, services just rebooted. Automatically. When, they failed or maybe reverted. To like a previous working version when they when they went down well, turns out this is actually possible, kubernetes, for example is able to monitor, the health of your your. Application, and find out if it's down and. If it is kubernetes. Will try, to spin up a new instance of, your service, or, maybe you can even configure, to revert to some other previous, working version. But. In order to do this you, need a way for your application, to report, back its health, and.
It. Turns out. You. Can also. Sorry. You. Can also do that so in low he we use spring, actuators. To do this the actuators, they, essentially just add additional, endpoints to report the health of your application, back to kubernetes and if. Configured correctly you, can even take advantage of other tools like micrometer, to gather statistics about, your services. Like. Uptime, and your overall application health. We. Just covered. A lot. Using. A service registry, for service discovery the, circuit breaker pattern when services, go down and, reporting. The health of your services, but. What does all this look like in code well it turns out spring, has these wonderful things called feign clients, that, can basically do all of this out of the box with. Only a few lines of code the code you see here is is how we set up the api's for the store service register. With the service registry implement. Method retries and report. Back to the services health and status. Pretty. Crazy oh. Yeah. Oh spring, uses these sort, of naming conventions. To make all this happen in the background. It's. Really cool, so. Last but not least, how. Could we forget everyone's, favorite. Thing tests. Array. You, may be wondering how do I test and deploy my. Services, safely, and securely, there's, also ways to do that and essentially. I'm gonna sum it up in one word. Automation. I. Don't. Have time to go into detail but use. Existing, si ICD, tools and frameworks, such, as cloud build for example to continuously. Test and deploy your services, automatically. If. You're, willing, to upgrade, your monolithic, app to micro services you. Should be. Willing to use modern testing. And deploying. Infrastructures. Just. Putting. At it your. Life will be easier in the long run if you do all, right so there's a lot we didn't cover, you, should have a separate. CI CD pipeline, for each micro service for example, have. A rollout strategy. And have a rollback strategy. Decide. On a strategy, to migrate, your data and, again beware. Of the, data driven migration, pattern again. Martin, Fowler is is a really good resource for not, just this but also for micro services in general. Set. Up error reporting, logging, and metrics for your services, and don't, forget about security and testing. Finally. Use existing tools as I mentioned to automate, your infrastructure, as code or, configuration. So. What. Can you take away from this presentation, well. Earlier we. Talked about some of the pros and cons of monolithic, apps versus, micro services and how. Micro services came to be I told you why we initially built low key and glass pane as monolithic, apps and why. We decided to migrate them to micro services despite. The refactoring costs. Keep in mind it is a costly process potentially, right but. Micro, services allowed us to iterate quickly into, focus on the business logic once we'd handled all of that migration. Throughout. The presentation, I showed. You how to take advantage of some, existing patterns, like the strangler pattern to slowly break down your monolithic app saga. Patterns, to enforce atomic, operations, and the, circuit breaker pattern to handle when a service goes down, we. Used examples, from Loki and glass, pain to show you how this can be done using different text attacks as, well as some anti patterns to avoid. Ultimately. Though there's no one-size-fits-all, approach, for, this. The. Perfect application doesn't exist but hopefully. You learned from our experience, with Lillian glass painting and took away some useful tips for, how to choose a good architecture, for. Your app and I. Hope. The. Road ahead is a little clearer now and I. Wish you the best of luck on, your journey. Special. Thanks thank. You. Special. Thanks to these folks without whom this presentation wouldn't, have been possible. Alright. Now I think we have a couple, of minutes maybe, five minutes to open it up for any questions from. The audience oh. And. Don't forget the survey, app. Tell. Me if I did a bad job or, not oh. So, let, me start by saying that, you have done a good job yeah. All right let, me start by saying that you have done a good job, no I explained. It quite nicely thank you the, question. That I have is of course everything, originated, from service-oriented. Architecture. I am, from. A whole school SOA person, you know and I can see lots. Of similarities, with this away for example you, have an aggregator, or whatever. And if the aggregator goes down yeah you, have essentially lost all the services behind it yeah.
So There are pros and cons to every approach and. Whenever you want to decouple, having. A pub/sub in between or having a queueing system for example to. Ensure that, you you, know when, the circuit, breaker breaks it, essentially, starts flowing into a queue and then things get processed, later. On but. Is there anything within the, API gateway, pattern for example where. You, don't use an aggregator but the Gateway itself does all this for you is that is, there a functionality, like that within Google. So. You can do essentially. That would be I, think what you're describing would, essentially be. Orchestration. Where you have some. Centralized. Place that's responsible, for for. Handling. All the API requests, that are being sent throughout your application is. It is that what you mean so. Yeah. So the orchestration, pattern is is is, pretty, useful, I mean it does have some. Disadvantages like if you need to if you need to carry out really complex operations. It might take it a while there might be some overhead, because it's just one thing and. It. Has the nice caveat, that that, if you're not using, orcish. The coordination. To emit. Events across, your services then you don't have to worry, about like, where. Are all my events going when they're just you know bouncing, across all my services you don't have to worry about that as much so, yeah orchestration can be can be nice for that. Any. Other questions, Oh looks, like we've goin back there what's. Or either. One whatever, sorry. Be. Sure to get to you after. Hello. Regarding. Testing so. When we move from, a monolith to. Micro-services. We, changed. The way of testing in general it's, not just, about the. Business logic is, also about the communication, between services. What, kind, of yeah yeah because what, kind of approaches, what kind of part on DDA's when. You had to test. The. Wool picture, the, entire picture yeah, that's. A really good question so. There, are several different, types of tests, that are, really useful too right there, are things called unit, tests which, are you, know just, testing the business project and maybe, if you're familiar with other. Types of tests like like, and to end testing, as. Well as integration tests, and those kind of things so if you're if you're doing if, you're creating. This micro-services, architecture, you do still want to have those types of tests where you have integration, tests that actually test the, interaction. Between two services for example or end-to-end, tests, where you where, you say if a user requests, this and this, whole series of events happens, through the micro services, that. You ensure that everything along the way happens, as you expect atomically, like all the things that I mentioned in the presentation as. For. Patterns. To use this there are there there are existing, tools that.
Will Help you test. Across. Your micro services architecture. But. Essentially, you can think about it as testing. Any other application, that might need to send and receive, API. Requests, so, like HTTP, requests, or or if you're using like gr PC or something you can test those endpoints the same way you would with any other application. Does. That answer your question yeah I was thinking more about. Sometimes. Some, people can't. The mistake to not test. Entire picture you can end up in having, that timeout, cascade, right. Yeah. Well, that well that's. One thing that continuous. Something. Like continuous, testing. Will, will handle right because if especially if you have multiple staging, environments, you want to make sure that when you're deploying changes, or feature flags or whatever configurations, you deploy them to separate. Staging environments, like you you still want to have QA, versions, of your binaries, or, dev. Versions, for example that, you can deploy across so that you don't end up just deploying straight to prod and then waiting to see if everything fails right so yeah tests, are still very important. Thank. You it's. A good question. All. Right maybe we have time for one more or, Oh looks like we've got to wrap it up maybe a quick. Is. It quick okay. So. When. We decided to break the repo. How, would the follow up on the reporting, and other things would work do we go back to individual, services, once again to bring all the data together. Oh. You mean you mean doing like collecting, metrics and reporting errors and those kind of things well so that that's you. Can use something like the the, fan-in. Approach, that I that I mentioned earlier as part of one of those those, saga. Patterns like the, the coordination. Pattern. Where essentially, like, we had we had the same exact service in loci where we needed to collect metrics and and and, report, errors from multiple services so essentially, we'd listened to events from a bunch of different services and display, them all in one UI so, you can you can do that using the fan it's called the fan in approach of the. Coordination, pattern, thank. You. Happy. Yeah have it out. You.
2019-12-11