Keynote | Maxim Fateev, Temporal Technologies
Welcome. I think we're going to get started. Thank you all for coming to the main day of Replay, the first year of this conference. I'm going to be your host for the day, my name is Charles Zedlewski, I'm the chief product officer at Temporal. I've been in the company for a little over a year and I'm extremely excited to welcome you all here today. Replay is a conference hosted by tim pearl
but it's a conference that we want to be for uh the larger developer community and we see this as something we want to do every year and of course it's going to be a lot about temporal but we want it to be about all kinds of interesting things that are developing in software development and particularly in back end or server-side software development and we think that there's an enormous potential for for for uh for the future years um and when i thought about the fact that this was the first ever conference uh and i thought about the the adoption of temporal i kept thinking about beginnings because i've had the fortune to see movements technological movements start very small and get surprisingly big in a short amount of time and that's obviously why i found myself at temporal and i like to think it's why you found yourself here and the uh so i think what's with for all of us at temporal what gets us so excited about um this event and getting to be with all of you is you know when you when you work with people at temporals the sense of just how big this all could get someday and how important it could be for uh technology and for software development generally but how small it still is right now but you could kind of see where it's all going to go and so there's this sense of kind of pent up energy of some some kind of chain reaction that we know is is getting unleashed and uh this this sort of uh this sort of event is part of that um and what i think what a technical movement becomes uh is a lot is is is as much a function of the users and the community as it is about the technology and uh one of the things which makes replays so special for us is we could not be more proud of the community that we get to be a part of um we at temporal we thoroughly enjoy the interactions that we have with the user community on slack or on discourse or at meetups and the fact that we get to do it all together here in person today is just phenomenal and if you look at the speakers we have today and i'll be thanking our speakers multiple times but uh please know that at temporal like for speakers for users we are flattered by the quality of the company that we keep and we know that it is a a big reason why the interest in temporal continues to grow is because of all of you so thank you very much for helping make this real um just to illustrate the point this is kind of the current uh adoption of temporal these are ping backs of usage of temporal in any given day since the inception of the project and the interest has been growing uh exponentially but uh that and is now we some i can't get into precise numbers let's say it's the many tens of thousands and perhaps even hundreds of thousands of developers that are using temporal on a weekly basis um but the thing is is that for most of you who are here that's not news to you you are way back over there right and what i what i would you you were sort of ahead of the curve and what i would submit and i think everyone who attends this conference and has the first t-shirt which we won't make any more of will eventually look back in this too and say oh yeah i was there i was this that spot on the curve all the way in the right is going to look extremely small in scale in another two years time um so thank you for for kind of helping set all this in motion where uh we couldn't be more excited that we get to realize that we get to realize this vision um okay enough about about beginnings um i wanted to also just mention a few logistical bits we have an exciting lineup of speakers we also have a bunch of breaks that are gonna that are going to be in between the majority of the talks we have today there's going to be opportunity for q a which is all going to be done through slido so could we pull up just like a representative example of a slido qr code for a second possibly there we go so we'll have qr codes like these for all but one of the sessions uh and i'll take i'll be reading off all the questions and i'll ask the speakers so that's the general order of the day thank you again for your attendance and with this i would now like to introduce the co-founder of the temporal open source project the co-founder of the cadence open source project the co-founder of temporal our fearless leader maxime fatiev it's certainly pretty emotional to be here it's first time first time ever i i give a keynote so bear with me and certainly it's first time we make a conference and a super big deal for us certainly when we started to plan that we didn't expect such a attendance to tell the truth it's you know first time it's you never usually they say if you have more people than your company people in the conference you already won but we have many more people i think right now so it's really cool and i remember five years ago i did my first presentation about cadence at some conference i don't even remember the name of conference i remember one thing that i prepared for a long time i come there and i get 10 people in the room and it's not even recorded the mistake i made is that this conference had parallel tracks and i put word workflow in the title and parallel track had machine learning in the title and obviously 100 whatever people went there and i've got 10 people in my room it's not wasn't that bad because technically one of the distant people was employee architect and one pretty well known sap 500 and he's still here with us so this is his crew so it means that you never know how it plays out long term um but i think i i want to just thank you all of you to be here and because i as charles said you're all early adopters we are just talking and uh just i think we'd be just super grateful to all of you so i want to talk about uh jim porter we always get this question what is it and it's very hard to describe it's uh it's the hardest problem i think we have it's not that people like our technology but explaining what it is to other people is always the challenge and i struggle with that and always struggle with that if somebody comes up the best way to describe it even after this talk come to me please i want to hear it but first i want to talk about vegetables um really i want to talk about application architecture but register was a cool so but technically think about it you write an application and it doesn't matter what technology you use what language you use whatever you do but these days especially if system is distributed it needs to kind of be useful in most systems you want to have some sort of bottom markup you have some business logic but then you need to make this business logic fault or touring and reliable and usually this reliability component is much larger than actual business logic and then obviously you have persistence because you need to kind of do make it durable and you want to make sure that if you have failure in recovery you recover from resistance in disk so it can survive machine failures and so on the funny part is that in real life do you do this you don't have them separate you take all of that and just blend it all together and this is the i might claim it's practical application architecture of majority applications these days i would call it a smoothie application architecture and this is exactly what we are trying to avoid but this is our life because it doesn't matter if you use the functions you use kubernetes or run on bare metal this technology helps you to run those processes but they don't actually allow help your application business logic to deal with failures in most cases unless just restarting processes and i think the main goal of us here is kind of get rid of smoothie application architecture and imagine situation if somebody comes and say i don't like strawberry anymore please uh your customer just comes i like banana and then you have this blender going and it's probably still going and like please replace one with another and this is what we are doing in these applications because uh it's all just mixed up and you have all your business logic and 50 functions and it's all state machines and so on and you don't want to have that so what's the solution okay solution is k architecture because you have clear layers you can separate things out and if you need to replace something with like strawberries banana you just do it and you do it in one place um okay it's all cool in theory right how do you get there so how we get from the smoothie architecture to cake architecture and my answer is uh it's actually durable execution you know like we call it workflow in temporal for historical reasons but i think i i have physics degree and in physics they cannot teach you to think from first principles and i started to think about temporal in general what is the most core abstraction we have and my claim is the core abstraction is exactly what we can call durable execution and let's talk about durable execution on a surface it's very simple it's just a runtime environment which allows code and usually caught in a general purpose programming language to execute to guarantee guarantee that is complete practically runs to the end actually completes is probably not precise because you can have infinite loop in the middle so technically you want to say it it guarantees it keeps running in the presence of all possible failures so if you crash kind of i don't know power goes out in the middle of execution after fully executed or something else failed the network event happened and so on i can talk probably next five hours about possible failures in the distributed systems but the point is that there is guarantee that this will eventually complete and it's very simple idea that you have code which will eventually complete in the presence of all possible failures but in the way that code doesn't see that it just keeps executing that but as soon as you have this property you you can do all sorts of interesting things you can guarantee compensations you can guarantee execution it is kind of i would say the next best thing after the distribute transaction and you don't want to distribute transactions right and this is the way people walk around the lack of transactions in the systems if they can have this abstraction other property of durable execution is that as by its nature all the state of durable execution is always durable persistent so practically every variable in the durable execution is practically saved in some persistent store and it's done in a way that you as a programmer you don't need to do anything about it you just go write your code you have your local variables so fields and like sometimes global variables and these things are just durable all the time which means that if you use durable execution to write your code you don't need to actually even program against the database you don't need to because in our smooth architecture all you do is load state from database update it save back load state from database update it save it back right call back loads so it's all together all the time and your code is broken in pieces here you just write normal sequential code and everything is fully preserved and i think it's very important property i think the other very important property is that just by nature of durable execution the state the any api call can take unpredictable amount of time any amount of time is necessary so if you call buzz and it takes 10 weeks fine it's okay because there is eventual guarantee that this code will return and you will continue executing and just think about it a little bit it's not something we're used to but it's very cool property and why is it i think it's cool property because it allows to do really cool things seamlessly for example you can just have an api call which is sleep and you can say sleep for a month sleep for 10 days so if you need to do something i don't know send a mail 10 days later you can just say sleep 10 days and send email and try to do it usually using any other approach like against doing smooth architecture you will watch your callbacks you will have crown job you will kind of need to tie all these things together you need to preserve state somewhere here's just one line the other interesting part is that durable execution is has and again we are talking about abstraction not even concrete implementation is a nice kind of this uh it lives in this world it has the all these guarantees but it has to communicate with external world so it uh an external world is not perfect and it can have failures it's it's normal and the the way it practically means that everything you do can fail anytime so connection can fail the external service can fail it can be human task and human can forget about it how do you deal with failures systems which don't use durable executions you do retries usually all compensations and so on the cool part here is that if you call full and you do it right for five ten days or five milliseconds you still call full there is no difference you probably hide all this complexity of free try and can be asked don't necessarily come as complex as necessary basically try exponential and so on besides a single api call we also can invoke functions for queuing it's very common to in asynchronous systems to put queues in between services and in between components and queues have nice properties you can greatly they do rate limiting they do flow control they also do retries but my point is that api to call them is still the same so that is kind of my thing is that durable execution allows you to hide so much complexity of error handling again that's exactly what i said it does allow us to take this error handling out and you as an engineer don't care it's all handled by the underlying system and i think it's a super cool property of the durable execution so just to recap a little bit this is the way practically to do this layered architecture durable execution allows you to separate very cleanly the error handling and recovery code so as durable is connected to execute any failures will not affect its execution itself and if it needs to talk to external systems and we need to deal with external system failures for retries cue any other asynchronous means we can just have an api call which takes a little bit longer and but eventually completes also as i mentioned uh we also get persistence for free practically i mean from developer experience point of view you just use your normal code you have local variables and these local variables are durable so you don't need to actually write the explicit database related code and also because it's layered you can replace one database with another without changing the line of code so you can switch from something like i don't know in case of temporal can switch from cassandra to uh to postgres and you need to change the line of application code to do that because these layers are very well defined and i think it's very important and my claim is and that's why i think we all hear is that durable execution is the best way it's the best developer experience to build resilient systems and not necessarily distributed people build desktop applications using that and in future i think we can do more of that my dream is to have durable execution every cell phone as a library but we'll get there one day but i think the basic idea is that uh this is the best the best way to develop software uh for certain types of systems obviously it doesn't solve all the problems but uh there are clearly big class of systems where it makes developer experience much much cleaner one thing i want to mention is certainly we talked about temporal via the temporal conference but it's not just one product which does that the whole idea of global execution started at the simple workflow service that it wears i i don't know if you know about that but i was technically for that service for the public release of that it were multiple iterations inside of amazon to get to the point where simple workflow is then who's cto and co-founder of temporal went to azure and he created a durable task framework which later was adopted by azure durable functions and we have azure durable function product there is also an infiniti project which builds a durable execution top of apache pulsar uh it's pretty young project and uh but i think some the idea is that this is not specific to like one technology it's a general abstraction which can be used in other places and we've probably heard about cadence and how temporal kind of grew out of cadence later and now we have kenyans project which is as you know open source project and also a company so one thing is uh stuff we started to think about these abstractions and one thing became kind of clear is that this cake thing technically uh i've saw it before in multiple places and one place i saw this kind of layered architecture was actually when people talk about operating systems because operating system kind of has the same general goal it is about abstracting out complexities of hardware complexities of some failure modes as well and given application developers uh high-level abstraction to write the applications right because first computers were just programmed directly against the metal now we write code in high-level languages but all this complexity of different type of operating systems hardware and so on is kind of hidden from us one thing which is interesting if you look around we don't have distributed systems like people talk about operating system for distributed world but reality is we don't have them probably we have systems like kubernetes but they are not problem with them is that they allow you to manage processes but as soon as you you write applications for a single machine then you say this application can run this process can run multiple machines you completely need to change the architecture of your application it's not that there is an operating system which allows you to seamlessly distribute things the same way why we don't have infinitely scalable sql databases just because you need to need to make certain trade-offs you need to do charting and so on because just general sql relational doesn't work like in if you want to distribute infinitely and thinking about durable execution what i thought is if you repli take for example id operating system an operating system operates on the level process if you replace the process with durable execution and drupal execution also name we could call durable process if you decided to but it's again it's naming but it would allow us to write see a operating system which hides completely like distribution part and error handling part and i think this is a um kind of realization which came to oh button realization that you can have kind of operating system but application layer doesn't need to know about these fillers so practically if you write it for a single machine or you run it on huge fleet your programming model doesn't change and i think durable execution is unique obstruction to do that so far i didn't see any either kind of attempt to do that because if you look every distributed system out there obviously we have things things like mapreduce for example which are high level abstraction but it's very very specialized to very specific use case kind of big data and pipelines but general programming model which allows you to write application and then you don't really think about this application as running a specific piece of hardware i think durable execution the first abstraction which allows you to do that and i i think and as you see here the main idea is the application software doesn't need to do you still will have all the problems of distributed system in the os itself but application is kind of hidden uh is shielded from that certainly it is a little bit strange to kind of call temporal the operating system at this point but i think it's more like inspiration let's think about because i've seen people oh you already kind of have product it's there it works it's kind of more or less complete i just wanted this additional maybe bells and whistles better ui and so on but i think my main idea is that we're just starting that uh kind of we need to understand what important is and temporal is just one of the existing things it is much more than that and my main point is that it's day one like at amazon they say it's day one always okay jeff bezos was telling it all the time because people oh amazon made so much progress you know it's day one we guys we're just starting we'll be much more and it always was true i don't know how he managed to do that but he did it and i think it's the same thing here is temporally today is day one it's a just uh idea it's a it's a useful idea you use it in production but i think the idea is like can we move it much much more can we have applications built on top of that can we have much nicer developer experience can we start thinking about different ways kind of to execute these durable executions can we host them can we attach them what about api ecosystem and so on there are so many things we can do with this abstraction and my main point of this kind of talk is that just think where we can move that if we execute on this vision so i wanna can switch gears from this fancy new future to today's and i just want to talk a little bit more about uh progress temporal as an open source project as a company made last year and maybe talk a little bit future a little bit some people who were here saw some of that but i think it's worth repeating so i think for me the main goal at least initially of the project was to put it in the hands of more developers and uh what it means is that if you had for example typescript developer you couldn't use temporal before before recently and right now we've got typescript sdk so if your typescript or javascript developer you're not js developer you practically can have the full power of temporal uh to uh and i think it's as nice as it can get to to get distributed reliable system the coolest part about typescript is dk i don't know if all of you especially people who use other sdks know is that closest approximation to the operating system abstraction because we run every workflow in its own container practically we use v8 isolates but the idea is that every workflow is fully isolated and determinism which is required by temporal is provided out of the box by the container which means that you cannot make no different determines mistakes inside of typescript code you can do random you can do time you can do whatever all these apis are deterministic and provided by the runtime so it's as close as you can get to the notion of the process it's not like 100 airtight yet there are some edge cases but in general it is very very cool system because it is not done by asking developers politely do the right thing it is provided by the runtime um in the future maybe we do webassembly which can for certain languages would provide similar capabilities other exciting development is python sdk certainly people ask us for typo this was one of the most requested sdks and it's still in better but please it's out there try it out play with it give us feedback because the more feedback we get faster we get it it will take less time to release that in production but we'll we've heard that some of some people already use it in production some some some some of you on the leading edge like to be there and we appreciate that but uh we are working to actually have official production release very soon and look it's nice i i'm not a big fan of python but uh no you know what it's kind of funny because uh i have son his uh he was 12 at the time and he was doing this scratch programming like visual thing and whatever and then i showed him like two pages like page of python code he's like oh it's a block he just like was immediately was able to defy things just by offset and he was able to read that code and even find a bug there and like okay then after that it was converted so other thing we did uh uh so these improvements um the temporal light if you didn't hear about that this is the ability to run temporal server as a single process including the database sqlite database so practically you just get one binary and you have full-blown temporal service including ui in in single binary it actually was created by datadog team and we are super grateful to them to do that to take because without them probably it would take very long time for us to implement and uh now it's this project actually kind of graduated to the top level because right now it's part of the official temporal io organization on github but we still collaborate closely with datadog team uh on improvements and next releases of this product so our goal is that for development single binary should be the best way to do development because it's uh much easier than docker composing all these other things um the other thing we've got we've got new ui um very wrote fully ui which we inherited from cadence now this ui more or less has feature parity and but it will allow us to do things like uh mutations in the future for example you would be able to send signals updates and so on for ui and just new architecture allows to do all sorts of cool stuff and move much faster my kind of dream is we have special ui features for them just for development because right now there are certain developments uh kind of workloads you need to click too much i want to get to the point when it's as usable as ide we'll see if we get there but now we have framework to actually uh do that other one thing we we talked about applications one thing we found is that a lot of people used one of the first use cases they used temporal for scheduled actions for cro has crown practically distributed crown and we had this chrome feature and this uh in the product which we built very long time ago and the way it was built i can tell you we looked at that and said okay we don't have resources but what would be the minimalistic implementation of the crown and we put it there and still there and you use it but you hate it because it lacks tons of features and it's hard to use and so on so we've stopped and said what it would take us to uh kind of think it from first principles can we go and look at the what would be the make tempura the best distributed crown engine out there for scheduled actions we spend a lot of time doing they designed that and now it's there it's actually in the open source right now you can use it you just need it's behind the feature flag but next release will kind of flip it you can use it so it's pretty powerful it allows us to do all sort of scheduling and pretty complex one and manage the scheduled jobs and so on so please check it out other thing which we productized recently was enhanced visibility so its ability to search workflows by custom attributes managing those searches and so on but it's not only about software i think from the developers point of view documentation is as important as software because if you get temporal and no documentation yes we have slack we have support but it's still not the best experience i don't know if i don't know if you guys know but temporal has five people full time working just on documentation uh so we treat the communication super seriously and it's awesome very strong team so uh and we did a lot of improvements application development guide is one of the recent additions and we plan to invest more and more in that for example we know that uh the documentation of the how to operate cluster how to set up cluster and production issues isn't far from perfect it's almost unexistent we will heavily invest in that uh this year and later and later in the beginning next year other thing is that documentation reference the community is not enough we wonder people learn temporal some of you yesterday attended our first 101 class and uh it looks like feedback was pretty positive we want to make sure this class is accessible through the website so we created learn.temporal.your website and you would be able to we'll have more courses which you can do on your own pace we also have in person courses so i think we will take a lot of as you know it's new technologies people takes time to grasp grasp it so we will uh we have education team which is building those courses we did some service improvements um some some of them are small but very useful like name space deletion now you can you see icd pipeline can create namespace around something and delete namespace simple thing but very useful we increase retention and we did a lot of refactoring of history service to make it much more stable and scalable and so on uh we can give you details but mostly it's about multi-tenancy so we get much better multi-tenancy support now one other thing i mentioned enhanced visibility but uh the way it is done right now it only works if you have elasticsearch we have kind of this weird situation when uh you have dual mode if you just run with one database without elasticsearch you don't get all these features so we want to have parity so we will add it has visibility to every single implementation which means that even if your sqlite or mysql postgres you would be able to get all the same features as if you run with elasticsearch and this is uh something which will allow us to make a nice ui because you don't need to switch that and it also will just allow you to actually do full development for some using sql lite because you won't have some features missing i think it's relatively small but very important addition the other feature which is frequently requested is the ability to send updates to the workflow because now you can send one-way event signal you can query workflow but you cannot mutate state but now you would be able to send update to the workflow mutate state of the workflow and update can take as long as necessary it can take milliseconds and it's a super optimized path as a single database right or it can be as long as necessary it can be it can be days and we will support uh all sorts of features around that also obviously workflows and other workflows would be also supported and um this will probably allow you to build interactive applications of temporal like you could press a button and get a next page as a response uh to the uh it will allow practice i don't know i don't know where it goes but i'm pretty sure there are a lot of people who would be very excited about that feature versioning is another kind of pain point we know about it and it's super powerful because you can update workflows which running long-running processes while in production but it's for situations when you just have short-lived workflows and you want to just be able to write your code don't think about that it's uh it's not it's far from perfect and we are working on that so we will have native support for uh allowing deploying new versions running multiple set of workers through different task queues but it will be kind of hidden from you it will be kind of done automatically so like all consumers will not care they will just use start workflows on whatever task you they are starting but behind the scenes we will do all the right thing to route to the appropriate version and also the other part of that is self-reliance we want to invest heavily into the ability to roll out code safely because again temporal is not only about how you write code it's about how you operate we want to make sure that we are the best platform for actually operating things in production so we will put a lot of effort into these features uh there are a lot of other features we want to like most basic ones like pause workflow and pause workflow like make sure that workflow there are a lot of things we want to do just to make sure that you can operate things very efficiently the other big area of investment we are going to make is what we call nexus and it's kind of code word i don't know how it will end up being called at the end but basic idea is that right now uh people you use temporal a lot to build single application and they build multiple applications but uh sharing kind of calling each other like if you have one application like set of workflows and then you want to expose apis to other applications to other set of workflows or services outside of temporal it's not easy very we have child workflow in a different name space but it doesn't work very well does it work with global uh namespaces and so on so there are a lot of limitations there so we have we want to make sure that you probably practice the best way to expose your functionality to other parts of your company or even outside of your company and this uh can be done very cleanly for very clean defined apis and and also we want to make sure that this technology works not only with temporal but people outside of temporal so you can expose http gateways grpc gateways to other applications and as part of that we also need to solve problem that industry doesn't have standard to define long-running operations because what we call a lo arbitrary line iteration because uh if you have i don't know standard grpc so http service it's it's very easy to say okay this is a request i support but then i say if this request takes five days how do i model that then you need to do web hooks you need to do long polling and there are different ways to do that cues and so on so there is no clean way to define interfaces with for operations which can take uh long longer than a few milliseconds or seconds and we want to make part that uh if if we succeed we want to make it more like industry standard which is not important specific but allows to define those operations maps and specific technologies like http or jrpc and then make them a callable from workflows directly or workflows being able to implement those operations directly it's a big area of uh there is proposal actually right right now no prop i don't know if all of you know we have temporal dot io slash proposals repo this github repository and we push every time we do any kind of serious change to the temporal we actually always publish proposals there all new sdks uh for example right now there is a ruby sdk proposal out there if you don't if you are in anyone in ruby uh that and this proposal like first phase of this nexus project is out there so please check it out comment we want to feedback it's a level of design right now but the biggest i think not complain but uh if you go to our forums or go to slack you you clearly see that people get temporal it takes some initial harm but after people understand how to write code they implement super complex applications on top of temporal without talking to us they can ask questions about specific age cases and so on but it's called you know how to program java know how to program goal we don't need to teach you that you can do it better than us and that's why i think if you look at our forums a lot of questions are about how to run temporal how to operate important especially at large scale especially with high reliability at the companies and the way we're planning to to kind of help you there is and obviously temple of the company is going to monetize this open source this is one thing which we don't want to play with licenses we don't want to go this open core model when we will kind of create this enterprise version like take some features i don't know something out of that and not let you to kind of use full version we don't want to do any of that we don't want to change licenses we want to keep mit license we want to have open source being very feature reach but at the same time we hit all the time people just don't want to run any infrastructure themselves we just want to rely on ours that's why temporal cloud is the way we are planning to help like this adoption you can develop and uh run production with that the idea of the pro cloud is that we're not going to host clusters for you we are giving you name space as a service practically the idea is that you don't know what's out there it's just uh so fully serverless namespace is a service offering on the cloud you just provision namespace using api cli ui and you get this namespace you can just start using that right away and it's production quality service and you don't need to think about clusters you need to think about capacity all of these things are taken care of and i want to just give you some preview of that uh mike i want to invite mike i was to to help there and he will give you even some demo and then we'll talk more about that thank you max so i get the opportunity to give you all that first look of temporal cloud and the new features that we've been working so hard on to make this adoption of namespace as a service even more streamlined so one of the exciting things coming up is that uh creating a name space is now going to be fully self-serviced so the design partners that we've been working with so far are going to understand how much of a benefit that is no more is zendesk tickets and so this is this is a really big deal so i'm going to take you through real quickly a process of creating a namespace in in the cloud and just as a quick reminder a namespace is the fundamental unit of isolation in temporal it's used for a variety of reasons however you want to want to isolate your services so it might be based on application team organization environment or a combination of any of these so real quickly i'm gonna go out to the interface and show you what it's like to create a namespace so it's pretty straightforward i need to provide a name which just needs to be unique for the temporal cloud account that i have and i need to choose one of the seven regions we're going into other regions and we're also planning to support other cloud providers as well i need to specify what the retention period is going to be for the namespace as a reminder this is the amount of time that workflow history remains closed workflow history remains in your storage and i'd need to provide a certificate because all the connections to the cloud are secured via mtls okay so from your workers or from your client i can optionally provide one of the certificate filters these are useful for that segregation reason reason it can simplify your certificate strategy at your organization so maybe you want to aggregate namespaces according to environment or however you want to approach that and then finally i can optionally provide search attributes whenever you are have a temporal cloud account then you have advanced visibility fully backed by elasticsearch okay so when i create the namespace it does take a few moments to appear um and so what i wanted to do one of the questions that we get that comes up is what does it mean to adopt temporal cloud how if i'm an existing temporal user how do i how easy is it to migrate my code over to embrace the cloud and uh so i want to go into that but real quickly i just kind of want to review what it's like to even adopt temporal at all so you're going to have your in your own environment you're going to have your own workers that are connecting to temporal and you're connecting to temporal services whether it's your own self-hosted instance or whether it's in the cloud so what does the code experience look like i'm an engineer i'm doing some local development i've got some things working so i'm just going to do that for you now right now i'm going to just fire up a local instance of temporal using temporal light and it's pretty straightforward a lot of you have already experienced this so i'm just going to start a workflow out at temporal you'll see some logging going on what i'm doing is the ubiquitous money transfer application so i've just transferred a million bucks from one account to the other but it hasn't happened yet so i need to start my worker and down below you'll see that the transfer has taken place and so on so many of you have already gone through this sample are similar so what do i have to do in order to change my code from running locally and run this in the cloud so this is the a typical client factory for temporal the temporal client and there's not a whole lot of voodoo going on in here this is just i have an environment variable where i can specify using tls to con to connect to temporal obviously if i'm running locally i don't need to worry about that but i want to run this in the cloud so so let me share with you just a few of the variables that i'm going to have to change in order to do that first instead of using localhost i'm going to use a address that's given to me from the namespace that's handed to me in the cloud so i have a namespace that i created earlier and here's what the address looks like so i have the namespace identifier is or the namespace was called paymentsdev and then my account name my account id is sdvdw so the namespace id then is a concatenation of those two things and this is just a typical grpc endpoint that i'm going to use as the address that i'm going to pass in so let me comment this out and one thing that i've got here in the comments is just an indicator that uh operating temporal is very much the same in the cloud as it is to your current temporal instance as well so i can use t control which is our command line client for uh operating uh temporal on the command line and um it's just i'm going to be passing in the same address into the command line there so now that i've got these variables set up this is going to be used by my client and this time i'm going to be using tls to connect so i'll specify that here and let's go ahead and start the money transfer again and i want you to think i'm lying so i'm going to stop temporal down below okay so i've just started a workflow out in the cloud and so before i go and start my worker what i'll do is pop into the workflows view it's out in the interface and you'll see that i have a running workflow this is what i just executed or just started okay so i'm going to go into the workflow here really i am right there okay so uh this is showing me that i don't have a worker running there's nothing picking off the tasks off the queue right there's nothing new here uh except i am going to point out that my access to this namespace was made available because i'm a global admin on this account this is another new thing that we're rolling out in the cloud which is role-based access controls so over here on the left you'll see a little usage tab that's only viewable because i'm a global admin you can also designate namespace admin developer and so on and this is super exciting if you've been working with the cloud already you can finally govern access to these name spaces and also the different affordances that are inside of there so that's great so now i'm going to go ahead and start up my worker and fail because i did not have my tls flag set up okay so i just deposited a million dollars from maxime's account into mine and so now i can go out to my workflow and see that yeah the workflow was completed so one of the questions that comes up a lot whenever people are moving into the cloud is you know how do i know what's going on inside of my workloads now whenever i was running temporal uh in my own environment then i had access to the metrics i had access to controlling things right all the little dot dials and knobs so so how do i know uh so there's two big improvements uh that are new features that we're rolling out that are super super exciting so the first one is the ability to go out and generate an end point down in the settings you can go into the observability panel and pass in a certificate here that you can generate an endpoint that you can use as a data source in your grafana dashboards okay so what this is exposing to you is a subset of those temporal service metrics okay so it's not all of them but it's uh it's the critical ones that a lot of your set have alerts set up on already and if you need something else then you know talk to us and and we can work with you to see what we can put in there to help you be able to monitor your own applications so you can here's a we have a dashboard that we can hand you that has all those metrics already set up another feature that we're rolling out which is thrilling for uh understanding your costs inside of temporal cloud and also understanding what am i doing right you're right we're writing these workflows we're using these primitives and we're spread across teams and maybe you want to have an overview of how many queries am i sending how many signals am i sending this is really important if you're paying for it right and um so this new usage view uh there's a couple of views here to look at one is i have an aggregated view of the namespaces i'm a global admin like i said earlier and so i'm able to go in and see an accumulation of all of these across the name spaces and here's a daily view that lets me go in and actually see bursts and traffic so you can see i have one namespace that i was just jamming i was using meru to jam a ton of traffic in and so i've got i was doing this uh all day yesterday you can see i was just jamming a lot of traffic in there so i can see what's going on through the day and be able to optimize my workflow so it's not just a cost estimator but it's also just to understand what's happening inside your your workloads and this is also available in the namespace itself so if you're a namespace admin or anyone that can access the namespace you can actually go in and see what is the usage on my on my namespace and it split up according to those primitives that's huge today it's kind of hard to tell what are the different commands you know what what are the actions what is an action you know what's going on inside of my workflows uh we spent a lot of time with our design partners trying to help them understand their costs and so this is this is a huge deal so i this is a real quick snippet on uh just some some of the features that we're rolling out you can expect to see other features like there's batch operations they're going to be available via t-control when connecting to the cloud we have audit logs for security logs and so on coming out so there's a lot of great new features that we have planned and the intent here is to get it so that you can start working on keep working on your features instead of just always dealing with the services right and so i think you'll agree this is a big step for us thanks mike i think one important thing is that uh we show you the cloud and i know that we have uh quite a long signup list so you can sign up for the cloud but we didn't implement yet the self account provisioning so once you get account you get this experience you can manage everything yourself so what we want to do is uh we will uh first we are working on implementing self account provisioning so you will get it probably earlier next year but we still we want to go through that backlog as fast as we can once we uh fully register these features uh later in september or begin in october and so uh if you are on this wait list please or you're not there please sign up uh you can find it in temple.io cloud link to that and you will get email from us right now you will get email and if you're interested in signing up please reply to that email you can fill up the online form and uh we will provision namespace for you as fast as we can i think the couple things about cloud in general is um it's very scalable we can do right now what we call up to 200k actions per second per namespace and we probably can go higher but talk to us and action is in this case is not event it's an actual activity execution signal workflow start so if you have workflow with 10 activities it's probably like 11 actions i think so it's pretty easy to kind of guess how many actions your workflow will consume and we can go pretty high rates and we have uh customers in production one thing about cloud we are showing you preview but we have uh companies uh we have over 50 what we call design partners which actual pain customers which are now platform for like over 18 months now uh because one thing to understand temporal as a product was pretty stable like data plane is stable it was production worthy for a long time just because we started uber and was production there for three years before even temporal was created so our cloud was robust and like very mature from the beginning from the data plan point of view but we spend time doing the control plan we want to make sure that we onboard more people it's all fully automated we don't want to do manual operations so all upgrades and everything all provisioning everything is obviously you can guess this bunch of workflows because workflows are the best way to do uh infrastructure provisioning so we are working on this workforce making sure this experience is as perfect as we can get and we hide all the complexity of managing clusters because we give you just name spaces and the other thing which is worth mentioning is that right now in aws only we certainly want to be in all major cloud providers this time so next we will start looking into the at least second provider we will see talk to us actually we want to know which one to go to so it's uh but and the other features we want to do is for example we want to be able to do live migration from what we call name space migration from oss to cloud without downtime and back because we want to make sure that we want to have guarantee that you always if code runs in the cloud can run back on oss without any changes and we want to actually even support live migration back and forth and then also between regions for example you should be able to take namespace me the best region one and migrate to i don't know azure region two then gcp region three and then back to your ss if necessary i wanted that demo we'll do it maybe next next next conference yeah i think this is a kind of uh end of this talk but i i just want to emphasize again thank you for being here it's a i think it's exciting for us but i believe it should be exciting for you as well because you can all adopters of this technology and i believe this technology will be something which actually can help you in your career as well and kind of in what you're doing in your companies one thing is just i had a lot of talks yesterday and uh it's interesting to see people from like the largest corporations on earth been here and at the same time i had talked to a founder of a startup which is not even founder yet he's saying i'm going to do startup and i'm working on the idea of the startup so trying to figure out what i'm going to do but i know i'm going to use temporal so i think this is exactly what we want we want every startup just assume that temporal is the default if they want to implement any kind of business logic thank you you
2022-09-01 11:58
