Azure Service Bus and Logic Apps for messaging and process automation
So I'm going to talk to you about service. Bus logic, ups and, messaging. In general today for a little bit, if. You make it to the end I have a pretty cool demo which shows actually, an end-to-end business, flow. Very. Used event, ups view stream analytics service, pass and logic apps and. More. And. Besides. It for, the rest of the time we'll talk a little bit. Through. Okay how, can you use these tools when to use which of these tools we talked a little bit about the different messaging services, in Asia. Give. A quick little overview about, service bus. So. Let's jump right into it so. Well. This slide needs a title. Anyway. So. These, are the four messaging, services, we have in Asia. And. I. Want to give like one two sentences. To how, to use each so my, person I'm service bus program manager I am working pretty. Much I'm sitting next to the event great program, manager, and to. The event ops program managers, we're all sitting in the same room and we're. All working on the same product and one. Of the most common question we get is hey. When, should I use what right. So. And. The. Answer I typically give on that is think. Of the intent, so. What, do you intend to do storage, queues there for example made for purpose, which. We call like task queue and that, means like high volume processing, you, have like 10,000, VMs you want these, 10,000. VMs to run all the same jobs so, you may want 10,000, queues and you want to send a message in each of these queues and you want to basically have. These VMs pick from these queues and process these jobs so, they're typically like simple, task queue scenarios, then. Service. Bus is really like an enterprise, message broker. It, features, certain things which you find nowhere else which that is sessions. That is transactions. And. We'll go through a few more features. Like. The Farrell ordering, so. Service. Bus is really whenever, you move business. Data from left to right we say physical. Goods or money. Or, you, have any other very business critical transaction. Typically. In service bus. We. Would say like is its, geared. More towards data transfer, than simply, venting. Then. Event, ups does really big data streaming, so. Imagine. You create a game you, call it halo and you. Have like 4 million gamers or 10 million gamers and all. Of these gamers send you tons. And tons of events, this, is really when you want to sing if saying event hubs so. Lots. Of virtual, machines. Lots. Of webpages, maybe. An online role-playing game or anything like that sending. You events, and then later, on you want to reason about these events. Event, obsessed. Is. Really. Like a high volume ingestion. Funnel. In pipeline, that's, really what is made for it's it's really not made for for. Like business, transactions, or any of that right so. And. Then. The new kid on the block which arm, is gaining. Momentum Mary Creek is event grid and event. Which is a little bit the opposite of event apps it's actually, a high volume fan-out, so. Something. Happens on one of your services, and, you want to tell the whole world about it so you will say I. Uploaded. A new image to a storage account now, I want to tell a logic app to trigger. Cognitive, services and analyze that image I built that actually once and. What people like stuff like that or, at. The same time though I want to tell. Stream. Analytics that, someone uploaded a new picture and I, put this event into an event hub and at the same point in time I'm on a trigger function, which. Is doing something this image like copying it somewhere so, you see like it's a high volume fan-out, scenario. Yeah. The. Key when you think about the event crate is. In this, one sentence, we have down there across cloud reactive.
Programming So. The goal of event growth is not to stay only as a it's. Really going across cloud and we're making quite good progress with. Working with Amazon with Oracle with Google on making. This actually across cloud, protocol. Service. Passing logic apps now. What's. The difference there's a logic apps if, you use it has many many many connectors, and. Very. Often you might ask you, service. Bus here right and in this in this. Constellation, so logics. Up logic, up strengths, is really, transformation. So. Taking. Data in modifying. The data and then, passing. The data along service. Bus is genuinely, not doing that actually. None. Of these services, I've shown you here is doing, that. They. All just takes a message and they. Transmit the message that's. All that they do. So. When, you think snow dispersing logic apps together. Think. About message, transformation. And. Message. Sent receive so. All. Right some, thoughts on surveillance several is an awesome, concept. And. Very. Few say well the, service world why would I still need messaging, right well. Truces, you. Need messaging, in service, because. In reality resources. Are finite all. Resources. Are finite at all times meaning. Yeah. Theoretically as, a function, scale infinite, logic. Upscale infinite, but, reality is you always want to transfer, state you. Always want to have, reliable, communication. And you, want to decouple your services, and. This is where always there will be messaging, services, so so. That's that's where you think of it, in the context of server less you always need a reliable broker, some in the middle if you have many many services talking, to each other. It's. Talked a little bit about service bus. Service. Bus has two, entities. The market terms channel. Which. It uses to ingest, data and then. To basically, transmit, data one, is the queue that's, a very basic. The very basic channel in, a queue messages. Are ordered by arrival, they're time-stamped. And a. Queue attempts. To, do FIFO, but. In reality it, does not because, it's highly parallelized, for performance, under the hood so, but. You can basically set, it up to do a first-in, first-out, ordering.
And Many, services, actually needs it like for example if you're an interest company, and you want to. Automatically. Create insurance. Claims. Then. You might need something like what we would call in messaging a convoy pattern meaning. You, send a message in and you need like five six messages one is a picture, of a car accident, one, is from, one filled out one is the title of the car so, and. All, of that needs to arrive in this in the right order so your your next process who is actually come creating. This claim is, able to process that correctly, right so, this would be one sample where you would need FIFO there's, many more sample so you want to for example apply, a job on multiple virtual machines and they need to apply the norther again, you would need FIFO so. So. That's. One concept service bus would support here then. Service. Bus has two basic delivery modes these. Delivery, modes are pulled and then. Pushed by event grid so, pull. Is classic polling meaning, you have a loop which the loop calls to the server multiple, times and. Checks. If there's new messages, then. We have a concept we call long polling meaning, you open a request to the server. The. Client automatically. Keeps the request open and then. After, some yeah. If, any, time during this let's, say 60 seconds, what, your request time is a message arrives the. Broker will right away give you that message so it's not, a loop in this sense it's. Just like a ping you, keep the you, keep the connection, open and the, program will right away return you the message as, soon as there's one, the. Other model is push, B, right away right, now is supported, only in the banker in our premiums cube sorry and. We. Move it to us downloads q2 and that's. Basically like you have a queue, has no permanent, listener, and, event. Great basically, whenever there's a new message we'll say oh hey. There's, a new message pick it up and then, call like a service and then say hey pick this message up so, that's, coming for a standard skew in the next few months right now it's only there for premium skew. The. Other. Entity. Or channel we have is the topic. Topic. Allows you pops. Up set up where. You can have. Yeah. You, typically have one publisher, and the multiple, subscribers, and. Interesting. Is that each subscription, gets, a full copy. Of the message, so. And. Another. Powerful feature when, you use topics, and subscriptions, is we, have quite complicated filter, criteria you can use so. They support like a sequel like syntax and that's. A little bit unique so, some of the other products, they maybe have pops up but they don't support quite, a complicated filtering, we do with service pass so. You. Can do for example a scenario like you have to 300, retail stores and you. Want to actually. Update. The assortments. In the retail stores and now. You only want to update the resort mints for 50 retail stores right so you can grouping, so filters which. Will address the messages, to the right Beatle stores so, that would be one one of the features and actually, one of the new, quick. Starts on, our documentation. Tutorials. One, of the tutorial shows you how to realize, that so it's like we. Build a little sample which does exactly that. Then. What's interesting about the. Subscription, and about the topic is they're, fully compatible with secure, so they allow you exactly, the same features imagine. The topic like your inbound queue and then they have a process which is sending. The messages to the subscriptions. For you you don't have to do anything for that and then. The subscriptions, are really like individual, cues. So. Let's. Go very quickly over. Some. Of the features we have in service bus we talked about queues and topics we. Have that, lettering, so whenever you receive a message, you try for 10 times fails. It goes into the dead letter Q you never lose a message you, can pick the message from the dead letter Q. Forwarding. So you can define forwarding, between multiple, cues deferral.
Means, You. Get a message Bureau. Client, realizes the message is out of order it puts it back. Sessions. That's. Mostly to guarantee FIFO or order to realize, scenarios. Where you need many many many small sub cues and sequences. Of messages messages. Batching. Or long polling I explained the long polling a little bit that was the 60, second call batching. Is is. You. Can send 100 message and a Goten message in a go you can receive 10 messages ago, that's what's behind that. Beast, report transactions, so if you want like an all-or-nothing. Sending. To one or two queues or receiving, from one or two queues and you say hey this. Must happen otherwise, I want to roll back we support the two it's. Not officially, announced but the. Client is checked in I actually wrote the documentation, and everything it's. Going through the check-in process right, now and through the release process should, be there next week. Standard. 2 premium migration, that was a long term topic, so we, just created. A new way of standard, premium migration, it's an hour to click migration, or to. API call migration, so in the past you had to copy your entities, it was quite clumsy now. It's just two clicks basically, off to, IP API called, scheduled. Delivery is what it says it is you. Basically send a message and the. Message arrives, you. Can say it I want this message to be in the queue only at this time filtering. In actions we spoke about when we talk about topics, you can basically data mind which subscription. Gets what message then. You can delete subscriptions. If they never get messages, that's. The orderly, duplicate. Detection, is a filter, on the message ID and. Some. New features we have is Arabic and a massage support, and the. Geo. Data recover, geometer data recovery. Yeah. Then. Some, of the integrations, we integrate tightly with Azure event with logic ups functions Allen, Dynamics 365, for sales and. We. Support multiple dev platforms. Which is, Java. JMS, Cupid. Dotnet. Dotnet standard. So. Biff. - scuse we, have a service bus standard and, we. Have service bus premium so. Service both of them are basically am coupon zero, supporting. Sisupala, nqp 1.0, and HTP rest. Both. Of them do brokered messaging. That's. A little different to the preme difference to the premium SKU, on. Premium, you have your own dedicated resources. Meaning. Performance, is all the entire always, entirely predictable on standard, you don't have that young shared resources, we. Obviously need, throttling, to prevent noisy neighbor issues, sometimes. Rattling, doesn't work very well you still get noisy neighbor issues. So when getting out with service whenever you want predictable, repeatable performance. You, probably want to choose premium, and. Most. New features like geo. Data recovery or metadata recoveries they're also on premium. So. I. Made. This slide to compare a little bit and talk a little bit about migration, so. Migration, is a two-step process you would be on standard you want to say oh it's. Done that there's not fast enough for me anymore or I need made. A data recovery or any other cool feature there, it's. Basically right now in PowerShell to API calls one is, called. Showing entities, what it does is it, just started, copying your cues and topics to a new name space and then. Migrate would be, you. You call migrate and then all it does is basically renaming, the standard namespace and, swapping. The connection, strings so, it's, seamless your. Client but right away start sending, to the premium namespace. Yeah. How. Does this all met and serving us quite, a lot service, bus is a generalist, tool the, other tools I've shown you before when, you've seen these for messaging services are very specialized. Service. Bus is really good for been, ever using decoupling. State, transfer or. Reliable. Data transfer. All, right. How. The service, bus worked with logic apps quite. Well so we spend lots and lots of time to actually. Have. Service bus like work tightly with logic apps so. Um, any. Concept, in service bus literally everything you have seen on this list. It's. Pretty much supported, by logic apps so. Much. Logic ups connects. To many many many applications I would dare to say that the service bus integration, is probably one of the best behaves. There and that's. Mostly because we see these tools together so if you would assume. I want this talk and Azure then. Assume service bus and logic apps that's, really how you should look, at it. And. Now, the, little promise demo. So. What we do here is we simulate. Some credit-card, transactions. And. You. Know how fraud detection works, basically it just says any transaction, which comes in if I get a transaction, roughly. At the same time at a different place in the world. We. Any. Of these modern the system say oh there might be a fraud so as I inform, the user and ask if that is valid or or. I right, away maybe block it but that's basically what you would do so.
Being Dressed this data into event hubs event. Hubs basically, clear stream. Analytics consumes. That data runs. Basically processing. It notices, all there's a difference and then. Sends it to service bus whenever there's a fraud detected, service, person, gets, that. Data into a logic app the logical processes, and says Oh inform. The user or do something with it right so that's the scenario. Let's. Go here. This. Is just my event producer, a little bit of code is doing really nothing but sending messages, to event ups and it's, simulating frauds whenever you see a yellow line here then one fraud was basically, sent. Like. This and. Then. Here's. My stream analytics job, which is basically it, has the inputs data inputs, is nothing else but an event up here. And. Then. The output data is nothing else but a service bus it only outputs, we never detect something. And. Then here's my logic up and this logic up is doing nothing but hey receive this messages. You. Can see the runs history, here. Refresh. That, and. We should have a few more runs so we see that right now 147. So. Yeah. You. Can see it goes through in my case I just insert, into a sequel database because, well normally, you would maybe send an email but I personally, don't or, you would send a text message or do something I don't want to have hundred text messages on my phone or something like that so I just throw it in a database leave it there. So. One. Interesting thing here is a, seder. Something we just had is fresh and back, at the booth he is doing something interesting, it's adding two curly brackets additionally, so, if you actually would want to build such a scenario I give. You a little head start I show you what, to do with it, but. All I'm doing here is basically gets a message I pass to Jason. And. Then I give it to my database and, let. Me show you this little trick. Which. You need to do if you would want to build something like that or consume anything from a say. Service. Bus into inter. Logic ops. So. First, of all service bus sends, a. Byte. Array so. All you need to do is always like a base64, to string and if. It's like a service pass message from a regular client you don't need to do anything else if it's from a si, you. Need to get rid of these curly brackets, it's adds like two additional, curly brackets so there's. A good block about that, from. Tom calc or encoded which does that which, explains you how to do it so this. Just a little tip if you ever want to build a scenario, like that well. And that is pretty. Much free like at 149, any, question we should quickly go through. Yes. No maybe. I'll. Be hanging out here for a little bit if you. Have any questions, to service bus let me know, below. Feedback, so. Please complete some, evaluation, yeah oh. Sorry. Yes, so. Let me repeat the question, so. The question was. So the question was service. Pass and event grid integration right now is premium only yes right now it's only in premium we, move it to standard gonna. Come in the next two to three months maybe six months. Depends. On engineering resources, and time and availability, yeah. So that was the end of the session. If. You, have questions I'll be here thanks.
2018-07-19 13:46
