Enhancing the Client Experience: Citigroup Explores ML to Build Conversational UI (Cloud Next '18)
Good. Morning, everybody. First of all thank you very much for letting to spend, some of your time here today talking, about chatbots and bigquery, and data sources my. Name is reza Rockne and I'm one of the solution, architects, here at Google been, here over five years predominantly, out of the UK office but recently moved over to lovely. A sunny Singapore. Part. Of solution architects, role is to look, at new use cases new, applications, of our technology, work. With customers very closely to, actually build out some you know Minimum Viable Product pieces etc, and. Then, if we find a good match that, our next role is to do things like blogs and solution papers to scale that out one, of the best photos of my jobs is working, with some fantastic people. Like, James. Thanks. Res and thank you all for being here so. Yep my name is James tremens I work, at Citigroup as part of the. Capital markets effects trading team where, I'm head of data science there and today we're going to talk to you a little bit about how. We've combined, different Google technologies, to ideally. Enhance our client experience, for both internal, users of our products but as well as external, users as well and we'll be building on technologies. Like dialog flow bigquery. And pub/sub this. Project, originated. Probably, two years ago when. Reza and I were doing some work with Thomson. Reuters and if you're not familiar with Thomson Reuters they're a market data company, among other things and at, Citi we were interested in consuming some of that market data very. Large amounts of it through bigquery so, that we could do, different. Types of mathematical analytic. Queries visualizations. And, that sort of thing and from there we decided what wouldn't it be interesting if, we could do some more unusual. Types of ways to get at that data and surface, insights for our users in. A different way so. The. First part of this talk is to click a working great. Is I'm going to discuss, the business goals and. Also some of the constraints. Within which we had to work in order to achieve what, we wanted to and. So I just, thought I'd give you an, indication of the scale of the foreign change in local markets businesses, City because that, points. To some of the opportunities, that we have in terms of being able to reach lots, of customers in different countries, but also some of the challenges so. We, have thousands of clients from. Corporate, clients their hedge funds to real money clients and we deal with hundreds, of different types of assets we. Actually have trading desks physically, in 77, different countries but. Obviously were one very large bank and we deal in about, 160, countries. And jurisdictions overall. And we actually do about four trillion, u.s.. Dollar, transactions. Totally. Per, day. So, with, that scale comes great opportunity, but it also brings. Some challenges. As. Our. Business has evolved, electronification. Is, a phrase that we use to, describe the fact that much more, is happening now to electronic. Channels through, api's, or, even just through chat less. Is happening through phone, conversations. And. So with, that means we have more data we. Collect more data than ever before both in terms of the market data that I alluded, to earlier but also trade data and with. That if we want to do computation, on that we need more powerful, compute we need more different. Types of compute we need more storage so. With, electronification as the backdrop, we. Also noticed that our users, are becoming, more technologically, literate so.
You Know a graduate, from fifteen years ago who came out of university, or ten years ago who. May have been using Facebook back then if he was an early adopter she was an early adopter is now the sort, of person that's expecting, more from their technology, in their professional workplace as well what, they're comfortable with at home they always expect at work and I think in, many industries. Professional. Technology, relationships. Take a longer time to mature in terms of how the users interact with their technology when, compared with their personal. So, we're mindful of this and we want to make sure that we can provide as good, of an experience in your professional, working, engagement, with City as you do with your your. Private ones with your other technology, experiences, like when you work with Gmail. Now. Technology. Is used to empower the existing teams that we have and I think as, a. Data scientist I'm always very interested, in how we can use data and modern, machine learning algorithms, to, empower, the people and, the staff to better, serve our clients so, city itself. Produces. A very large amount of data and that's probably one of our largest competitive, advantages, and so, we want to make sure that we provide the right tooling to our team so that they can expose interesting. Insights to our clients, when, they need to on demand at the click of a button or in a very seamless way and, we need to better do that in a, way that's gonna scale to, meet all of the different types of customers that we have in all of the different countries that we operate. Now. As I mentioned, you know with all of the different types, of businesses that we have we have to make sure that whatever we build now, is going to fit within our existing technological. Infrastructure, so. We, have our own data as I mentioned, we have lots, of existing technology, and we have our own data centers, and our own networks all of which already exists today so, there's no way that we can build something that's entirely cloud native that doesn't sit and interrupt with exactly what we already have working on premise, so, we need to make sure that whatever we build is going to work within this and that means that we're going to have to think about the architecture of the application, and how that's gonna fit with what we have and we'll talk a little bit about that today so. It needs to integrate with existing user interfaces, we're, talking about a chat board or a conversational, UI so, we have existing tools right that our. Own staff and our clients are already working with so, we need to keep this in mind it's not just about greenfield. Projects all the time and leaving, legacy technical debt behind with you as well so we've got to make sure we deal with that and ultimately.
If We do choose to go cloud native we need to make sure that whatever we are building, today doesn't. Need to rewrite in the future. So. Just to frame it all together we're interested in looking at conversational, user interfaces, chat BOTS to, address all of the challenges that I just mentioned our scale the amount of data that we have in the way the technology within, finance is changing and the way clients want to interact with us we're. Looking at using data sets from Thomson Reuters but also the data sets that we have on-premise, that are our own and we're using technology like dialogue flow which will demo you'll ater Google. Bigquery and pub/sub to explore how we can build this conversational, UI and make it work seamlessly so. With that I'm going to hand back to res and he's going to talk to you a little bit about the actual architecture and the application that we thought yeah, thanks James so, that. Last slide of James was interesting, it had quite a few components. In there and. When, we were, planning. Out our chatbot, one thing that we you, know rapidly, came to the conclusion, of is that no chatbot, is an island so, while, a, chatbot, you can build an experience, just purely, as a chat board it's, much, more powerful in the application, well when we apply, it with other technologies. And other backends, so being, able to service the things that the users wants integrated. Seamlessly with both stuff in the cloud and potentially, on-premise. And for. That what, you need is a platform, which you know if you define a platform, it's a group of technologies that, are used to base other technologies. And applications on, top of so, what we needed was a platform that could serve all of our needs around, both, the machine. Learning aspects around dialogue flow itself around natural language understanding etc, but, also you, know getting data from other sources whether that's internal, or within. The cloud and. This. Is the current. Working platform that we put together when. Working, with James on, your. Right hand side we see some of the systems where we're drawing our information, from so, we, have bigquery, as a store, of data, and, our bigquery can support both you, know the internal. System data but also the external data. Providers, and we'll go through that in a little bit more detail we, wanted to make sure that we could work with things like spanner if we needed transactional. Activity. And pub/sub. Form that that messaging. But if you notice that internal, systems as well so whatever, we built had to work with both of these paradigms. Also. On the on, your left. Hand side for getting my left and right the. We. Have the user side of the house so this. Is the converting. What a user says into actions, we use dialogue flow for that we, used firebase, to actually get the information that's been retrieved, to, the, user if, you notice we've. Had this we built this sub hub service now, what that hub does is actually coordinate, and interact all of these communication. Channels the reason we did that is because we wanted easy drop, in and out of other components, so for example we use firebase, but. Internally, within city they have other systems that up there clients, so it, the system needed to be able to deal with two types of things very easily to with. Any any part, of this, component, architecture and actually, looking towards the future intervals, other things that we'd like to add to, the. The. Other platform, sure, yes so. When. Will you think about what dialogue flow is going to be doing for us dialogue. Flow serves to sit between the inquiries, foremost from a user and then, what, you actually want to serve is a fulfillment, request, to the user and so I'm, interested in saying okay we have all of this proprietary data we, also have a lot of proprietary algorithms, right we're a bank and so if I have a user that's interested in asking quite, a challenging question that's going to take a, quant, library that we've build over 15 years to, answer that's a very much a custom, crafted modeled approach which we have on-premise working already so, we need to integrate the two but. Of course as we move forward and as we're already doing today we're interested, in using more modern machine, learning algorithms, from. Open source libraries whether that's. Actually boost or scikit-learn or tensorflow, whatever. We're going to service. The request, from the client with whatever the inference tool that we're going to use to, to, do that it, needs to work there as well so you've kind of got custom, completely, custom stuff on-premise a middle, ground with reusing, open source technology, and then. Obviously. We'd be silly, not to think forward to the future where we might be using some of Google's hosted, model solutions, so some of the existing off-the-shelf.
Managed, Solutions, from a machine learning perspective as well. So. Obviously. There's a lot of technology, that we used in building, out this application containers. Bigquery's. Spanner or firebase, and we're not going to have a chance to walk through all of those things there's lots of great sessions that have a lot of material, on that I'm, only going to walk, through two components. One is bigquery, as our one, of our data store or our sources, and. The other one is obviously dialogue flow itself, so, just a little bit about bigquery. If you're not familiar already it's Google's petabyte. Scale data, warehouse in the cloud like. Any data warehouse it is an SI sequel it has. Ability. To build dashboards to, run ad hoc queries, it. Also has a streaming, API which, means you can actually absurd, streams of information into bigquery and when you make a call. It's omits up-to-date data and that will come in useful in, a moment we show some of the demos, the. Other, part of it is fully managed and serviced in other words the bank doesn't need to setup anything, to make use of it so, one component and, part, of the architecture, for bigquery which was particularly helpful here, is this. Idea of separation, of storage, from processing, now, what that means there's two benefits for yourself so one in terms of processing, if you're not running a query then, you're not paying for any, machines. To be, ready. For accepting. That query but, the other benefit, is around, sharing. Of data so if you think of a traditional database you have the database software that you install you, attached Sandoz disk etc to that then, you load data into it now to share data you either have to give access to the database and I used to be a DBA and that's not something I'd give very easily or, you. Need to build an application layer or you need to do the ftp dance which. Is never pretty however, with, because, of this separation what you can have is data stored in a project as long as the correct security, and ACLs. Have been done can be shared with other projects, so within an internal you know very larger organization or city they can if, they're uploading, data they can again.
With The right security and ACL levels share data amongst themselves but. Also it. Allows you to blend in data from external sources. Very easily and a, very good example of this is work that we did with Thomson. Reuters with quite a few banks, so, obviously Thomson Reuters have, a huge data set that is available through their web service API but. For pocs they also work with us with providing that data on to, bigquery for, the banks to consume, and, that gives us access to 20 years of historical tick data they, also do a near real-time so, within seconds, they can do up sets to insert. Sorry int into. Our database and again the benefits here is that once. You've been gone through the right security and the access levels there's nothing else for the bank to do other than run queries. What. I'm going to do is just do a quick demo. Of this we can go over to the laptop. Please. Thank. You and that. Is a giant screen, I'd. Better not make any mistakes okay so this. Is the new bigquery. User interface, I'm not, gonna run through all, of the capabilities, here I just want to highlight, one particular thing and that, is that we are currently in this project effects project which is the one I've used, to do my development. Testing with for, James. And, within. This project on the left hand side we can see other. Projects. That are available because I've got access to them or been authorized against them so obviously there's the the, project that I'm in but, further down we, can see this TR EMS, project now this is a project that, Thomson, Reuters owned and they. Have given me access to data. Sets within that project I'm going to just quickly go down to one, data set which is a New York Stock Exchange information. Here. We have a table I'm, just going to select details, so. They this, table is five terabytes and 40, billion rows of information now, for, me to run a query I'm going to run this query on the top here. Against. That, data. So, I'm running this query in my project I haven't moved any data around and it's working against the data that I've been authorized, against in Thomson Reuters project, and so that will take about 25, to 30 seconds, that, particular query volume, weighted average price you, wouldn't normally run it against all the data I'm just doing that to, show that we can do it against 30 billion rows takes, about 25. To 27 seconds, to complete, so. There. Was no need for me to do my tap dance routine which is very good for you. Here, we see. 63. Was the result I'm, gonna just show one other table I won't run a query against it and. That is in this, data set 2 1 3 which has. Effects. Currency. Data and just. The thing to highlight from the details if. I can get my mouse. And, move this properly this. Particular table has a streaming buffer what that means is that Thomson Reuters is currently pushing, data into, this table and it. Is being updated in near real-time within, seconds, so when a query is run against that table we get live update, the information okay. So with. That if we go back to the slides please so. That's one of the data sets that we're going to be using.
Thank. You. So. Now towards, dialogue flow so um and as I said I've been here five years and it's really great, to see all the. Icons, popping in and how all these charts. Are becoming so out of our cloud, AI. Solution. Set here we are going to be talking about dialog flow and which, allows us to build a conversation, or interface, against. When. Users ask for information. We can extract information from the intents that they have and. One. Of the nice things about dialog flow is that it is you. Making more and more use of all the ML capabilities, have that, we have in built into the product itself so, natural language understanding so, we have sentiment, analysis now, within, it we have speech-to-text all of these components, are just coming out of the box with, dialogue. Flow in. Terms of the use cases that we are going to talk about so the, one that we predominantly work with James on was around the, business to employees, so this, is helping. The traders make decisions. And. We're. Looking in the future to potentially towards doing stuff, around the customer, that. They're customers, in. Terms. Of dialogue flow itself and we'll go through this in a little bit more detail, into, a demo there is an agent you can try and think of as an agent as a sort. Of a NLU module it's, an encapsulation. Of entities. Intense. And context what. An intent is is when a user. Provides, some information like says. A sentence. The, intent allows us to map that sentence, to actions, that we're going to be taking what. Entities do so often when we want to take an action against a sentence we need to extract, information and, entities allows us to extract those parameters, a, context. Gives, us where, we are in the conversational, space so, if. A user for example is, currently asked for information about euro dollar currencies, and then, the next question is giving me news then. We know because they were talking about euro dollar that that's what they intend, us to go. And fetch and then we have the fulfillment, service which, allows us to make calls to all these other back-end, services, and with. That we're, going to go now to a demo, of the, client working within the. City environment. All. Right. So. Just before I let it run I'm. Gonna just orientate you to what you're seeing here hopefully. You, can read that so on. The right hand side of the screen or I should say the right two-thirds of the screen is an existing, internal, application. Within, City we call it the trader toolkit and for, the purposes of this demo I produced. A simple, dashboard which, comprises two widgets one is a chart widget which is in the top half and the other is a table and that table contains. Interesting. Economic. Macroeconomic. Events that are relevant for the particular asset that's being plotted, on. The left-hand side you'll see that we have if you're familiar with dialogue flow already, an. Hour of the box chat UI now. Obviously. You'd integrate this with your own chat system. If you're going to embed. This into your product so, city has some of its own technology which, this would embed too and of course anything that offers an API this could work with so. I'm going to set the demo to play. So. We recorded this approximately. Week. And a half ago I think when. We do it the 16th, to the seventh and so. The user on the left is going to ask, to draw a chart of Eurodollar so. Once. They did that immediately in real time and we haven't said anything up the client, application on the right hand side took, that information and, was able to change the chart and the top half of the screen and the second bottom. Half, of the screen updated. As well and so in order for that to happen two things needed to take place the. Idea of drawing a chart this intent, and the, entities needed to be extracted in this case euro dollar or the euro dollar exchange rate you'll. See these circles on the chart which, show the relevant, economic events, and the colors of the circles depict. How important they are and the number is kind of the weight of how important that event is and so, the user can now interactively. Say okay how, about dollar, yen different currency pair and the Japanese, currency against the dollar and so, you'll notice that they didn't have to say chart or, show, me something it's just like you know very conversational. It's carried forward the context, of the fact that the users interested, in charting and it's obviously understood, that it's extracting, a different entity in this case dollar, yen so the user can now interact with the application they, can fiddle around they can explore, and do some analysis. So. It's, getting slightly more complex now we. Can do other things so we can have other types of intense so the user is now saying okay please scatterplot, Euro, against, Yan and because, the application, again is context, where it knows that the user is intending to plot euro dollar against dollar yen so, here's an interactive, scatter plot the x-axis, being euro, dollar and the y axis being dollar yen and some.
Of This existing functionality that we have within the application, the user can play around and instead of plotting just the raw values, they, can plot things like percentage rank and they can change the look-back windows over which that's calculated and, so on so, we've, got simple charting intents we've got slightly more plotting. Intents embedded. Into this application but, we can go further we can do more so. The next one I want to show you is this concept of a back testing intent so, the. Users now going to say, back. Test what. Do they do euro dollar, after. Dollar, yen, is greater than 110 so, currency, rates obviously have values, and the users interested in knowing what happens to one currency when a different currency goes above a certain value and so in order for this to happen we didn't just have to get the and, the, intent, and the the, entities we also had to have this condition and so, it's brought up this study of what happens and you'll see at the top we have this sort of time series, and. The blue dots represent every, time that event occurred. And. In the middle chart we have the average price path up, to 24, hours later showing. You and what's happened to the euro dollar exchange rate after dollar yen has gone greater than 1/10 as you click in you get the distribution of returns below. It's kind of like a nice study for. The user to be able to interact with so. Shifting, gears slightly moving, away from just controlling the user interface what happens if we actually wanted to engage with our customers so that they could buy something work from us and, we have a lot of custom lingo and Finance which you'll see some of it now so the, users asking to chart euro you're, against the dollar that's, the standard when they say that and then they're going to ask for a price so the way in which a customer. Or salesperson may choose to do that is quite specific they. Say how wide in nine million so they're interested in a price that we'd be willing as a bank to buy to them but, so he'd sell to them and buy back from them nine million euro dollar and we only quote the last two digits of the exchange rate we have these so, we're showing 49, on one side and 51 on the other and the. Users saying mine, that's lingo for bi they want to buy so they're interested in picking one, of those prices they didn't tell us in advance which one they want to buyer and so they said mine and we've returned with transaction. Complete and here's an example transaction, I'd be explaining. To them what they've done. So. Following. On with this kind of idea of having, to train up on different lingo we, have this concept of cables so if you're not from finance cable is the great British pound against the dollar it's a specific terminology, just for that pair and so they've. Asked to plot that and then they're gonna instead, of by this, time users gonna say okay make me three. Million different, way of asking for the price instead. Of how wide they said make me three million so we've given back them a quote specifically, for 3 million cable, and it's. The same price because we haven't integrated this with our pricing. API but that would be the next step and then, they said done left hand side and that means that they're interested in selling and they've done, so so the chin transaction, is completed so. Just, to kind of summarize what's taking place here we've, controlled the UI to enable the user to do some exploratory, data analysis, look. Around at charts look at the relevant economic events, do some back testing and then having done that analysis, they've decided that they wanted to move into a transactional, business. Engagement. With us and we've, trained dialogue flow on very customized, specific. Terminology and we'll, talk a little bit about how you do that in a minute over to res for another demo thank you can you switch, the laptop please thank you so, obviously I don't have access to their internal, system so when I'm playing and trying to explore stuff for, James we have we built a sort. Of parallel agent. And. I have no talent in UI so one of my colleagues in Singapore, built this for us in principle, it's the same kind of thing, it has a graph of information.
We, Also have some news information underneath, that we populate, from Thomson Reuters again. The, data that we've provided for is this, is 2015. That we have access to from Thomson Reuters so, what. I'm going to do is just show that this is fairly similar to the, application. We had before, here. We have sterling, being shown so I'm going to do show. Me, a. Graph. Of. Euro. Now, I go back that. Should have changed and let's. See how dry my mouth is so. You find out. Show. Me a graph of pound, sterling. There. You go and. Show. Me news, on pound. Sterling. Okay. So my mouth isn't that dry so that actually changed underneath as well sorry I didn't have that highlighted, that. Was just dialogue flow the. Same intense, were being used we're now using the speech-to-text to. Actually, convert what I was saying into text that was being, represented, on the, graph, so, in, terms of dialogue flow itself just to go through a few pieces before we do another. Small. Demo around some new features that have been added to dialogue flow. So. This. Is the dialogue flow user interface, I'm, currently in my FX tester, environment, which is just a play environment to work, around with intents, for. Those you're not familiar with, dialogue. Flow we have the idea of intents to actually convert text, into actions, and entities, what I usually start with is entities, in. This case the. Entities, that we have available are, things. That are put in including, currency, now. The. Currency entity. Dialog. Flow does have a system, built entity, called currency, that, would normally suffice, for any currency activity, however as James. Demonstrated there's, a lot of domain-specific, language. That, we use here, and therefore, I built my own currency, entity so I can overload it with the, domain-specific language, that we need for. Example if. We go to. Australian. Dollar so, by doing that I brought up the entity Australian dollar this is the symbol and these are the synonyms, that are associated with it at the end we have Aussie, as. A you, know one of the domain-specific languages. GPP, so, I'm going to add one to this so, this. Is great. British pounds and where, I grew up in north oh we don't actually refer to pounds, as pounds down where I work it squids mate like. To to power to quid for a pint showing, my age it's been a long time since our two quid for a pint so for that I'm going to put quid. And. Essentially that's all I have to do to actually teach it that quid, equates, to pound. Sterling, now, when I go to do an intent, so we have various intents I'm just going to show one of the intents which is the graph intent which you've seen quite a few times in the demo now this. Graph intake takes a series, of phrases, I'm. Gonna put. One in now show, me a graph, of. Quid. Versus. Dollar. So. I just, added quid as a synonym, for that entity what. The dialogue, flow does very nicely for me is when I type that sentence in it's already figured, out that some of the values that are put in our currencies, it's, already highlighted those, for me and this is step one of getting the, extracting, the parameters, that we need from, this intent, so if I scroll down, we. Can actually see the actions, and parameters here, I've defined the input, graph action. This is called my code at the back end to actually. Invoke. That handler. We, also have some parameters so for example the currency parameter, if, you notice on this side we've ticked this box under required, so, if a user says show me a graph and they don't provide a currency, to me I don't, know what to draw so, we're, gonna prompt them to give us the currency and that's all inbuilt, into dialogue flow if, I click on the prompts so, these are a couple of the prompts that will be provided, to the user as we get them to give us the currencies that we're interested in if.
You Notice we've added more than one prompt this is like good UI design, for conversations, to. Give make the user feel a little bit more comfortable using the chat bot if you give different, sentences, for for the same questions. Outside. Of this piece. I'm going to just show a new. Piece of functionality that has been added to dialogue, flow so I'm gonna go to one. Of my user, environment. So effects, this is where we've actually got things wired, up and. One. Of the things I find particularly useful when working and developing on here is the, test, UI that you can use on the right-hand side so here, if I do. Meaning, a graph of y, n versus, dollar. So. What this is doing is we can see there a response, that was given but much more interestingly it. Gives us information about, what dialogues, are extracted, from that sentence, so I can, see the action is correct input graph, I can see that the parameter, is currency and it's picked out yen US dollar, we. Have, some. Extra information which is not part of the demo today also. Particularly useful is diagnostic info, so here. We. Actually see all the raw non responses, that it is sending, to the back-end services, and this, is super useful when you're doing debug and testing just take the JSON from there put, it into a message and then call your back-end web service so the thing I'm going to show you now is a little bit of a different flow from the one that we saw, with James so I've got a currency, now I'm gonna say quote, me. Two million. My, one is not as integrated. As James, is I just say you know here's a quote for 2 million xx, YY. If, you notice however there, is a sentiment, score attached to it this has been built into dialogue flow he was just recently announced, and it's this. Provides a negative. Or positive sentiment, to the statement, given now, that, statement that I asked give me a quote didn't have any sentiment, let's, say I was not happy with that price and. I'm, British so I'm not gonna swear at the system so we're gonna say that's, a, terrible. She. I had that there terrible. Awful price which, is as strong as I like to get in English. Here. We. Can actually see that the response was oh please, don't get mad now. Obviously city would put a far more. Professional. Response to that response to that thing but, well the reason it said that is if I scroll down we, have a sentiment, score of 0.9. Negative. Now, if. I go to some of my code, to show how we implemented.
This. What. We do is that intent the handler for that intent, looks. At the sentiment, score for that sentence, if, it goes over a threshold and you'll, notice that the thresholds quite low I've deliberately, said that, it's. Going to fire a different, response, to the user instead of sorry we couldn't help you oh please don't get mad, now the. Other thing that it does is it. Logs. To pub/sub which is our global publish, and subscribe API, the, message, with, the negative sentiment, and what. This allows us to do is to have systems, that can react to, if a customer, for example is getting angry at us now the sentiment, score I showed you was for that sentence however, you can have an overall sentiment. Score for the whole conversation and this. Starts. Getting very interesting. In kind of things that we can now do with. The chat war the, other thing of integration, that we built was. All of the intents, I logged to bigquery. Now. The. The results, I'm just dumping as raw, JSON I use. Bigquery can, extract. JSON, information. So this, every, intent everything that's being done is being logged if I run the query it. Actually will show just what's on at the bottom anyway, but. Here, we have the different intents, a, confidence. Score in what you know what we how confident we were that was the intent that user had and the, sentiment, score that. Was associated with it now, the, reason this is interesting is, if we have a mechanism for pub/sub. To notify us something's, going wrong and we, have the history of the conversation, we can write yes I mean you can imagine if you're building this for your own teams as. I was thinking about it it's City you. Know if you're automatically, serving, requests from from customers, in, our case it's both internal, users. An external, especially, in the case of external customers. You want to be thinking about okay well automatically, serving them but what happens if something goes wrong and. It, might not even have gone wrong perhaps just the customers. Dissatisfied. With something when, they came to you even before the conversation and so, you may choose to actually. Have something published, on to pub/sub obviously, that's real-time and you can surface an alert to a user and also a city. Employee in my case you can actually Benfield that request manually and slot in back into the chat and take control and by publishing all of the previous. Requests. Through the dialogue flow interface whether that's your own or something else into, bigquery you obviously have the history so although we didn't build it for the demo, you can imagine that you've been alerted you have all the the history of what's come before the user can read up understand. The chat and start getting, involved in the conversation, very quickly so that's obviously quite, an important, I would say prerequisite. As you start getting into beyond, traditional, just fa Q's and you start doing something a little more involved this would be something that's quite nice to happen and now possible, thank, you can we go back to the slides please. Fantastic. I mean. Here. We go so, in terms of the architecture patterns. There. Are two types there's standard deployment so there's a lot of are things. That you can do just without other box integrations, with, dialog, flow that's. Both in the front and and the backend on the we using firebase functions we. In. Our example, we use this very decoupled, deployment. Because of the complex environment it that, we are in and, some hybrid requirements, so, just to cover the standard, architecture the. Out-of-the-box integrations. Couple of clicks and you get things integrated, it takes care of all the things love like when a user makes a call to the car to the front end the queries get sent on to dialogue flow for you you. Have things like you can write, simple cloud functions, to deal with the fulfillment side of things in. Terms of our architecture we start if, we dive into what that actually, means that flow let's, start picking this a little bit apart so first, of all the fulfillment side of the house this is a web book call so, now. If you want to decouple that you can just stand up a restful. Web service and absorb. These calls and give the response wherever. You choose to, stand, up your web service on the, other side of the house although, we didn't show it in the in, the demo we were using standard client it.
Will Be important, for the client, to be, outside. Of the standard integrations, because, for example we, would want the. Authentication, authorization for. The client to be done within, city using. Cities mechanisms, and, this. Means we end up with a topology. Where we are going to have the the client be essentially, something that the user builds and proxies, to dialogue. Flow so. This is going back to our complex. Environment document so here, we. Have that flow right so we have a user that is. An. Application that can use authentication. And authorization from. Within the. Environment. That exists within city it, can then communicate through, the hub service through, to, all of these other systems. This. Is the, flowchart, of how this would work if you imagine the, client, is now a piece of code that's sitting in a. Container. At City. The. User logs in they're going to do authentication, authorization with. The hub and the hub the code in the hub we put into a container which meant it, was easily portable between our. Environments. The, authorization, once done the hub. The user can then start putting, in requests, and. That request is sent to the hub the, hub then, takes that requests piggybacks, it onto another HTTP. Call over, to dialogue flow we, get the response from dialogue flow the, hub can respond, with we're, going to be creating the graph for you if you imagine a trader is typing on one screen they've got and I've been into those rooms they got like tons of screens it's really cool. Then, in the, meantime the hub will also then do things like make calls to bigquery to extract. The TIC data that, was, available from Thomson Reuters, it, will, then. Once, it receives that data we, send that information on to firebase, to, allow. The client, to be updated, so that's when I was calling into dialogue flow and you saw the graph being updated that's, what we were using to achieve that. So, lessons, learned yeah, so I think, when. You're thinking about using something like dialog flow yourself I mean the architecture that we've come up with was something that I. Asked. Fez to help us with because we, have quite. A lot. Down Network we, have a lot of existing, constraints. And rules that we have to follow and so the, hub architecture, really enabled, us to work, within those those. Requirements where, as out-of-the-box. Dialogue flow gives you a lot of this functionality, if you're going to be cloud native already. Or it's a completely greenfield, project. There's a lot you don't have to build so the hub gives us that flexibility, to take something that we already have and move, towards something that's going to look like production, whereas, you. Know if you were just starting this from scratch you would be able to do this more simply so you kind of have the best of both worlds. When. We started this in my mind you. Know I kind, of had the idea of what conversations, I wanted to you encode. Into the dialogue flow framework. I had, the lingo I knew. The way this was gonna work so, we kind of jumped in and I'm, you know there's me talking, with rares and what we're going through this and we're kind of getting excited with how quickly, and easily were able to build some of this stuff but of course I'd recommend, like.
With Any like architectural. Design. Process, or even like your actual software. Implementation, process you know you actually do sit down kind of with a pen and paper and work out what you want to do first rather than just jump in because you, know you need to make sure that you, make some. Assumptions, that are the correct ones at the beginning because otherwise you're gonna end up I'm picking some other stuff you've done later, so, are your conversations, correct for dialogue flow in the first place okay let's say they are you want to make sure you know who the target audience is for your chat book, like. Any product and then, you want to work out for that target audience what are the number one things that we're spending time doing now that we don't want to do or, that we can do a better job of doing by automating, so what are your key use cases and then, you need a persona right so in finance, you, know generally, speaking people are doing a lot of work, every, day it's very transactional it's quick it's fast there's, not much small talk really and. So you want this persona to be quite you know efficient. So, that you can move on then do the next thing that you need to do so. Then you want to like sample, or you want to design things out you want to test and that's the green the green section here so spending, more time doing that at the beginning is going to help you a lot later on down the line. And. Actually one thing from that previous slide that I found particularly, interesting is. It's. Very easy when you're building out the conversation, and the intents, to assume. That other people will ask it in the same kind of way, being. From London and working, with a lot of folks, in the US etc, I know that to be very, not true now and. It's, very useful to get, a get it get some intense up and running and then get colleagues and actually, say colleagues, from different countries if you can to actually try it out. It's amazing. The different kind of variations, that you get and one thing we didn't show you was how easy it is with the training with dialogue flow so, you get one of these sessions like you know it's almost like a jam session right you sit down and everyone tries to fool, the system you, find. Different ways of actually asking the questions and then, use that to train and when you're doing the training make. Sure that. Part. Of the training hasn't, exposed, a different, branch of the conversation, that you need to build in right, so it's very easy to start trying. To overload a single branch of the conversation, and rather, than saying okay actually the intent has now changed we should create a separate intent to cover that kind of flow. The, other one from that first one is the tail latency, so what. You'll find is that you'll very quickly get a the. Chap were up and running Lee in terms of the standard. Flow of the conversation spend. Just as much time on, the the. Bits of the conversation that might be broken right so if, the user goes down the wrong path the contexts, get incorrectly, set up spend, just as much time there because a couple of negative experiences. From a user and then, they're, gonna be reluctant to use the chat board again just on the training I think one thing I should add is when, I started, doing this at. First you like a cake-like chart me Eurodollar, show me this plot that and you kind of like showing a bunch of examples and you're thinking I could just write this myself with regular expressions like what's the big deal and that. That, was how I came to this and I was like hmm but, what you quickly realize is through the work flow that dialogue flow gives you you, can quickly tag like okay this is the intent this is the the entities but you've given it like maybe half a dozen or whatever three, four five and, someone. Then uses the tool in a way which you hadn't trained it on and that's, when you start like ah that was my aha moment when, I was like oh actually it's gonna understand, this thing that I didn't train, it on and subtly.
And I think quite importantly, if, you're into language like, understanding that, you know something. Plot me whatever that might be and then the the entities you know you can also in a different way right you could ask that like you're a daughter show, me a chart of that and dialogue. Flows gets robust to these types of things as long as you've given it you, know a sensible amount of seeded information, and that that was one of the key things that I really liked about the tool that at first I didn't quite get and. Thanks. Jason and the other piece, when you're drawing out these conversations, you know draw, out some flowcharts using the tools of dialogue flow has in, terms of like slot, filling which was where we asked. For currencies when one wasn't given now, context, which we didn't show you allows you to store the context of a conversation as you branch, I think, it was very very useful Jessie just pencil these things out and. Go on build the intents well you'll probably find it as you've been setting these things out as well you'll find some branches that don't make sense and you'll. Fix them up so, with. That some final thoughts James yes, so hopefully you guys have got a good feel about how you can use dialog flow within existing. Applications, also within, existing, infrastructure, and depending, on the company that you're coming from and I think. I'll loop this back to the beginning which is what seeded this initial idea which we actually haven't. Fully fleshed, out yet but for, us because, we have so much data and we are a large bank and we service a lot of different clients we. Think that. One of our unique, benefits. Is that we have we can bring, the right type of data to the client when they need it and, so for, me I was thinking about ok what happens if both, an internal user comes. To a trader who's, very busy in that they're managing that their portfolio, and that user wants to say to the trade you know why did the exchange, rate of this asset just spike you, know over the last 15 minutes now what caused that now. We have all of the trade information, that happened up to that point we have all the economic, event data up to that point we have all the market data up to that point it's, not beyond. The realms possibility, and we've been building this so, that you can programmatically, produce, a, natural, language response, to that query that draws on all of this data and frames it in such a way that is human readable and it's nice to read right so we, can say you know we're seeing a lot of, leveraged.
Investors, Buying because, you know the, economic. Release that just happened was weaker than we thought and. You know we're now at the 94th, percentile, rank for the month for this exchange rate and so we're seeing a lot of you, know momentum, in that direction you could produce that report programmatically. Through the data that we have and you could imagine a user asking for that just quickly on a chat being like why did this happen you could respond in chat in kind and so, for me where I want to take this project is more down these lines, I think, dialogue flow is great for FAQ it's great for just the simple things and for sure we'll be looking at that as well but, I think there's a lot more interesting, subtle things that you can do which. Create the sort of overall ecosystem. Of of text, and of course there are some client you just want everything over API and that's fine too but for the people that are in between this, is a great way to service many many more clients with high quality content, so we've. Covered a lot of ground hopefully. You've got a good feel for it, that's it from me and. Just. To say there are some more sessions that. Are available running. Tomorrow. And again. Thank you very much for spending some time with us this morning.
2018-08-11 17:34
