Let's Get Chatty with Conversational Interface in JavaScript - Tomomi Imura | Render Conf 2018

Let's Get Chatty with Conversational Interface in JavaScript - Tomomi Imura | Render Conf 2018

Show Video

First. Of all I got a big mistake I grabbed. This sparkling. Water instead, of still water so, I might barb sorry. I had, to let you know beforehand. Seriously. Plus. I'm still quite jet-lagged, and am i bringing it's not really clear so. I might screw up today. All. Right so. My name is - mommy, -. Mommy Maura and today. I'm going to talk about a conversational. Interface, with JavaScript, JavaScript, yeah. I'm already screwing up. That's. Me again my, name is Tommy Mura and on. The social media I'm known as a girly Mac. Girly, underscore Mac sometimes no underscore, that, was kind of dumb name but I've been using this handle for like two decades already. So. As you can tell from this picture actually, work at slack but. Today's talk is not really, entirely talk about slack I do mention some little, bit but. I'm going to talk about like conversational, interface in general, and. The first half of my talk it's more like. High-level. Talk about what, what. Is you know what, conversational, interface, is and what you can do with it and the. Second part it's more like Chantel I'm with. My demo and Accord. BOTS. So. This dude. Some. Guy from Microsoft know I should I should give him some respect, it's, a dealer from, Microsoft, same both, the like a new application, that you can converse, with so. He's talking about some conversation. With nonhumans, and. Another. Dude well. Sundar pichai from Google, he, was saying I will evolve. In. Computing, from mobile first turn on the I first world so. He mentioned that at the Google, i/o last the I believe and I was actually there too and I was quite surprised, here because, Google. Used to be saying all the mobile first but, now I guess mobile have get like you know its own maturity, so now this, ship in to, the mall to an AI now. So. This is some, Google. Home voice assisted, device and, we got so many other things like Amazon, echo and Apple. Has a new stuff I forgot a name home pod or something like that, geez, I already, forgot about it but, there's, so many voice. Assistive devices out there. Traditionally. Traditionally. Speaking. When we you know you design. Like web and app, with, the youth user interactions, what, are you using like form input, right so, user can just type in something and submit. Oh maybe, we can let user to choose from menu, or, maybe from, keyboard, shortcut, and such and I, say it's a traditional, but of, course it's still valid we're moving still using it all the time. But. Later on we, get some more like social, web, so. Like modern web and app with, social interaction, get popular, maybe. I run like several. Years ago so so. We got some, like. Button sort we some. You know like intense, like you can tweet from somewhere, else rather. Than twitter.com and, such and that's all had like a real, time interaction as well. Then. On a train to go to a conversation or user interface and. So, you can interact with user and a hardware device it's with let's, say it's voice in this, case, example. With Alexa, in theory, say. Hey, Siri you. Know we're to Pancho Villa to Korea that's. My favorite to here in San Francisco and, I'm. Glad I didn't, activate, any of you know, I devices, they happen sometimes the, hey Siri, sorry, I won't do this again. And. This is another one it's actual conversational, user interaction, for kids in different, form factor, in this, case it looks like some toy. It's, a cognitive dyno. That. Actually. Have a Watson, IBM, Watson, brain so. This little. Toy, can talk you. Know like teach kids math maybe. Telling some jokes. Play games you can do so many things. And. I can't come as a different, form factor to could be robot. So. This is actual picture of my niece my, name is live in Japan and she interacted. With this little robot called pepper all the time so she knows how to deal with already and when. I was visited her in Japan last year I had no idea what to do by. The way this picture is from a sushi joint so we went to eat sushi and this, robot like you. Know sis something's like, you. Know like hello welcome such, and I have no idea what the hell's going on what I'm gonna do and she knew you know she. Told me how to use it like, digital. Natives, that's crazy I have no idea. And. Of course a conversational, interface can be in a voice and text, so. This is just a screenshot actually. Umm, animation. Of Google, assistant, so. You can talk or, you can type as well on, your mobile phone. Then. It could be just text, maybe text with some sort of graphical, user interface, on a mobile phone or web so, this is an, example. Slack, what, kind of doesn't. Have to be slack but so. It can give you some sort of notifications. If. You ever watch this movie the office space it's kind of old movie but, maybe. Now every, day like a 4:45, p.m. maybe. Can just notify I say hey don't forget your TPS, report. I. Guess, no one have watch this anyway.

Super. Freaky is going to face the conversational, interface this. Is another example. Somebody's. Ordering pizza so that could be done by either text, or voice. Deliver. Me some large margherita pizza and bot, might give you some sort of options, well, maybe this robot, I mean in this spot might, already know the preference her, preference, from the previous order instant and maybe, that can be you, know connected. With some. Sort of payment. API so, you can make a payment all together, that's, a one of the example. So. The conversational, interface achieves a natural. User interaction, with minimal, visual interface. The great thing is about great, thing about it is there's no UI clutter, so. Ideally, each, user spend less time. Here's. Another example, so. Two, maybe. Two girls like conversing, and it, may be one stay hey and then come over to my place and, another girl's like also wait what it's you know where you live what's your address and, as. Soon as another lady and say. Her, home, address but, reacted. Because you mentioned home at night straight address so, but like hey request the right I know. It sounds kind of creepy but, you. Know of course you know in that case you they have to know and she already have a configure, that way so you. Can't just you know randomly. Activate. Your bar we saw in, a notice in your users, first obviously. And. Of course in this case yes get me all right so give another, person give, you the permission then, the but can you, know get you some. Right, share of taxicab, whatever. And. I, say there's, no UI clutter, but that's a meaning there's no like zero UI with. A voice assistant so. This is examples. Actually, from. Amazon, developer, at amazon.com, so. It's actual, Alexa UX and. It's it's something on Amazon call a voice from example. And voice from, so. It has some sort of indication, on, the hardware device it stops so, you don't know what's going on, if. They're listening it, is speaking you. Know it's funny though because. Amazon. That's freaking upgrade, its that Amazon echo so, one day I came back home and I'm showing yellow, light. And now that's kind of freaky not because I didn't know what it means but. That. Was like a automatically, updated, and it simply, meant you know I had a some missing message or something but, he initially I didn't know so I thought was broken something like a yellow, yeah. Yeah. Other than that once you know what it is you know it's always nice to have some visual indications.

And The hardware. So. Conversational, interfaces in. Twitter and, accessible. And productive. There. Are some mesogens platform, out there so. Why. Work for slacks I put it's like an adult but slack. Facebook. Messengers, and there's so many other platforms, out there and. Lots, white but it's line that something, incredibly. Popular in Japan so whenever I'm chatting with my friends, in Japan or family I always use line. So. It's just example. So. In the case of slack we use BOTS like a lot of third-party bots out there by. The way I work at the slacks platform, team so I'm actually advocating. Like API is usually, so, you can create you can use like API to create some BOTS, in this case like it github Dropbox. Twitter oh by. The way it's, yeah. I'm, kind, of unrelated but if, you being using github app with slack and you, haven't updated it for a while you, should reinstall. It now because we get a new version came out and that's far better. And. Well, this is not yeah, it's just that stuff, that I use put, some often times you know people ask me Oh what kind of box you use that slack so I'm just example, there's, something like I personally, like so, you get like stage and deploy chicking so, when I emerge some code and somebody, else you. Know just push the code it - um, you. Know deploy the code and push the code push. The code to the server I always, get those stage, you know when it's pushed to stage when deployed. It all let's get notifications. Oh sometimes. When somebody, up updated. JIRA tickets I get notifications as, well and. Something more social, it's called lunch string this was created. By my colleague, so, when you wanna go to lunch or, coffee with, your colleague you can just always post it and sometimes, you make new friends you know in office, we, can almost respond, ok let's go to lunch. Another. One says silly one the flip flipping, table bought so every time I got somebody pissed me off and I flipped a table the, bot fixed it for me. Not. Just fixed in serve me to respect, tables, all, right I shouldn't, flip in table and. Every day in a three o'clock flight. But always not find me to get a coffee. That's. Cute little things and this, is not done by us this is by Taco Bell, I guess. Taco Bell is not really huge in here and they okay I don't know because I haven't seen any in like. Woundn't area but, anyway this is the fast-food, yeah. And they have created, this lack what it's called taco bot so, you can order, tacos, from. Your slack it's. Kind of funny, I kind of want to try but I'm a vegetarian so, I don't have. Any were trying. But. I thought was interesting idea, yeah. So. The. Top available maybe I'll show it again so you can actually converse, you can you know a chat, like as if you're talking to human, and about, reacted, and that takes order and, dudes, example. So. They are using one, of those natural conversation. API is out there so. There are plenty of ApS you can use and there. Are also other parties from different companies and our first one is dial the flow I still. Refer this as API de AI because, I've been using API, for, a while before, they got acquired by Google, so. They renamed, it to the dial flow, now. We are is another one similar, API. Is similar, servers, this. Is Facebook the same way Facebook, acquired a company it's.

Not Like they make it and. Microsoft. Has own framework, and there's so many other, you. Know APs and services, out there and they're, really like a high level api's, and, relatively. Easy to use it's, lightweight. Easy, to use and, it comes with you. Know nice graphical, user interface, - so. In this case, either way I I took this diagram. From their website, so, basically it, understands. Natural conversation. By trying to just block like you, know grabs and. What users say and try to understand, what comes what the user tried to do so. In this case it, reacted, the world like, grouping. The words they feel the baby. As a test and tomorrow, at 7:00 a.m. means that's a date so, they know where the user tried to have some tasks. You know so that intent here is a Jade and create event so that's how they. Be I mean the. You. Know natural, language, processing understands. What you said seen while you try to do. And. It's something I call it well it's not real word i I just made it up but AI. At the service and those, example, like again they, have a really nice user, interface, that. You can create, a cart, official conversations. And, you. Don't what you can call it could have API snow, SDKs, too so, you can cook but, you don't have to you, can create, you. Know all, the interaction. Artificial. Conversation, by just using the UI and, the. Miniature. Has something like you, know it's really, nice noodling. Stuff you can just connect dots and create a conversation and, they, are really, intended, it for like. Customer, service rep you know kind of other people so you don't have to be any engineers to use those services but, it's more powerful if you know how to code and. That's. Not another example, so. Somebody try to book a flight and. This. Rep say hi Lena I fly from San Francisco as usual well, in that case you, know what she, a customer, doesn't have to know if this is real person, or BA if it's a text you can you, know hear the voice so, that. Can be a but really, and if he's like yes from San Francisco, and we're flying to London. So. This is actual example, done by KLM, and. Well. I haven't find it I'm certainly yes I haven't tried this happen for myself but. Apparently they use a diagonal, flow to create this conversation. So. That's really interesting, use, case.

And. Of. Course you, engineer, so you can, use more powerful NLP, you know natural language processing platforms. And api's out there. So. There are I be M Watson, net section my favorite and, Google. Has their own natural language API as well in a Microsoft as Louis Emma's, and, why do have something too so all the big Flay is actually going into this field right now. So. Then how about a conversational, interface with JavaScript. So. Of course you know most of my API so you. Know just, totally. Accessible with, JavaScript, then, my habits DK they, might just use the recipes, you can just use it with JavaScript no, problem. So. This is an actual example that, I use is Facebook, messengers, with a pietà ai that. I didn't use a way I, because. That was before I created, this before, they were yeah it's wired by Facebook, maybe. It's easier to use both together now, I don't know so, this is something I actually created, for, fun so. It's really nothing, it's, just the artificial, conversation, people can chat, with my bot, and. In, that case I'm using it's, a note Jas, so, API de AI that's, dialogue before but it let's before rename has. A no Jes SDK, here so I'm just including. And, I'm not listing. Whole, codes. And code and code snippets here but. Basically what it's doing is Facebook. Has some event api's. So, when some, user type, in post message, I get, from sovereign. Gets event so I know what's you, know somebody's, type and send message, and I, can just grab the message and pass it to API, that AI which. Returns. A reply, so. When I say hi and API. De AI might return me hello and I, just post, in to, the Facebook so. I just, leave it on wild. I haven't even take a look at but, every time I just try to pick out what's going on I'm, surprised, so many people, actually have tried to converse with my bot but. Sad. Sad, human they're, always talking, something, nasty. Poor, but not always, but I occasionally, see, it is I'm like oh. Yeah. I can't really talk about what, it is it's just nasty. So. Again I use a dial to flow so. I well I don't know if you can access, from, here. But that's like I don't know what it's called like something, like a QR code apply the Facebook's use so you can scan it to actually use this. Any. Yes yeah I use APA is, something. Called small talk that, comes free, all that service it's like, booking. Whether or respond blah blah blah they all paid like a fee-based services. But small talk comes to its free with API so I can just, use it without paying any and it. Really you don't have to well you can you can configure, the conversation, as you, want but, well, the small talk already, comes with you. Know simple, chat you don't really have to add more conversation, by yourself, so basically just somebody say how am i doing and the. API returns I'm great, you know things like that. And. I. Made a move things with slack as well of course and, this. Example is a I'm. Using slack API with IBM Watson, so basically doing, a sentiment analysis, so, letting Watson. To analyze, how. User field, so, analyzing each message. So. Basically that's the workflow. I'm. Kind of running out of time some not explained too much in, detail but, basically read a message in a Grantham throw, the message to Anna Watson and Watson analyze, the motion from each message and. Again, unlock, uses iPad based API as well so, once events back that, means some user to send a message read, a message and throw that tuna Watson and Watson can't, figure it out if the message feeling sad or happy, angry. And. Like. Yeah everybody's, still give our hardware cool stuff so of course not I'm saying I'm the same way I love to hack Hardware as well so, I actually. Install this application I wrote in Raspberry. Pi with. A led. Hooked up and has. Some cute little robot, you, know shell that IBM came to me. So. This is actual, Raspberry. Pi robot, so, I'm, using this led at the indication, of the emotion so. Based. On I mean depending on what emotion, the user feel, LED.

Lights Up in different colors I forgot, about dream means, maybe just calm something, I, didn't. Use a really good example in this video I should've used some angry examples, red LED, but, I didn't. So. Yeah it's great because when you, know we get an angry customer and let's. Say the, customer service, it's being used in a bar but maybe they can switch to human hair could someone is angry that. Can be actually useful maybe. But. I did the full funder. So. I have, another example so this is kind of different because. I used to be working with w3c and I still care about a web standard so much and, I wanted to do something with browser. So. This is my, project. Or different voice chat here so. Basically what happen is when user talks to the browser a brother. Convert. From, voice to text then. And secondly, I'm using actual, third party in LA in, a NLP. Api's, to. Create some artificial conversation, like a reply from the browser then, browser, speak back to the user so this, one is a text to invoice, conversion. And. Those. Are API is actually well. Kind, of sort of supported, by, major. Browsers, not, really. Though so. For the speech scenes at this I put, three because that's number three in the process here but speech. Synthesis, has, been actually well adapted, by all the major browsers already, so we can use that now but. For speech recognition maybe. Is being only supported, by chromium's, right now. So. When we are initiating, action when you know we using this feature. Detection here as a window object you, have to use a WebKit speech recognition, with, you, know prefix. And. That's. How you can, create an instance of a speech recognition and. You, can, have some properties, with this you, can sit language, you, can set entering result, I put false but, what it does is it, converts you. Know your voice to text as you speak but in this example I wanted to wait, until the user polled, so I put false, and. You can have some method like start/stop. Abort, in, once it's successfully, done. You get the result. And. This. API doesn't, just steal your voice doesn't, do until, the use each user you know give the permission so, always have this done in, a permission UI. Okay. So I. Have. So. This is how you work. So. I have, this example called pan so, the first example is when you see quick hello. Let. Me see it I don't think it's quite working. Hey. Hey, hey, alright working oh, it's. Working it's, working it's working yeah. Okay. It's. Kind of silly but. So. That's what it is and I have another example so I'm gonna just use. Okay. So another one it's a speech synthesis, so, this is in widely you, know supported. By browser so you don't need a vendor prefix anymore, and, therefore, you, can just get the property of speech synthesis, utterance, interface here then. You can configure which you, know what text, you want to convert it into voice you, can set a pitch you can say language, and you can have it to speak so that's another example I have, only five more minutes. Okay. So I can let's. Just use the default. So. This is really I'm just giving a string, and let the browser to speak I can say like you know what. And. Of course I can set some other languages, like Japanese so, let's say this one. Alright. And yeah. I'm a native Japanese speaker so, I can say this is pretty good, actually, surprisingly. Good. And. So. This is what it is it's basically text, to voice so, I combined, to save everything, using.

An Agile server and I'm using a socket IO here because, I wanted to have some sort of a real-time connection in between my, server and Brazel you know in a client-side and for, the third party API is here, I'm using a Pai again. Again. So. This is another demo, shouldn't. Quit. Now. All the AI stuff, and this, Web API is example I just show you all combined, here. So. I'm going to have, a conversation with, my browser. Hello. How. Are you doing. Hmm. You. Know what I'm wearing cool socks today I. Don't. Think you understood maybe did him, hey. I'm wearing cool socks I. Say. I am wearing cool, socks. Yes. My body's not so smart, because. I didn't really configure, right but, this is fun right so. That's what he can do with the web speech API eyes and the third party, in a natural processing API is here. And. I wrote about how, it works and how you can create a tooth on actually. I'm smashing magazines, so if you're interested and I, have a source, code and github as well. So. The. Conversational, interface is for human, and a. BOTS interface, really is a human interface so, even when you're dealing with bots, and they I still. You use as a human, so you have to make it easy for human, so. Remember me we, used to Clippy, he was a little annoying assistant, wasn't quite helpful because, it wasn't really created for us but. If Microsoft tries, now I think that can be better, all. Right I, was, gonna say thank you but I have one more slide. So. I didn't really introduce myself, and initially, but I always do. Some funny, cat, hack, I always, combine, the cats and technologies so most of my talks, involve so many cat pictures, but. This dog didn't have any cats, in it and. The. Last time when I gave it a similar token in the San Francisco, people, complained it to me that I didn't, use any cat photo so, i had a cat photo and at the last light today. Okay. So take over this first it's my cat I. Know. It's really all. Right so that's thank, you so much.

2018-04-13 14:19

Show Video

Other news