I hope that the coffee that you have is going to wake you up in a couple of minutes so that you can attend, to all our amazing speakers, my name is Shawna, and tell I work in Microsoft Research in Cambridge in the UK and, Abbey's hxd. Group and we're here to talk about the future of communication. A critical. Part of course of the future of work. Of, course in some ways the futures always been communication. Right everybody always asks, well what does the future hold for communication. At work and in, some ways you know the questions are sort, of the same like is it about just old, ways of doing or new ways of doing old things or, are there in fact new things that are possible, being made by new technologies, but, how new is anything really, and I think what's great about this particular panel, is it will have speakers, who'll, be able to speak to sort of old, sociological. And HCI, concepts. But, also new ways of thinking about those things and new ways of leveraging those, things for, the future of work. And. That's because we talk the, world of work, into being and so. I ask the speakers to consider these four questions. I ask them to ask how. Should we cope with the increasing communication, technologies. Right all the different ways and all the different parts of our ecosystems. That we'll need to sort of talk through talk, amongst, talk across I, asked. Them whether there was a communicative, backbone, to, the work environment, and how that related to communication, more, generally, I asked. Them whether communication. Patterns are changing and how they might be able to to map those and ask, them of course to talk about how communication relates. To productivity, broadly. Speaking. The. Four speakers that we've got are really, tremendous, and quite, varied, so, we have Elizabeth Stokoe, who's professor of social interaction at Loughborough University a, specialist, in conversation. Analysis, which is my sort of home field as it were and. She'll, be talking about agents. And agents and communication, how we can build better agents, and in particular the, scene but unnoticed methods. That people have for, making themselves understood to one another and how that might relate to building. Better agents, then. We have Konstantin Sanders, a philosopher, from the University of Hartford sure who, are we talking about how we are to deal with this sort of increase, in communication technologies.
And Talk about the concept of unhurried. Time, making space for unhurried, time, we've. Then got Yvonne, Rogers amazing, professor of interaction, design from UCL, who. Will also be talking about agents, but, in a different kind of way for than Liz Nevada. We're talking about looking, for, intellectually. Appropriate, ways context, appropriate methods context to appropriate models, of interaction. And then. Finally we'll have the inimitable, bill Boston talking. About ubiquity, versus you by to you or your body versus a big video and they were quite sure which way these things have to go but I'm sure he will set us all straight on this and, he may also offer a few comments on the other speakers. As well we're, going to run all the speakers together and then, we're going to have time for questions and discussion, at the, end so. Please write down your questions while you hear the presentations, and then quickly raise your hands as soon as as soon as everybody's finished, speaking and then we can have a really good sort, of interesting, discussion, so. I'm, not going to talk much more because I really want to make space for our amazing speakers so in fact I'm going to finish a little bit early on my section and bring out Liz to talk about her. Section. Fantastic. Hello. Oh. The. Future of communication, is bad. It's. Good job I'm here to tell you about conversation. Analysis this is what I do I'm a conversationalist. And that. Means that I study real. Talk talk in the wild not talk, produced in labs, not simulated, interaction, or role played interaction, and I, don't ask people to tell me later post, talk how. They talk I study real talk and I. Collect, recordings. Sometimes, single cases sometimes. Tens hundreds, or thousands of cases of people talking in real settings doctor-patient. Police and suspect people on first dates and then. I transcribed, them in a lot of fine grained forensic. Linguistic, detail to, try and then discern all of the actions, and activities that, comprise complete, encounters from the moment they start to. The moment that they end I. Also. Do a lot of public science, around, conversation. And communication and, in a way I have the opposite challenge, most scientists when it comes to explaining, what we do to. It let's say a lay audience because something like a black hole doesn't really exist for, humans to understand, it but talk and conversation. Is is only there for us or to understand each other and of course we all talk so we all think we know everything about talk already, so, how do this. Is this is one of the challenges that I have is, communicating.
About Conversation, analysis to two audiences who think they already know about communication. And people, ask me questions like this will, we still communicate in the future will. I take, over our conversations, is SMS, ruining, our ability, to communicate with each other and, generally. I answer, that question by starting, with, this. Which, is that we're always going to need to do things in interaction. At least for the career. That I have left hopefully, people will still be needing, to do stuff in, interaction. So, they're going to need to do things like this this is my dad talking to alexa. Okay. I'd like to order a pizza for delivery we're, going to come back to how one orders a pizza for delivery a little, later in my presentation but. I want us to start with the idea of doing things and in particular there's. Something like a request so, when i say that we do things in interaction, and we're going to need to probably do them for, the foreseeable future i mean things like greetings. Assessments. We're going to need to make offers, we, are going to want to flirt with each other all of those things are things that we do in interaction, whether it be written or spoken and the. One action, that i'm going to focus on for the rest of my talk is requests. Like i'd like to order a pizza for, delivery. My. First example shows you how, nevertheless. When, we think about a request as a broad action that, each each composite. Word as one, builds the action, is crucial, and my, example comes from a really dramatic setting. Which, is police. Crisis, negotiators. Talking. To suicidal, people in crisis and their. Request. Is for, the person that they're talking to to, talk to them so. We're zooming in on a very you, know a moment it could be four hours into the negotiation, it could be just, like 10 minutes into the negotiation, but, I'm going to show you one, way in which negotiators. Ask persons. In crisis, to talk to them hopefully my audio will work. So. Here comes the request it's going to come out in real time line by line so you experience, the interaction as it actually unfolds and you're, going to see dots and dashes and squiggles on the transcript, which is to do with how we deliver, talk but don't worry about that you should have the audio which. Has also been anonymized. Can. We talk about how you are what, happens next is a silence, it's only seven tenths of a seconds but it's quite a long silence in my world know just what I've highlighted it because I want you to notice that whatever. Happens next it's delayed. So, here comes the person in crisis responding, to the question. No. I don't want to talk and you, should be able to hear even through they're not the anonymous Asian they're sort of inverted commas around no, I don't want to talk I'm, objecting, to the very thing that you're requesting. That I do and then, what happens next isn't more silence. So. In this little naturally, occurring, experiment, of real encounters, we see that, requests, to talk lead, to disengagement, in this setting and of course in this setting, every, turn matters, because every time the, person in crisis says something it engages then they are you, know further towards staying, alive rather than jumping, off the roof however. There is another way of asking, persons. In crisis to talk and if. And, it's going to work and you're going to see it working really rapidly, so in the next clip you'll see how quickly the. Person. In crisis, responds, to this different way of requesting. Communication. And. If. It was obvious, that, what happens next is the thing that works then that is what people would do so. Here is a negotiator, different negotiation doing. Something a bit different. So. Here the negotiator, asked the person in crisis to speak to them and the. Overlapping, talk from, lines two to three though I've highlighted it you can see how rapidly the, person in crisis starts to talk now. This is just one quick example from our, from our large data set but what we found was that when, negotiators.
Request, To, talk people. In crisis resist talking but, when they asked them to speak they don't resist so, every, word matters and the thing is you, would never know this unless, you actually bothered to look at real talk because, we don't really think that people are pushed and pulled around by language in quite this way because, we don't really look at it and we don't really take it seriously enough. Okay. Here's my next example which is that in requests. Not, just the words that we use to put the request together matters but what we ask for matters, and where we go um also, matters, so. Here are two people calling. The vet and one of them is a real call in other words this person really does have an animal and the other one is a simulated, call it's not a chatbot it's, a mystery shopper whose job it is is to test out how, good the customer experiences, is when phoning the vet they, don't have a script they're just told to phone the vet and and report, back on how good your experience, was so. Here comes one of the calls. Okay. And here's, another one. Okay. So in case you are wondering the, first. One is the mystery shopper who doesn't really have a dog she, doesn't really have a puppy. At all in. The second one it's somebody, who really does have a cat and they are phoning to make an appointment and just very, basically what you can see here is that when somebody, is told imagine, you've got a pet and I want you to test out the service they, phone. To find out about the cost of the service they don't phone to do what real callers do they, don't give interactional. Tasks, to, the receptionist, that actually, real callers do so, we found that pretty systematically, across their data sets that basically mystery shoppers do not make, the same kinds, of requests, of the service. Provider and that means. That when they go back to their company and say you know this is our experience they're not really telling you anything useful I would argue but, there are even some tiny. Details which are you may have noticed as the transcript, came out which, is that when. People, really, have an animal they tend to use the present, the possessive pronoun my, to. Formulate the request and they. Don't hesitate. And producing, the category, of animal that they're phoning about their hesitations, their little home is somewhere else in the term whereas. When somebody, is a mystery shopper they're more likely to stumble and repair the production, of add puppy. And they're. More likely to use the indefinite article, a. To. Produce. That the animal as a category, round rather than the possessive pronoun my cat so. All of these things are you might think tiny details, but of course they matter when it comes to producing things. Like, duplex, so here is duplex. Phoning. Book and address, appointment, just a few lines from the opening calls that have been made available, online, by, Google. Okay. So you might think and we could all think, that actually duplex, is getting close to sounding. Authentic. But. As, soon as you have start to see some of the analysis that we produce as conversation, analysts which is on the precise, placement of, things, like stumbles, and. Exactly how it is that people build requests, then it allows us to start inspecting these kinds of data and sort, of raise the bar in terms, of authenticity. Okay. So we're going to come back to pizza now for my my final examples, in. Requests. I'm, gonna say turn-taking. Might, be more important, than abstract, notions of context, now. We all know that one of the challenges to producing, at least turn by turn by turn by turn interaction. With agents, and so on is. Action. Recognition. And. This is partly because we we, know that meaning matters and we know that context matters but we tend to prioritize. Abstract. Notions of context, over, practical, action and that, means that we often ignore the, detailed, and sequential structure of talk by. Emulating, scripted, or often stereotypical. Dialogue and it's. So nice I think to see something like the mystery shopper study where you've basically got a human being so, the simulating, being another human being and seeing, how do you produce being, another human being when you are one does.
What What does it what does it actually look like and what can we learn about how mystery shoppers produce, tasks. And compare, that to something, like a conversational. Agent also. Producing. A task so. Here is a request for pizza and here, hopefully, less likely is this is my dad talking. To alexa and the response. The. Top search results, for pizza is kitchen craft masterclass, nonstick Pisa Crispin train Brahman would, you like to buy it now. You, might think that well here he hasn't got the right skill he often asked in the right way for whatever it is that he might want when he says I'd like to order a pizza for delivery but, the thing is we can ask the pizza in lots of different ways and. Interrogative. Grammar produces, a request, can I have. Declarative. Grammar produces, a request Pizza, sounds good you. Could say that pizza looks nice and that could be treated as a, request so, all of these different formats for building a request, can, lead to challenges but. Here is the context, in, which we. Originally, see, why. I got my dad to say dad can you can you just record yourself asking Pizza in. This way because here is an original, setting in which that turn comes, out. Okay. So now. What we have is, a really interesting challenge because, if you've Google. You, know scam, calls fake calls nine nine nine time, wasters and all that kind of thing you'll find lots of calls where people are phoning and making calls that look a bit like this you, know what's. The date and my cats up a tree or whatever it might be but. What you're going to see and you might have guessed this already of course is that this, is not a request for pizza this is not a scam call this, is not somebody wasting. The time of the dispatcher what. This person, means is I'm, in danger my partner's gonna kill me please send the police that's, what she means if you like so. How, is it that the dispatcher. Is going to discern, this and as. I, play you the rest of the call until, the point of recognition if you like I want, you to notice not that he. Not that he's picking up on words, but, or even. Semantics, but he's picking up on turn-taking, placement. So, I'm highlighting the bits that I want to focus on and it's, the point at which the caller comes back to. Start her turn which. Is communicating. To him something like if, you were pizza this is where I would now be taking the next turn and that, is seems to be what he's picking up on so here it comes. Okay. I don't know it's wrong with the audio but I keep going. So. There what we see is the, call taker the dispatcher picking. Up pretty, quickly that. The this woman who's asking that, for a pizza is in fact asking for the police the police are dispatched and, they and they they save the woman from from her partner and, it's. Gorgeous because it tells you it, gives you it gives us something to aim for which is to understand, not just abstract, notions of context, but here the speaker is able to come, back at line 21 there do you know how long it'll be this, is the point where I would be asking you that question if, you were the pizza person, and he's able to discern, that from from turn-taking. Thank. You very much. Thanks. Very much Liz that was phenomenal. So you have questions about Liz's, amazing talk chilling. Talk really please, write them down now so that you can ask her at the end now I'd like to bring out konstantin sanders to talk about a future, for unhurried a time and. Hurry communications, hi. I. Was, going to have a handout, because I'm, a kind of old fashioned philosopher, but I'm trying to persuade you that I'm not a Luddite at the end of this talk so I thought I should stick, with. Slides. And. Let's. Get started, so, we. All know about the gains of. Communication. Technology, I don't need to tell you about the gains of technical, communication technology. But every. Gain has losses and there. Are different kinds of losses that come with, different technologies, I want to focus on psychological. Ones and what I'm going to call fecundity. Costs, and I'm using the term for Kunda T in sort. Of jeremy bentham sense of if. There's, some utility to. The technology. Or. To anything that. Utility. Doesn't bring further utility, so the utility doesn't, reproduce. In a different sense. Tech, communication, technology, does have fecundity, in the sense that the more you use it. It, kind of replicates, itself, and it's kind of designed in many cases, to be used more and more so the more you use it the more you need to use it but, I want to say that this has a kind of cost in terms of, productivity. In a certain, sense of, the term so, for example, we, might have. Technologies. Where, you. Get notifications. That someone has read your message and it makes us it's designed, so that we use it more and more and you might think oh we're, more productive, the more we use the technology the.
More We're doing things and. I want to suggest that doing things being busy, is not, the same as getting things, done. Likewise. We have all these notifications, popping. Up telling. Us. Something's. Happened, check this check. That and. It makes us kind of active, but, in a way which, I think has a utility, costs, of a certain kind. Now. You'll, see a message. From. My, from. A friend, and scholar who I deeply respect so, I haven't put her surname on there, but. I get, this message and hands, up I was on I was on Facebook doing, things, I don't know what hi. Constantine, I see you are constantly, active on Facebook. But did you receive my email send. A feedback please Marina, and. Now. Her email had come an. Hour, before. And it, was four chapters, of her book with, the request for feedback right, so for the kind of thought that oh look you, haven't had time for for, my book you're, doing this stuffy. The. Expectation. To reply, why because someone has seen that I am online right. That's the point they've seen I'm online and they haven't had a response to, their email and as, the online, offline. Distinction. Begins, to collapse a danger. Is that the kind of. Distinction. Between, the. Workplace, and, home. Or any other place outside of work also, begins to collapse so. If, like me you sometimes take a commuter, train you'll. See people who are already working on, their, phones on the way to work and on the way back from work but, that that work while. Technology can allow us to not be in at work and do things from home or from wherever we need to be it can, also in many cases people, are just working. Before and after work but it's not getting. Rid of anything in. Between, so. These are some some of the dangers, and, they. Come with high. Psychological. Costs we heard in some of the keynotes this morning, worries. About stress. Anxiety, and. This, sort of permanent, state of work. This. Is from a book that I be. Sellin who who's, here co-wrote, some. Time ago now the, myth of the paperless, office and. They. Write the, use of email in an organization. Cause an average 40%. Increase. In paper, consumption, now, initially, some, of that increases because people. Would print out emails and we do that a little less now than, we used to so there was this kind of treating, the email as anything else and then printing, it but, it wasn't the, only thing one other thing that sellin and Harper talked about, is. That, things that would have a high cost, in terms of time or money to, do physically, come. Very very cheap. In terms of time and money electronically. Like, sending, various, kinds of forms the, longer the form the longer the document, you, would think twice about do we really need this electronically. You, don't think about this as much it's just an attachment or, three attachments, or four attachments, and before, you know it you, have this kind of world. In which people are sending forms, left right and center things to fill in spreadsheets. And so on that they wouldn't have done before and everyone, is kind of busy and I've, had I've had colleagues in academia who, feel a sense of you, know research can take a very long time and they get a sense of satisfaction that. I've, done five things today because, I went. To the exam board and I fill those documents, and I filled my five-year. Plan or, whatever and I did my, form. For I don't know what and there's a sense of oh I got this stuff out the way I've been, efficient, but, actually it's stuff that you in a way didn't need to do an entire rolls get, created, by. People giving you this stuff to do and then, so then it's their job to do it so then they've got to do more of it and then, you're sort of thinking, you're being efficient, by doing these things and so you're always doing things but. Are you getting anything done, so. Some, people talk of unhurried. Conversation. About slowing. Things down I'm, going to talk, as. Fast as I can about slowing, things down in the next five minutes and. There's, a sense you might think that there's something sort, of maybe, lazy, about. Being unhurried, strolling. In the park. So. On and so forth and you might think oh you. Know this, guy's coming here to tell us to just kind of work, less, but. That's not really, what it's it's about I'm. Gonna give you a quotation just, from a personal. Letter this. Is from my favorite philosopher, Ludwig, Wittgenstein. And he, says my father was a businessman, and I am a businessman, - I want, my philosophy, to be businesslike, to, get something, settled to, get something, done so that's got the sense in. Which, there's.
Something. We're trying to sort out and our, work is to do it right, there's nothing kind of lazy. Relaxed. Or leisurely, about, it in a way he's quite an intense guy to be honest. But. There isn't this faffing, around this, doing, all these busy bodying, things there's, just the thing we need to get done. And. In. Order to do this in. In various, disciplines it requires, us to understand. Our colleagues. To understand, the person with whom you're, trying to get something done and that's, something, which I think fast. Communication, can be an obstacle to. Now. This is part of a general movement. And like. All movements we should be somewhat suspicious, of it I'm not going to read everything out on this list in front, of you and you'll be glad to know I'm not going to talk about everything, on this list in. Front of you and. You. Can get this kind of suspicion, of well why should we suddenly is this just the latest fad why should we suddenly do everything, slowly. Vick. And Stein who I quoted before, also, thought that philosophers. Should treat each other with. The. Salutation go, slowly and he sells he says elsewhere in philosophy the winner is the one who finishes last that's. Going to be built today by the way, but. But. There's. This thought that. When. You're thinking when you're doing certain things. You. Do you take, a certain care in them if you do it slowly and if there's the sense of why didn't you respond, to my article to, my book to the you, I've seen that you've read my message, and. So I'm. On the train with all these people, and they're. Sort of I, can, hear what they're saying and nothing is really happening and, that's time that could be used, much. Better and this isn't about philosophy this is about thought more more generally. There's. A book which I haven't finished because I've been reading it slowly called slow media. And. Jennifer. Raunch writes, who, wrote it writes many people have found their mediated, lives to be too fast too digital to. Disposable. And too, distracted. And she, kind, of tries, to fight this with things like, localism. Self-reliance. Fairness. And so on so in, a sense what, we've been experiencing, is there's been this kind of phase where everyone was always. On. Their, phone online, doing this doing that and then people sort of suddenly are moving. Towards, I don't know listening to vinyl and. Sort. Of staying. Away from social, media and, there's a kind of reaction. To this stuff and so, there's a kind of question here how, reactive. Are. The things I'm saying now, slow. Media and the other kind of slow things I've been talking about so far are, a little, different from from, slow communication and, I want to end with some thoughts, about. What. I've, been calling on hurry. Time. Now. And we heard some things this morning about, the right to disconnect, for, example, and over, lunch I've been talking with people with. Various, ideas that I call patches, about, how to fight. This, kind of pressure to, always be doing things, but never getting anything proper, done, so. We people, have different strategies about, checking. Email, or blocking, email from coming, disabling. Notifications. Dumb, phones that only let you do. Certain things with, them and so, on but. And I think while these things are all well and good they're. Really just patches, and what I mean by this is that there's. Something negative that. We're trying to kind of stop to try and keep in control. But. They're, not something. Positive. And it's not the creation of new technology, that enable, that enables, us to think, slower, and to get serious, things done and I think this requires. Reimagining. What communication, is for and I, think the, clues kind of in the name information. Communication. Technology, there's, a there's an assumption that communication. Is there to, exchange information. It's philosophical. Roots go all the way back to. John Locke the philosopher. But. It's there in a lot of contemporary, stuff and if you think that the purpose of, communication. Is to exchange information, then. All you're really interested in is how, much information can I exchange and, how, fast can I exchange it and you and there's no so, it's all about volume and speed in a way but, I think we need to rethink what. Technology. What.
Communication. Could be for, and in, this particular context. I'm interested in what it could be for within. The workplace but I think there are wider questions. I've. Talked, a bit about understanding. One another how, can we use technology to better. Understand, one another speed is not often a, good, way of understanding other, people, and. You I'm sure you've all experienced. What. Happens when we all too readily reply, to an email for, example. Getting. Things done I've spoken about and also, other things like building. Camaraderie between, colleagues, this, is something that technology. Can. Be can be used for and these are just kind, of a few thoughts, I'm. Not trying to quickly. Provide, an answer to. Everything but, I do think we need to rethink, what. Communication, is for. Thanks. Very much. Thank. You can't think that was fantastic all right now it's my great pleasure to invite to the stage you, von Rogers. Good. Afternoon how many of you have an Alexa, at home put. Your hands up I'd say that you are. Representative. Of most people, in the US and the rest of the world, they. Are becoming part of the family home and whenever I tell people I do research. In this area they always tell me an anecdote, about how it's changed their family life, how. Are these types, of devices going, to affect work. And my talk is going to be about thinking, about how these types of voice interfaces. Can, be used in business I, want. To go back in time though back. In the 80s, when small. Groups collaborated. And how did they do that around technology well. One of them would have the mouse and the others would look on and they you might be able to grab. The mouse or wait until someone would pass it to you and then, there was the vision of, people. In Star Trek talking. To the ship's computer where they all, could interact and have multiple devices. So. There's a big difference between. Then. And, the. Star Trek I want. You to look at the affordances, now of these smart. Speakers, and you, can see here these this family are playing in that game are paper scissors. And stone with, the device and the their attention, is very much, towards, the device but also each other and so. It sits in the middle between. These people here now. Look at where, alexis has been developed, in business. And the, idea is that they're going to be used increasingly at. Work and it's, going to be you, know placed, somewhere in, on. A table, in, meeting, rooms and. There. Are many claims that are being made about what, voice user interfaces, can do for businesses, and, it's. Very apt for this conference but, sorry the summit that the focus is largely on productivity. And efficiency. Gains, and there's, claims being made that, they, can provide an intelligent, a system. That will help people with their tasks, in particular. Providing. Ways. Of scheduling, meetings, and while they're doing their multitasking. They, can write up notes for you make suggestions, and collect, resources. All enabling, people to work more efficiently and, has also been claims made that they can improve. Communication. And. In. Particular the, claims that are being made is that they will be able to listen to, conversations. During meetings, and they'll, be able to make notes on important, points, and they'll. Be able to help employees. Identify. And retrieve. The right information, that. They needed particular. Times from, various sources and as. I already said they'll be able to create. Reminders. Summarized and, tasks and so on and so, they will be. These. Types of assistance. That will do things in the background and this is voice Sarah's Ava and as you can see she's. Listening to what people are saying and then, writing this down and summarizing. It and. So, we're all working in a background to make the the, workers life more, efficient, now, my question is.
Can. They be more than a flying cliffy, can we do more with voice user interfaces, than have them doing our tasks, in the, background, and that's the research that I'm currently doing with them Leon, and Ethan, looking at how we might rethink. Voice. Assistance, as voice collaborators. And. What. We're doing is trying to design voice. Collaborators. If you like and we're doing this by thinking about when can they be proactive. As opposed to always being reactive. So currently, you will ask. Your voice assistant, questions, and what, we're thinking about is what would happen if if the voice assistant, could ask you questions, or make suggestions, to you whilst you're working so, how do we think about switching. That what. Sorts of roles might they play, could they be a moderator, in a meeting could they be a facilitator. They facilitate, conversation. Could, they be an instructor, they can instruct you what to do and, if. We are shifting I'm thinking, about voice our user interfaces. When. And how should they intervene, in human conversation. Should they but in like, human, beings do in conversations. Or should they be more polite and, also how should they be incorporated. Into the existing, ecosystems. Not, Ecology's. Thank, you very and. So. What, we're trying to do is to think about how we can, turn virtual, assistants into virtual collaborators, and we're. Doing this by doing. Two strands. Of research one is to you think about how we might use AI. To. Understand, conversations. And context, awareness in, order, that when. These voice assistants, are going to interrupt or make suggestions, it's timely. And that, they don't disturb. Or disrupt. Ongoing, group, working and then, we think that making them human-like, is also important, and not, too so. That the people, think that they don't really are humans, but that they are polite, and they behave like humans, so people will. Take, them seriously, and adding. The arms and our house makes. Them seem more human-like. So. Those are the two strands of research that we're looking, into and, what. We're trying to do is. We. Want to not, to, make people's work more efficient, or to optimize, their tasks but we want to to, amplify. Human. Cognition to if you like empower, people and, so. Our focus is not, to design a general, voice user interface, but, for specific context and we're working with Great Ormond Street Hospital looking. At how, clinicians. Who, discuss. Emerging. Data clinical, data how, we could help them to. Think. About the data and come up with different explanations. And and ways in which to use that data so, just as in the previous talk, I'm, all for slowing, down interaction. Rather, than speeding it up and to get people to externalize, their thoughts and doing. This we're arguing that that's this can help with creative, exploration. Of ideas and. Also potentially, leading you to think differently to go down different routes and avenues about what the, data is that you're looking at and even. To see new connections, so, our aim is very much to encourage more, talk around the. Technology, rather. Than trying to speed up what people do. So. What we did it's currently doing is developing a system called voice phase. And it's a multimodal interface. And you. Can see the Alexa. There and Alexa. Or this device. Here is. Being, designed in conjunction with. Data. Visualizations, and the, way in which it. Works is that we have used. Existing clinical, data and this is about obesity levels and, that. The voice fist can make suggestions to, small. Groups about what, they might want to look at next and they. Can talk with voice, fist to ask. It to show different, types of data and this data that, we're using is quite, simple at the moment to, make our lives easier but it's looking, at obesity levels for developing, and developed countries, for boys, and girls and, for adults and children and even, with those variables, it can be quite complex as to looking at trends, over time.
So. We. Are modeling. Or trying to think about the conversational. Mechanisms, rather than trying to do natural language processing, and. Some monitor aspects of the conversation, as. To when it's opportune. Or appropriate, to intervene. During. Conversations. So we're looking at metalinguistic, meta. Linguistic features, the, silences. That might go in the arms, the arse the, pace of speech the, velocity, of it the number of questions asked, the, body movement the head movements we're looking at a number of these things as to see which of these might be useful to. Use. To know when to intervene and, our, output from, visi is to make suggestions, at, opportune, times for, people looking at the data so things like do, you need a hint for, analyzing, this variable, what, do you think of this, shall. We move on did, you consider the difference between. Developing. And developed countries. What. Might have caused the sudden spike in the data and if, I would say, one of them is slowing down in recent years which, one would you say it is so, it's constrained. What. Visi can say by, the context. And that makes our lives a lot easier. So. Here we are currently running experiment. Looking. At whether. Or not having voice inputs and voice output, actually, encourages. People, to speak, more and. To discuss, the data and. We're comparing it with a touch, interface, to. Show whether, or not or to investigate whether or not there. Is a difference in what people say and do. Having. Voice. Input and output or just voice output, so we. Are using a Wizard of Oz technique. For those of you who are not familiar if you look at the image on the right, that's our Wizard of Oz he, is pretending to be busy and will, press, buttons to give canned speech at appropriate times, and can, hear the, participants. In the other room what they're saying and what they're discussing. So. This, is. Some. Of the participants, and you. Can see pieces of paper on the table and those are things that they commands, that they can say to visi to. Change what's appearing on the screen and. Sometimes. When they are a bit stuck they've, been asked to describe. Or. To explain the data that's appearing it. Visi, will say something so let me see if this will play. So. They're not saying anything. So. They won't say anything visi says something and that triggers a discussion. And. Then when they get stuck visi might interrupt. If. I would say one of them is slowing down in recent years which one would you say it is. So. That. Again, they was they weren't really sure what to say visi, into interrupts. And that triggers a conversation. To, continue but. This is only human and sometimes, visi gets it wrong and will. Think that there's a silence, and will interrupt the flow of conversation and. As. We know in humans we're having a you know talking to someone if someone tries to interrupt and that can derail. Our train. Of thought so here is an example. So. They were able to get back on track but, what, we're looking at is when our opportune, times for, the. Viz to be able to interrupt and to make suggestions if you like to scaffold the, ongoing. Conversation. So. We're, currently finishing, off running. Our participants, look, at comparing, the voice condition, with the touch condition, these are some of the initial findings which we're finding that participants. In the voice condition, whereby, they. Speak. The commands and visi speaks back they, speak more loudly to, each other and expressively. So there's more intonation is the pitch of their voices is L more loud and that. They more thoroughly, explore, the data visualizations. That appearing on the screen there's more in-depth discussion and identification. Of trends, going. On and that they all spend more time actually looking at the, visualizations. Whereas. In the touch condition, where they. Just touch a touch screen when, they want to, ask physi something. They'd. Sometimes miss patterns in that and trends, in the data so the, power of voice here is that it can trigger. People. To, discuss, more, outside. Of. What's. Appearing on the screen so, I just, want to I'm coming towards the end of my time we're looking at these as proposed, features, to model when, to intervene in calm Visayan so turn taking patterns, we heard in the first talk is an important, speaker, activity, shared.
Attention, The commands given by whom and the. Degree, and the speed of exploration, or of the datasets appearing, on the visualizations. So. I don't have time to go through our system architecture, but we are developing an, architect by which to analyze what's being said and. Just. To summarize I. Think, that the value of voice user interfaces, in workplaces, we. Can think differently, about those, we can encourage more discussion, and even more equity of participation. So busy could say. Yvonne. You haven't said anything for a while what do you think and, we can introduce something. Like that it can increase creativity, by, exploring, different trains of thought and questions and most. Importantly, we want to switch from voice assistants, to voice collaborators. And when. To intervene a. When voicing assistants should intervene, um is key. Challenge, and finally, I was struck, by a paper I read where. It's saying that simple, models of conversation, can lead to surprisingly, complex. Emergent. Outcomes we don't need to do a, full scale and natural language processing if, you have a constrained, context. And the context, here is one that looking at data visualizations, and discussing, those and so. We're able perhaps to get by with just doing simple, analysis, of what's being said what mechanisms, are being used in order to think, about interventions and I think at that point I'd like to say thank you very much. Thank. You very much Yvonne and now last but not least of course we have the amazing bill Buxton to give, us some amazing thoughts both, on his stuff and also of course wrapping, up and some thoughts about the speakers. So. However. In. Some sense this whole session is. Each. Of the talks fold. Into the same talk from, just different perspectives, but just as this session, to, the previous session on. Devices. Working together is, another chapter in the same talk and just as Sarah, or glorious Marx, talked, earlier, is actually part of the same talk, in the same session and. It's really interesting to see how these things scaffold, them each other so and, some of that's actually kind of backs it but some of it's not and. I'll try and keep. That in mind as I go along I'm gonna start with the this is a picture of Mabel and Mabel was a motorcycle, I used to have said yeah w, 900cc, I took. To Cambridge with me I flew it over when. You could do that for $200. From, Toronto, to Cambridge, with my seven-year-old son and, we. Landed at Stansted, putting, our letters hook up the battery and just, drove up the Cambridge, and. I would tell you that because my son was 7. He. Had a very very short attention span, he had some learning disabilities issues. At that time and to. Keep him falling asleep on the pillion, on the back seat especially because we just come on an overnight flight I had. A little microphone, and, we drove, to Cambridge. And. I'd. Never had a conversation a sustained conversation longer, than two minutes or a few minutes with my son up in the first seven years I think that's maybe an exaggeration, and faulty, memory but but. The fact was, that, really, poor microphone, with. The ambient, sound of the wind and everything. Else the. Entire drive through Stansted to Cambridge we. Had a completely, sustained. Fully. Engaged. Conversation. One, which in some way senses, wasn't repeated for another 10 years in, terms of its coherence and length, now. What's interesting is allowing that conversation, is saying Wow is it evergreen it's, amazing and stuff like that but, we kept, the thread through that whole thing and. For. Me that's always been a reminder, that if, you take away bandwidth. And you, take away modalities. You, can sometimes, improve, the communication, because.
It's Focused, and there's. Nowhere else to go the, distract even though the scenery's there there's. A different, kind of distraction that caused interruptions, the server is talking about earlier that's. Stuck with me and so. I'll give you another example how that dist, anecdote, which is just helped shape my, my thoughts comes. Into this, I work. With a couple really neat people here in a project called soundscape, project which is basically using 3d, space Audio to. Enhance. Mobility. For people who are sight impaired, and. One of the founders, of that project is guy named Amos Miller who, lost. The sight just, around the time he graduated from University, and. He. Is insane. Actually. In. The most delightful way he decided that he was going to run a, regatta. Kayaking. Regatta, for blind. People who. Had never been in a kayak before, and haven't. Go into a competition on, Lake, Sammamish just, down the road from here and. What they were doing and this is one of the participants. And and. This is a guest, last October, and the map and the right sort of shows and what he did is he put markers. Like any sailboat, race or boat regatta that you had to follow a course of seven markers, now. The markers, were spatial, markers which are put on the lake they were virtual, locked. And sound but, because of this special, audio that. Enabled, you to hear things outside, your head in locked, in in space it's that kind of augmented reality using, non-speech audio and that's really important the communication, was non-speech. Blind. Person was in the helm piloting. A sighted. Person was, in the front of the do two-person, kayak. And. First of all all of them never haven't been in a kayak before finished, the course and. That was so first of all he the audacity to even do this in the first places is I told. Amos he's crazy don't do it and and, he did and I was wrong and he was right but. Here's the point. Then. We put some sighted, people in, the, boats with. The same headset, the same software, and. The. Blind people kicked their ass. He. Sir asked us well why. Because. They're so ocular, dominated. That. They were being distracted, and second-guessing. And. Audio. Had been pushed into the background and. They. Knew how to. Hear but. They clearly didn't know how to listen and, they. Didn't know how to listen with confidence, that's. My theory no proof okay that's not experiments, that's my interpretation. And I'd, argue that there's, these potentials, and humans for communication, let's go way beyond speech and this. Obsession with speech it's, important, it's hard but it's not sufficient. But. Also to. Look at these edge cases and see where we might be able to learn and also, learned that we can provide information in, the periphery, the same way you can tell if it's raining or windy or. Or, if. There's a car coming without. Interfering. With the, foreground task and so you can push things in the periphery and there. Do all of our design issues have been around foreground intentional, actions as opposed to background peripheral, in what. And Sarah Bly and um, come. To a minute when, they're talking about big screens thinking about them as information. Service awareness, service how. Do you serve up awareness, in, a way that's not interfering with your foreground tasks, and, so, you can actually focus and not be interrupted and integrate, sometimes that knowledge and the. Lessons here are. Really.
Important To think not, only because I think it's a it's a great project but in terms of what we can learn from it we can bring to the rest of our practice. Now. There's. This. Whole tension that's in AI and anything so it's strong specific. Week general systems and that, the and the conflict, between the two but. There's. Two Swiss Army knives here now Victoria, Knox that make Swiss Army knives is probably one of the companies was most influent, impacted. Negatively from, 9/11, because we, saw all carry these little pocket knifes with us and none, of us do who go, on airplanes because they keep. Getting. Confiscated, but I want. To point out the, difference between the, little one which I own and the. Big one which I also own the big one cost me four but I'm like four hundred dollars has 80 some odd tools in it and and. There's. It's really interesting because. The. Big one has. Everything and, it's. Useless except. As a piece of art or a piece of absurdity, where's. The little one which, has way fewer tools, because. It's always with you and it, doesn't interfere it. Brings real value but the choice of tools are really important and, yet. There's. No. Tool in that big one which I don't use sometime. But. What we miss in this and I'd argue that that. Big one is what. People have in their mistaken, interpretation of, what Mark Weiser, meant when he spoke about ubiquitous, computing that's. What I think it is the Internet of Things we're just going to populate the world everything's everywhere all the time for everybody as, opposed. To a, few. Things, they're the right things at the right person the right time the right place that are with you some, of which are portable, and wearable with you but, the rest are better than the environment, using. The Swiss Army knives it's a really good example it seems to me the primary, use of that big one is on a camping, trip is to use it as a boat anchor. Now. One, of the ways you can look at this is this if you look at that big one these. Are just some of the tools my weight has two socket, wrenches not just one is. You, get the tools you can this all saw spoon fork scissors leather punch and nail file corkscrew, we. Can say well what's the function. But. As soon as we've done that we look at the functions and map them to the tools you, can associate a place with every one of them, so. Why would you have them in places where they're not typically. Used and, don't. Design so, you always say well that might I just might need it I might need no did, some work on the probabilities. In. General, this. Is where you need it park it there and then take special action, for special cases as opposed to be, prepared all the time Saul. Greenberg has spoken in their session before we, go, back. Country skiing now we can load up our packs with safety gear for every eventuality but we so overloaded we're, gonna never get up the mountain that we're trying to climb, up to ski down, you. Have to find a compromise in a ballast in, that as you do in what we do and, the, core of this it seems to me was came from an exhibition I went to I think it was in Munich or Vienna of Louie Kahn's work who's one of my favorite architects and.
He. Has this line which i think is fantastic thoughts, exchanged, by. One another are not the same in one room as in another their. Semantics, based in location, and place. Matters, place. Space and place. And space in every, dimension time. Relative. Places of a talk overseer I'm coming really close to get it closer right there's a very different thing if it was a stranger. And not a photographer, it would be a different, act even though in the same room I'm in the front but because I'm in the front I'm the speaker you're not anybody. The back of the room knows that, even. If they're from a different culture. It's. It goes beyond being in a room, it's. And it's the movement and a gesture of movement as well that's really critical and. It's. Within the room and so I'll give you some examples of just some work that Abby's Ellen and I and some, others did at, Toronto and this is actually a floor plan of my office and, this is work that dates back to. Some. About 89, to 94. So. That's my office you get my desk as part of obvious where it is it's the one on the, top. Left corner, and. So when I'm sitting at my desk and I'm working there's my workstation as. Of. 1993. Probably. And. If, somebody comes in to join me electronically. From telepresence, where should they sit if we're having a conversation. Well they across, the desk and. Principally. Or beside, me at my desk however. You want to imagine it but where. We have a shared workspace and, it's intimate because we're producing. Work but a certain type of place but, I would tell you it's very very clear, that if. You're, a student I'm about to give you a congratulations. Or say you're, you failed, they're. Gonna sit in the far side of the desk I'm gonna be sitting this side and they would never come to my side that's something, that Abby would do because she's a close, colleague but, that's a lot of interesting, permission that if. You're, there and I'm here I'm professor, Buxton and you're not and, it's. A formal stance if. You're calling in a friend that's it differs, there's these permissions, this is called the moral order and this is kind of stuff, that I've learned from Richard, Harper that that, this whole notion about how we start to define space and movement and and, and from, these knowledge --is which are culturally. Specific. And. And and and can evolve we. Can decide how, to make informed decisions to, limit, where we put technologies, that don't, need to be there in the first place so we can always try to keep to the minimum necessary to, achieve our, goals and we say the same thing if somebody comes to the door I know, if they're at the door in the middle talking, to Abigail and, we're having a conversation, I can complete, lately and, politely ignore, them and if I ignore them for 15 seconds, they know I see them I know they see me, and and. They will leave and I'm. Not gonna interrupt the flow of the conversation because, they have that awareness that, I'm in a conversation, and they know that moral order and the soul and the social mores and they, will just apart in about that amount of time because of where they're located they're. Not in the room and and, away, you go, and. So. You build a proxy, to do that so we place the video there and what I'm trying to say here is by putting the more of the right technology, in the right place for the right function then. You have less complexity, for. Every location in this room there's, a place for the remote person, to occupy, you. No matter that's consistent, with the social mapping. Of that, space and so. Even, when they're approaching, down the electronic, corridor you, use aircon, so you can hear the virtual. Footsteps, coming, so as they approach you can hear them coming to repair it's not a sudden switch it's an analogue gentle approach because this approach graceful. Approach departure, which came up in Sara's talking came up earlier. Today I mean in the session as well and, and, then, if you are going to sit around in a collegial way around the table there should be a place there to occupy and, if, you do not have those things the conversations going to be broken because. You cannot use transfer. The skills that you've acquired from the lifetime and living in everyday world to, the this, hybrid. Virtual physical. Space as you and. So. You have to think, of all these ways to go over these speed bumps and. You cannot have flow you cannot have inner. Uninterrupted. And, if you look at your conference rooms for example how we think. About these things and by the way this, has nothing to do so far with what the technologies, are is simply, where they're deployed and, remember. This has all been done and working and functioning in in.
In In the 19. It. Laid it 80 80s. And, early 90s. When. I'm giving a presentation at the front of a conference room Isis now I'm up front my work is up front this, is before you had projectors, that cost less than a hundred thousand dollars and so. We. Had at the time what I thought was a huge display. But. I'm working there and I can write on with a light panel where you go. But. When Gayle is now you're looking at it for back from the front of the room presenting. To, the group the. Audience is in front of her around the table and there is a proxy, at the back so she can see one of the team members who's from Ottawa putting. Up their hand ask a question and they're seated where the audience is seated, even. Today in Microsoft. We I'm, typically. If I come into a meeting I'm on the projector, at the front ten, times bigger than anybody else in the room peering, down and, it's, completely, inappropriate. And, it's. Got nothing to do with her ability to have the technologies, it's not understanding, the relationship, at technologies, function place and what the moral order is and how, conversations. Work it's. The social science and the architecture, that's broken not, the technology the digital technology, but, the main point is. There's. A total awareness of what's doing and and you can use the skills raise your hand you got a question that's fine I don't screw around and see this person. Looming don't worry do you have a question and. Likewise. If the remote person's there they have a separate space where they appear on the smaller screen and their talk is on the big screen you, do not put the if. You have your face, of the real person on the same screen as the work with the camera somewhere built in the screen it's already, broken. It's. Not about interface, design it's out of my face design get, your face out of my work that's, my works place but, I'd love you to be here so I can have gaze awareness, that you're here and the works there and I can direct your voice you voice, is going this, is all things. Attributes. Of how we design and lay out the technology, architectural, if you will that, actually takes out the speed bumps it gets rid of these interactions, and if you start to go back and if you had this type of architecture I would love to, have glorious, studies about where the eyes go, redone. Because they would change dramatically, because actually, the movement from eye to that screen that that screen would be show the continuity, of action and the continuity, of conversation, as opposed to the disruption, of the conversation, it's not about the screens it's about the content of why you're going the screens and how these things chunk together and. If. We come back where the task space is in fact the person they're just giving a talk with no slides and they should be up at the front full, size in. The case we only have a big screen skree at the front. So. What does this all bring us to in terms of how we think about things so I worked I had, she had the pleasure of working for a number. Of years with Mark Weiser on the ubiquitous, the, ubiquitous. Computing was coming up this sort of most. Intensely, from let's say 89, to to. Adelaide's. 94, but. In.
His Paper, the computer. For the 21st century he, talked about ubiquitous computing he talked about this notion of equity which. None of us knew the word we. All had to look it up and. And. And. But. These two principle were is that things were worth everywhere. You needed them and they. Were transparent, and, that sounds like a paradox, but just, my example of that is simply saying, if. You, walk in my house you come to the living room to see and the dining room you see a dining room table the dining room and then, you have to go the washroom you see a toilet in the washroom you come out there's you go to ignore it it's transparent in. The, same house same person, everything else they put the toilet in the dining room and they put the dining room table in the bathroom all of a sudden it violates, the moral order and you will notice that really, will I mean it's, you're. Probably gonna start questioning the relationship with me that that, that's the situation I'm. Just saying it's not about the, visibility of the toilet or the dining room table it's where they're situated and, and. And is it appropriate to the purpose in the function and and. The moral order of that particular culture, and. So. What. Happened, is, ubiquitous. Has this notion and its definition. If you look it up on the Oxford English Dictionary is. Everything, everywhere all the time for everybody it's it's ubiquitous, and. We, think about that because we're geeks and to think about the technology because, we're materialistic, we, think of that everything is everywhere, as. Opposed everything we need is everywhere we need it that's. Okay but. That's not how the Internet of Things and so on interpret, it. Mark. Used the wrong word. There. Is another word, that's. Adjacent a close cousin. That. I only discovered because, I was trying to look up I just am obsessed with this kind of stuff there's a few things I'm anal about and one of them is I really, understand the language. And I. Accidentally. And trying to get the precise definition in, the OED for, ubiquity, I came, across a word you by a--they. And. You by T has. A very, different notion, the. Core, of it is this notion of place. That. It's and so it's it has this thing and, place, in every dimension. What. Mark talked about was, you buy ax T not ubiquity, and. Likewise. Nobody knows this term so of course it's neutral, its unencumbered, with baggage, and. So, then one of my things here says by changing our language I let's. Switch. Over there so I. Believe. That we can get rid of the speed bumps and. We can get rid of them. By. Using this, notion of design think about things architectural, II and, understand. That and this comes back to the early talk, to. Get away, besides. Part of communication, is to be able to commune commune with nature commune, with your environment commune, with your family and. So you have that peace and it is slow and it can be peaceful and thank you. Alright, so I want to invite now all the speakers back onto the stage and we can have some questions so I'm sure you've been industriously, writing down, during. The during. The time. All. Right here though are bill. LaVon Liz and Constantine so. Some questions from the audience. I. Know. So. On great. Thanks. Everyone, for all, of those inspiring. Talks I wanted. To explore. This notion of. Conversational. Agents, and. Whether. They need to be human-like so this is something that you said Yvonne. Can, you unpack that a bit because. There. Are many reasons why you might want to do that or not do that so is it about natural language understanding is, it that they're contextual, is it, that you want some notion, of emotion. Embedded. In these and. Especially. In a business, context. Why, is that important. I'm not sure that I completely. Buy it so maybe you could convince me. That's. A good question and I think we made that decision as, to not. So. How, is that our, voice. Assistant, going to appear if it's going to interrupt would. People take it seriously would, they listen to, busy. Saying something. And we thought if we made it more human-like they, may based. On earlier work from. The. Stanford, Nass. But. If. It's polite, then people will respect it so that's basically, our, that's. Our take on it it's not about emotional, contact that people will, relate, to it more it's more that they'll take it more seriously, and. In this context, the business contexts, that we think it's important that they will listen and they will respect what.
The The, agent is saying I think in other contexts, it may be that you don't want it to be human-like and, I think the the the. Ongoing debate, about duplex. About whether you're, duping people into. Thinking it's human when in actual fact it's it's an agent I think that's a different context, and people, don't like the fact that they're being duped in this context, we're using it people know it's an agent and it's been designed to be polite and civil I. Want. To get a clarification because, there, may be it. Duplex. Domains means it's two-way, conversation, as opposed to duping. But. In. The telecommunication, so did, but but the two points are perhaps, the biggest but also in. Speech. The. Languages, are, using. Spoken language in both cases and the thing I find interesting in some cases that, it's. Less common in the world is that speech, is answered by non, speech or by, gesture or by some, other mode and that's far more common when we deal with technology and therefore, the two, directions. Could be in completely different modalities. I agree. And maybe they have to compensate for the fact that you can'
2019-08-07