Richard Harper - Are We Made Smart by Smart Homes?
off you go um thanks Kia um uh what I'm going to do today is a little bit about me and then I'm going to explore some um some technologies currently available to help us live in Smart Homes um say explore a little bit about them do a little bit of computer science um then explore how we might think about um what smart home is how we might shape our thoughts around what's smart home is but I also want to force a shadow and something which is although I might have originally been a professor of Sociology for 20 years 30 years ago actually I find the most interesting thing is to actually build something when you build something not only is it difficult but enjoyable tasks when it works also tests your propositions as I'm not a sociologist who's just critical the whole point of looking at three smart handles is to build Technologies for Smart Homes whether you call them smart perhaps is another issue if they make home life better in some way that's my ambition so I end up with some um devices which we built and tested don't end up also with a little video which will not tease which is a bit playful or something we actually built but um so it might be critical but that's not we're thinking about what I'm doing so let me um zoom dialog boxes that's what's going on there we go so um [Music] I have arranged it okay um so I've written lots of books um many books I just live in a few examples of them um a book on choice and how we think about Choice that's obviously relevant to lots of arguments and artificial intelligence long ago I wrote a book called Inside the smart home I've written another book on the smart home called the connected home I've written an early collection which was a pretty unpleasant experience trying to get philosophers to actually abide by what you ask them to do is like pulling putting teeth I must say I asked him to write about philosophy and Trust in Computing and let's put some great papers in that I wrote a book on texture which is probably the one I'm I'm most um proud of in terms of expressing my research and then a book made a reputation on the myths of paperless office but lots of other books too um what I'm actually talking about today is for my latest book and my magic of my age probably the last book um and it's called the shape of thought and it's basically about how we shape our thinking for different kinds of situations and I'm going to start this off by making comments about being an Englishman Stockholm um there's this picture of your underground or tube as we would call it in England and one of the things that um I found very difficult although I was reflecting on it this morning I don't find it difficult anymore is that when you first get on the tube in the stock comb and when you first go into shops in Stockholm you don't do something which just ladies doing that this lady is looking at me now on the tube in Stockholm no Swede looks at anybody else nobody catches your eye nobody glances at you now what does that matter well in London when you go on the tube you're standing very close to a guy you're just glancing and you notice that he's noticed you and you're moving closely so he moves away so you have this little dance to do with glancing and you glance to see whether you should move or not move but in Sweden Nobody Does that so what did I do the first week I was on the tube I kept bumping into people because I'm waiting for them to you know I'm really on onto the underground with my bags thinking I'm a big fella I'm not small so I'm going to take up some space is the lady in front of me going to notice me well I expect her to look up and give the impression she has noticed and make a decision she notices me and does nothing which is often the case I move around but in Sweden that doesn't happen on the tube in Sweden I had to adjust my behavior now why am I mentioning this what I'm trying to say here there's a moral here about technology an AI in particular and especially deep learning systems deep learning systems are incredibly effective they might use an awful lot of computational power they might struggle with graphics cards but they're patterns that they work on are determinable and they can be inferred from data they presuppose as a pattern which also means that those systems cannot adapt they cannot find a pattern of not being trained for deep learning is fragile in that regard it's very good at delivering inferred patterns of patterns that are known it's not so good at inferring patterns that are not known as it happens this is well understood in the Deep Learning Community even people like Jeff Hinton admit that deep learning might be a at end so what they're doing now with gpt3a gp3 and things like that is what that re-labels generative deep learning and the idea is that generative deep learning combines various deep learning patency inference engines to do something new I'm not sure they're successful but I'm going to come back to that at the end of my talk so bear that in mind so what is what are Smart Homes Smart Technologies the kind of Two basic kinds ones that support things in the house the intelligent manager heating most of you probably got these Google apps so quite nice they're very straightforward they're very simple to engineer some people have these delightfully amusing autonomous vacuum cleanings which our cat used to love but we've given up with that now um and then you have more technology more AI not for the home but to let you escape that and in search engines and so forth but the important thing is that what we have in terms of artificial intelligence as artificial intelligence which is quite narrow quite specific you don't get digital butlers you don't have robots with English accents going around saying would you like teaser can I do your laundry for you can I pick up the chaos from last night's drinking that's an aspiration but certainly not uh and achievement so what shapes the thoughts of smart home researchers two agendas I think and the first is what can we do now and the second one this one most of you are familiar with the claims about generalized intelligence which is a manifestation of the related but um generative AI I just want to look I can't Smart Technologies before I come back well let's look at some an example a delightful example a fun example a lightweight example of some heavyweight technology if you think that language processing natural language processing is a heavyweight technology it certainly is in terms of the years that gone into teaching deep learning algorithms and inference engines but when you buy it when you buy an echo and speak to Alexa doesn't seem like a big bit of kids so with Alexa all of you know what it does I presume you can talk to it they'll do things for you but other things um it's appeal is it does what's called offers what's called a natural interaction and by natural interaction this is one of those Mr begotten phrases computer scientists use because people speak to each other computer scientists think that's natural interaction but we do learning things when we're with each other I stand at a certain distance is my distance my proxemic relationship with this gentleman also natural how do you know what his natural it's a funny phrase but speaking seems to be natural so therefore things like Alexa offers you an easy mode of interaction a natural mode of interaction and with Alexa you can ask it to do something for you you can ask it to search you can ask it to search and buy it offers you this if you ask by me something by Merry Christmas tree Alexa will offer you a bunch of Christmas trees but Studies have used as safe they don't seem to be very happy about something they don't seem to like long-spoken lists so they seem to interrupt Alexa before she if I can call it she finishes the lists why is that a problem of Alexa or is that a problem with the user well I'm going to explore this a little bit more but foreshadow what I'm trying to say it's a conversational rule of Audrey love that users prefer to agree and they prefer to the conversation to move on so if someone starts offering a loan list the recipient the personal listening gets a little bit embarrassed and said no I'll have that one because otherwise it's the kind of inelegance about it now is this a rule well Harvey Sachs who wrote discover the hardest just used lots of examples to show basically it's a kind of morality you try and have balanced turntaking but one consequence of that is that when Alexis starts off in a long list you stop her and you ask for one of the early items in the list now um what does that mean well the technology device state-of-the-art natural language processing and it does what's called best fit so when I ask for a Christmas tree it doesn't ask for some it doesn't assume some particular Christmas tree it means general Christmas trees doesn't say large small it just generalizes from it the best fit is pretty good for most most consequences sometimes Alexa comes up with really weird things for you so when I ask for a Christmas tree it might offer me theater tickets for a plane through a Christmas tree but mostly it doesn't it doesn't it doesn't fail the crucially I wanted to highlight here is that Alexa is a smart technology the problem is not with the technology the problem is with the user behavior and what I want to suggest from users behavior is we can understand it by thinking about the user from the Technology's point of view I wonder now those of you who do data structures in computer science might be vexed by my summary or some aspects of how data is stored on the computer but bear with me is it it's a metaphorical argument there's an analogy I want to suggest between data structures in Computing and speech driven search inquiries here are some data structures we have a list a list it's just a long list and a list is organized in all sorts of ways most often alphabetically lists are often joined with other lists and they have another order often again alphabetically that's one way of storing data another way of storing data computer is to stack a stacking first in first out you just stack on top what that means to the computer is if you ask a computer to a truthful mistake it takes the first out I mean not first the last out the reason why I have different arrangements for data storage and computers is all about speed all about efficiency it's all about what the computer is trying to do not really about where the data is stored another type of data store is an array and an array allows the computer to see all the instances at the same time this is computationally quite burdensome but sometimes useful what is a spoken dialogue search query with Alexa like if it is like a day to stay I think it's like a static procedure I don't want to listen to a lot I just want a quick answer so in effect I want what's lasting is lighted speed the last at the first out now if that is the case who is in control of what is lasting first down it's Amazon when Amazon offers a set of targets or Services they're called in search engines research it's going to offer items that it has some preferential deal with the first five in a certain Alexa are Amazon promoted Goods Amazon knows that you're probably going to listen to three or four but you're not going to listen to 20. so if they sell those three or four first ones first in first out stack type items they make money now to do just to help understand this example traditional graphical user interfaces for search engines can be thought of as an array when you see a search engine results page it presents the number of different targets either from top to boss more left Variety in carding but in my location think of it like an array what I'm saying now is that when I interact or when people interact with Alexa they're interacting with a stack like procedure it's taking me this is where it's taken me is this intelligence is this system intelligent yes but the question is who's intelligent it's good for Amazon it's not necessarily good for the user in a sense the user is being lazy so am I about to say that people have Amazon are foolish no does it matter but you can be whoever you want at home why does it matter if you want to be lazy why does it matter if you use a search engine which doesn't search it just gives you the top of the stack at the top of the stack provides the seller with a little bonus surely you can be whoever you want to be at home what's the problem with smart technology which is smart for the technology but not necessarily smart for your pocket thank you so what is a smart home what is a home is it the smartness of the technology that makes the Home Smart or the smartness of the user or is it a sum of the both so if you have Alexa which is smart for Amazon and you have a lazy user a home occupier do you put them together and where does it put them or some other criteria how do we shape our thinking to think about Smart Homes don't forget my book I'm writing is called the shape of thought and it's about how we shaped out ways to think about things and now we're going to go into I think some bit more fun my view about people is that we can be lazy I'm late middle-aged man I don't bother fixing lots of kicks because I have teenage boys they get on with it I don't know that's it but we can also be fastidious we can be Artful I'd like to propose that we express that in the way we make our home our home is it our material Arrangements which express who we are now we are Supreme we are what are home servers clearly that's kind of a bit of a constraint for some people but homes say many things when I say it's a constraint at my age I have more income I didn't last 20. so when I was Romancing say and I mentioned this because obviously pictures of my wife shortly um the home that was expressing me was a tassy bedsit but that's the best I could do so I would tidy up and make it look it look as if I was going to be affluent today I'm my circumstances of difference so you can see some of the circumstances which say different sorts of things two of those images so ah we are our homes but okay when I was 20 I wanted to express who I might be who I was what about what we were yesterday do we have to manage how we are how we express ourselves at home so if the home expresses me what's the me that's being expressed the media is today well the meal that rented the home we bought the home the me that moved in 10 years ago and if we have to balance things like who we were with Who We Are God what a lot of Hassle and one of the reasons why people like Alexa is because when you're at home ah I can't be bothered the Lambs and make 10 key to you know whatever 10 Kroner and I get the same product I don't really care but at home there's so many problems so many tasks and all of us live in homes where you'll find these sorts of things I want to suggest that these sorts of things that I've written a paper on this with colleagues bowls drawers with tiddlers putting them out embarrassing cupboards they are ways of dealing with who you are in the center they're saying oh bloody hell I can't sort that one out I don't know where to put it I don't know whether to keep it if I throw it away is that saying I don't care about what I was doing five years ago because I might pick it up again in the future I was doing that hobby but I don't know so what do you do you just hide it or put it away you put it in funny sorts of places you use cutter bowls and what about us the old and the new how do you how do you celebrate that so on the top left there's a picture of me 10 years after my marriage that's a picture of me now what does my wife want to have on the wall what do you think does this is my wife when we got married and I'm very proud of this picture it's not allowed at home why because it reminds her that she's quite a lot older now to me she's still the beautiful woman I married but that's not what she sees so this picture is banned from my home I cannot celebrate the partnership I'm part of we have to manage those things that it gets worse what happens when you settle down oh my God it gets worse worse we make a home with your partner whoever your partner is and what do you do with your partner you make some new members this is my tiddler this is my little one obviously a long time ago now and aren't they lovely when they turn up and then what happens this was him when he first went to school around school Eaton first day putting on his bow tie who's lovely he was absolute perfect uh boy voted father for father on him and then three years later we were 16. he wouldn't let his mother near him if she'd touched him he would go bright red guess what he turned into a teenager right so what's the moral here what the moral is homes have made things these are my other two children the older two I could say the same but with slightly different colorations with those two both the handful especially my daughter is the eldest so the moral is here if there is a wall is we make home for ourselves but we make it home and put and create people in it whose goal is to leave home so what is a hope does the home express mean or does the home Express the children I've made who want to get away from home for me home is where I go to but my boys and my daughter want to leave home so the orientation to the thing that is homes are opposite so who is the home for how does it express who for now I think we're starting to shape our thoughts about what we might think Technologies for the home might need to let us do now key thing I'm trying to convey here is that if home our organs organs we're trying to express identity and part of the problem is how you manage an expression of density requires constant gentle sometimes cruel adjustments you keep having to change change occurs over time it happens and who is uh representing the home who is a member changes in identities remember what I said about deep learning systems and AI do that can deep learning systems do that can it learn from our domestic patterns yes it can if I was a single person in a house and I never changed but what have I just suggested that the problem with a home any home everyone's home is they do constantly change this in their nature to change and they change it willful ways think of teenagers in unpredictable ways think of changes in finance in sad bailful ways think about the nature of Aging can it deal with contradictory patterns can AI allow a teenager to do something like close the door on a parent years ago why was Facebook immensely popular amongst adolescents why did it become like a Contagion because adolescents are at that moment in life when they want to close the door on mum and dad about closing the door mom and dad is actually quite a difficult thing to do but with Facebook you can just not give mum and dad an invite so you can create a virtual Facebook a virtual space virtual daughters talk mum and dad participating in life and if he looked at what The Adolescents did on their Facebooks all they did was talk to their friends from school who were doing just the same thing they were just chit-chatting like 13 year olds do about nothing whinging about parents whinging about possible Partners whinging about teachers complaining about their terrible predicament trivial stuff but they kept mum and dad out can AI technologists do that not currently I don't think so what do we do with technology I believe we should let us be smart and I think the smartness should not be in the technology the smartest should be about letting us be us that's a complicated thing it's about our relationships it's about how we express to whom we express and what Manner that's how I think we should shape our thoughts if we're designing the smart home so I'll give you some examples of things um that we we built on us at once if we search and this is a few years old now but what we were interested there was that if home is a place with lots of different members instead of having a Facebook which is only accessible on your laptop why can't you have a social media posting place in the kitchen and we built this thing from an epigraph and in this case the family had five members and each member of the family from the tiddler up to Dad had a window that they could post it like a Instagram and you could send messages you could post a content and we did Trials of in people like I said the kinds of things they objected to was that they didn't like the glowing light they wanted to turn it off they wanted to do other things with it so because it was an appliance I don't know if you know what that means it wasn't successful it didn't have enough functionality to it but they all really like the idea of being able to post to the family and to having a presence in the kitchen especially parents who are working away they wanted to post content about where they were what they were doing they wanted an Instagram that didn't go to the digital ether there was an Instagram to their family here's another one which is equally simple and that's of course a bit but this is called a bubble board and one of the things we were trying to explore was um the tenderness that can be vote is things like um it's a coffee machine um when people leave messages sometimes they live rather beautiful leday voice messages sometimes voice messages are viewed as Treasures by the one who's received them I had some voice messages from my eldest boy all of that and he's still got a very high pitch of the voice and he's and it's just sad and I've kept it on my phone because it to me it's Little Treasure so we thought one of the things you might have in the kitchen is a visual answering board a visual monitoring messaging service and when you come home and people have phoned at the landliner was then possibility then and someone left a message there'll be a message icon here with a picture if you're in a dress book and you play the message if you like the message you could drag over here and it'd be on a notice board and you could put other messages you could send messages to this noticeable Via SMS and they'd restart that so the idea was that you would have a digital notice board in the kitchen and some of these messages might last years and years one of the things we found I'm surprising is that not everybody agrees is to watch a girl on there so my boys for example wanted to study all the messages from them even though I said that's so sweet because of that I hate it I don't like this I don't they'll just they're dragging pretty down the pigeonhole they're in the thing that's another one this is one that puts most everyone it's not very good image I'm sorry about that I couldn't tell you last night this is the device um about the size of that and we've built this it's called home notes and it's meant to look a little bit like a Post-It note I'm almost messaging application you would write with a stylus and you get this little dialer box and you can send it effectively like an email message and the email unless you would go any anywhere an email message could be a bitmap could be the image or you can convert it into text you get a little bit of AI to do character recognition or you could take a picture and you could use crayons and we propose that this might be a kitchen messaging platform for families to message each other and that it might reflect some of the modes of behavior in the kitchen you don't even want keyboards you even Rush a stylus might be easy at first uh people will be to try this with didn't like it especially the Men At Work but well after two or three weeks what we found was that kids discovered that there's a camera here and they could take a picture of themselves and then they could use this crayon to do colorations around themselves and make them so it's like little angels or so forth and they would send these pictures of themselves with no content except for picture themselves made playful to their dads and those pictures would arrive in Dad's Outlet in Trey at work and suddenly their little one is there on the screen and as soon as that happened all the dads said we want this application I want to pay for this and we work with Hallmark in the USA who said that this was going to be their future we did and I hate to mention this Define though a difference between the daz reactions and mum's reactions mothers in the homes we studied in the Midwest and in England didn't like the children having too much control of the messaging platform because they thought it was undermining some of the ways of managing the kitchen there were other issues afoot to terms of how you might message in the home that we didn't have time to explore for reasons nothing to do with the success of the technology but to do with mobile networks and who was going to pay for the traffic this this product didn't get developed although there's one I spent most of my career on so AI deep learning yes absolutely yes the question is where you put it not as our Butler not as a generalized AI I can't think why are people even think that's a useful thing you don't want an area to replicate this no what I want a robot to be a similar of me in my home my home is about me I quite like to have a robot but I don't want the robot to be me my problem is being me at home it's bad enough doing that let alone have to deal with a robot we have to be us right I started with the problem with glancing and I'm going to finish now the little video and application we built and one of the things we're trying to explore um and to suggest to our sponsors was when you message one of the charms of messaging is the different ways in which a message is experienced so one many years ago when I first met here in fact I did a presentation to Vodafone the Vodafone board trying to explain to them why SMS was very successful and popular and they were convinced it was because it was cheaper than voice messaging or voice calling and we said to them it's because SMS arrives in the pocket and it persists it's like a gift and if it's like a gift that opens up a possibilities we might think differently about all sorts of messaging formats it's not just to send and receive it might do all sorts of things it might also be different ways of selling and receiving and one of the things many years later when I was working with Outlook we try to persuade them one of the things you do is really get someone's attention to this gentleman several times and occasional radio here a glance now what I'm doing glancing I'm seeing if they're listening in this instance but wouldn't you communicate and sometimes you go out to see if someone's like to respond I said why can't we make a glance phone so we built a glance burn and all the Gladstone is it's a phone with a little application let me show you this thank you so much [Music] foreign [Music] [Music] offices or like a library there's no sound at all um so I want to be available but I don't necessarily want my phone to ring so if I had a glance phone functionality with a glance phone you put it on your desk and others can glance at you and it doesn't ring it just it doesn't show in the video but we set up so you'll just whisper your name so say Richard I'm dancing and it would say he was glancing and they would just take a single bitmap in one image and they could see where they were available or not they instance the designer was wanting to be playful but the point was you could express anything in the glance you could react in the way that swedes don't you could say something in a glance you could adapt your next ACT and adaptability is the problem for small Home Technology but it's also the essential motivation and problem that people inside homes have when you make a home you constantly have to adapt it to it and yourself and as if adapts and as you adapt to it so you change too it's a dance of constantly changing phenomena and that's the problem for AI but it's also I think the opportunity for AI when it shapes it thought to think properly about what you might want to do in the smart home of the future and that's me finished
2022-12-29 10:39
