join us this new year for new conversations at the commonwealth club hello and good evening my name is dj patel and i'm pleased to be today's moderator for today's session at the commonwealth club program focused on jake ward's new book the loop how technology is creating a world without choices and how to fight back this program somewhat ironically unfortunately due to covid and omicron is virtual but at least we're able to do it through all this technology and that's going to be one of the things we're going to get into but here at the commonwealth club where i serve on the board of governors they're actually we're starting luckily to be able to get some in-person programming and it is fantastic our in-person events are picking up in the months to come and i encourage you to learn more by going to the club's website at www.commonwealthclub.org and following them on twitter at cw club i'm pleased to welcome jake back to the commonwealth club of california to discuss his new book the focus of the book on how technology artificial intelligence are working to limit the choices we make as humans is a subject that's really critically important to me personally but to begin with let's just say some quick housekeeping items um if you have a question to ask jake or me but really should just ask jake put it in the zoom chat of course you can always follow along with jake on on social media um on his twitter handle uh by jacob ward did i get that right jake you got it yep awesome or you can follow me at dpotel um but today we're going to be here on the zoom on the zoom channel so maybe to start jake because there is so much to get into and first i i just want to say congratulations on on getting this book and for those of you it's it's available on all the usual places i strongly encourage you to get it especially given that the the conversations that are timely at hand but maybe to start jake i remember how you and i first met which was actually at a session that i'd been trying to wrangle people to talk about ethics and i remember the insightful just great questions that you were asking and now i read the book and there's so many dimensions that i hadn't even thought about that you've been able to bring together and so congratulations on doing so much and maybe just to start is you're a technology correspondent you cover everything why this subject of all the things that are the you know of this of the matter of today well i really appreciate it first of all dj thank you so much for doing this when they told me that you and i were going to be in conversation together i was like oh no he's so smart about this stuff and i remember being such a [ __ ] when i first talked to him about it oh god he's going to blow me out of the water so i'm really flattered that you are uh here and taking such personal interest in this um and i remember very well uh you and i first meeting and talking about this stuff and and um and that was at a time when i was having a couple of parallel experiences and they really set off the book so one was um i had just done a documentary series for pbs uh called hacking your mind and through that we went around the world and met all of these scientists who were studying human behavior studying the patterns in the unconscious decisions we make and one of the big findings of the last 50 years and this is best popularized through the work of daniel kahneman who wrote thinking fast and slow which i know you know about you know is this idea that we have these two brains these two cognitive categories essentially one is a a fast thinking brain an instinctive snake stranger and fire detection system it grabs calories off trees without us thinking about it and gets us out of a burning building you know without having to coordinate our movements right it is an incredibly useful thing that's kept us alive for millions of years they think it's about 30 million years old then there's your slow thinking brain your slow thinking brain is a very new development in the history of our species it's probably only something like seventy thousand maybe a hundred thousand years old and it is what got us thinking what else is there beyond calories and snakes and strangers in this fire and got us on our feet and exploring the world and out of that has come all kinds of stuff art and law and politics right all of these invented human systems that are part of our higher society so all of that is in this pbs series and i got to then go meet all of these people talking about this and one of the big findings is that the vast majority of our decisions are being made by our instinctive fast thinking brain even the decisions that we think we are using our higher cognitive functions to to accomplish at the same time i was meeting folks like you who were turning me on to this world of data science and pattern recognition algorithms and while i wish to god and this will be one of our themes tonight that there were lots and lots and lots of people like you who had been you know trained in this stuff and then deployed it for good right as the chief us data scientist unfortunately i was meeting lots and lots and lots of people who did not have that goal they were instead about making money using pattern recognition systems and i had a very transformative evening in which i sat in a uh basically i i went to a dinner party of a bunch of entrepreneurs they were young app makers and it was pizza you know i think it was indian food and beer and a very chill kind of atmosphere and most of these folks were trying to deploy this stuff in the service of some really nice things there was some money saving apps there was some exercise apps you know stuff sort of for that but one of the things that they had come together over was a lot of them used to be real scientists in some form or another and they were really interested in learning the latest behavioral science and trying to bring it into their work so we were sitting there and we had this presentation that night and i you know i went through all the process of saying listen i'm a journalist anything you say tonight is going to maybe end up in a book you know careful nonetheless i got to witness this incredible thing so these two addiction experts these phds come up in front of us and describe their findings around addiction and the habituation of uh compulsive action and they for instance described a study that they were really interested by in which if you took people who had been addicted to cocaine and used to do it in nightclubs and you then and they had since gotten sober and you bring them back to a nightclub you thump the music in their ears and you flash the lights out on that of the nightclub and then you show them a mirror with baking soda on it and you say this is baking soda and you make sure they understand that and then you ask them on a scale of 1 to 10 how much would you like to do this baking soda i don't remember exactly the methodology but something like that people absolutely want to do a line of baking soda anyway and the reason they were telling us that story is because they said once the habit is built into the brain it's just there and the human brain it forms these habits so thoroughly and it's so you know and and can be so compulsive and the moral of the story was it's fantastic news for you in your business and that's why we are here tonight offering you and and anyone you know our services as addiction experts and and at one point they were asked well is there any kind of company you wouldn't work for and they said and i quote i'm quoting here we don't want to be the thought police of the internet they were absolutely agnostic about what they were going to do with this and they said that a lot of their colleagues from the program that they had studied in were now working for the big casino companies so i'm having this experience of meeting you know of learning about the patterns of human behavior learning about the raw systems that we have developed these you know very powerful pattern recognition systems and then i was finding this weird in-between world in which very qualified behavioral scientists were looking at the patterns of human behavior and the patterns in our circuitry and trying to make businesses out of them and i realized oh geez i think i gotta write a book or do something with this and so that's why i got into it it's i mean it's amazing the the book i mean it starts with this you know really it's just a mind trip because you start by talking about time scales and and what time is really like and if we think about it how short of an existence we actually have as you pointed out with our you know our fighter flights versus our higher order functions and and how much technology is changing uh and this this first loop i think is is really as you're describing it this this the systems i'm wondering if you could go into a little bit of that because you go this is almost like i want to describe it almost as like oliver sacks meets wired ooh i like that right it is like if you could write that and get that published in some place there i can then put it on the cover of the book that would be really excellent thank you so much it's like it is this is like unvarnished view of like wait am i in control and maybe could you talk a little bit about like what your favorite sort of unnerving finding about who we are as humans to could you go into that yeah sure so so uh the the the concept of the loop for me is like you say actually three loops in the book and you know everyone wants to beat up their own work after they've read it and so i'm not sure if i did it again i'm not sure i would i think i'd do a better job of articulating these three loops but here's how here's how i i came down to it was there is basically a loop in the middle which is the loop of our unconscious decision making again this system one fast thinking brain versus system two slow thinking brain kind of cognition right that is the central loop then there are there's a there's a modern construction and this is the second loop which is this sort of manipulative set of business models and mechanisms that we have deployed to take advantage of that first loop and these are things like cigarettes you know it is stuff that plays on our uh you know at gambling is another one that plays on out on that circuitry to make money now the third loop for me is what's going to be made possible by artificial intelligence and when i'm talking about artificial intelligence here i'm not you know i'm talking just about pattern recognition systems and the ways in which we're gonna start essentially behaving in response to manipulation and analysis by businesses already we can feel it right i'm i can't find my way from place to place anymore without google maps i'm totally beholden to google maps and that's starting to change the way i drive the way i navigate the way i make plans with people and as that movement pattern is analyzed by businesses this third loop is taking form in which pretty soon i'm not going to know how to do anything except follow google maps around um you know i was talking to a friend of mine the other day who's trying to become a pilot and he was saying that they still insist at the faa that you learn the manual system on the flight computer of how to calculate where you're going and all the young pilots like why would i do that i just want to follow the blue line i don't need to know how to do that right we are losing a set of uh choices and and abilities that i think we used to have now well i think one of the examples is if you google um you know gps and cliff it's shocking how many people ignore their their higher level thinking functions because the car said turn right and you drove off a cliff that's right so here's here's an example of one that i think is the bigger picture that we're looking at here which is so you'll remember there was a incident in 2017 in which um uh a flight out of chicago was overbooked as so many are and a doctor on board refused to give up his seat and was beaten up by chicago aviation police and it made the news and the united ceo had to apologize and these days they are paying a lot more money to people when their flight is overbooked i'll tell you that right now at the time somebody said to me they asked this question that i then went on to investigate which is why why did they choose the people they did and how did that decision get made in the process and it turned out to be this parable for what i was talking about which is it turned out that the computer basically they tried to get everybody to volunteer to get off the flight and people would not because the last flight out of chicago nobody wanted to give up their seats that was that so they offered a certain amount of money nobody said yes second round of bidding nobody said yes finally they said okay then we're going to choose names at random and those people will have to get off they choose these four names and three of those people dutifully get off the flight this guy dr david dow he's a pulmonologist he says i can't get off the flight i've got rounds in the morning i've got patients to see he was right in the end you can't you're actually not supposed to take off a doctor who's on call the next day but something about and there's a whole parallel i talked to i talked about all of these in the book i talked about all these different experts who could if they were sitting on the plane could have said careful everybody this is the moment where you will all abandon your critical faculties because they will have told you a system has chosen people at random and right there anthropomorphism kicks in the technical term for attributing more sophistication to a system than it actually possesses simply because you don't understand how it works and and everybody in the chain of command down to the aviation police are just told this guy's name was chosen by the computer get him off right and i went and looked at like how do they choo choose the people oh it's because of do they have status on the flight and rallies and do they buy the ticket and well it was all stuff but nobody was talking about that all they said was computer says get him off get him out of here and everybody abandoned their critical faculties until the aviation police you know they beat this guy up he had to be hospitalized you know and for me i'm just seeing over and over again the ways in which our brains are uniquely vulnerable to being given a verdict by a system we don't understand and how desperate we are as the gate agent that night was not to have to make the hard decision does not want to be the person to stand there and say i'm sorry sir you are getting off or i'm sorry you are getting off instead says computer's going to choose for us and we're going to see that in hiring we're going to see that in loan making we're going to see that who gets bail and i don't think in the same way that you and i can agree we don't know how to find our way around anymore right and people are driving right off a cliff using their gps i think when the computer says hire these people you know these people are kiting you know are committing fraud whatever verdicts we ask ai to render for us it doesn't matter if it's right or not we're gonna believe it because that's what our brains do and we i think nobody is has has taken that adequately into account and regulators certainly are not thinking about it yet and i think we have to rethink how we consider our vulnerabilities to these systems before we start deploying them on generations of people as we're about to it's interesting because that example in the book you also come to the defense of the of of the aviation flight police i think that's what they're called so that's right they in fact they so one of the officers sued the um uh the chicago airport authority i can't remember what it's called but the aviation the civil aviation authority whatever it is in chicago and he at the time was made fun of by like local newscasts you know another spurious lawsuit you know of course he's trying to you know hand off responsibility for this but in his lawsuit when you read it it's so interesting what he says is we were not adequately trained to know what to do when we are told to get a non-compliant passenger off a flight who's been chosen at random and so you know i don't i don't think i don't think he was reading you know uh the cognitive people that i you know was reading i don't think we came at it for the same reason but his instinct was the same as mine which is wait a minute i didn't have the faculties or the leverage or the anything he just understood that like the system had everybody in its grip and and terrible decisions were made as a result uh you know and for me i i do i have sympathy for him not least because i think we're going to see over and over again and all these agencies right let's think about um you know i was just talking to somebody the other day at nbc news he was talking about the difficulty of hiring um uh this was somebody worked in an industry in which you know they would love to be able to hire people that the ai doesn't think is qualified and they can't even they don't even have enough people on staff in the hr department to go in nobody's got time to go in and say wait you know this person actually is really cool in all these other ways if they're if the if the check boxes are not checked then you know by the the pattern recognition system you know then that person's out of the running and so i am i'm i'm deeply sympathetic to anybody these days who says wait a minute why are we doing it this way and shouldn't we think about a different way to do it because i think the profit motive and the incentive structure is going to make it harder and harder for us to do it if we don't stop you know if we don't slow this process down now and we see i think in policing too is is if an officer questions decision they're fired also because of some form of insubordination and so it's almost like you're damned if you do damned if you don't and you can you can get the human gets fired but there's no accountability is one of the things i've taken away from you know the loop structure that the way you've you've articulated it well i appreciate that and i think i think you're absolutely right i just think you know there's very little incentive to push back against this stuff now i i want to say here you know i mean i want to be i i am trying to push back against this whole thing because i think it's really scary i also think that there are in fact some really extraordinarily positive uses of some of this technology and and and some of it can i think be great so there's a very fascinating guy named michael knapp who runs a place called green river ai um he's way out in the woods in vermont uh working by himself and and he has a team there but you know he's just he's off the beaten path and he he only works with non-profits and um some he'll do some hospital and health work but he has these very high standards for himself about who he will deploy ai on and with and i was running my thesis by him you know that we're going to lose human agency over time if we rely on these systems he said man i wish i had that problem you know he's like my people aren't moving fast enough on this stuff if i if only they would grab on to ai the way that for-profit companies had he said for instance if you gave me every birth certificate in the country and i could just feed it through a machine learning system um you know a gan would kick back at me every apartment that needs to be repainted to avoid lead poisoning in this company in this country i could save millions of years of life if you if you gave me that opportunity but i'm not allowed to do that or he said in social services agencies they a person comes in and applies for help and then we find the services for that person he said it should be the opposite ai should look at the available resources and feed them to individual cases go out and find individual cases but we're not allowed to do it that way you're not allowed to to approach somebody that way so the problem i think is that nobody's making money taking lead out of apartments nobody's making money matching social services people with the unhoused uh we are making money on gambling right we're making money on addiction and i think ai has an incredible capacity to in theory amplify our slow thinking brain our sophisticated moral brain right this is the the thesis in fact a new book by daniel kahneman i was so disappointed by that book because i wanted to say to him yeah but nobody's making money doing that they don't want to sell to system 2 to the fast thing into the slow thinking brain they want to sell to the fast thinking brain they want to sell to your instinctive systems that can't help but you know get angry and make a rash decision on the rest of it so huh to me it's capitalism that i worry about here well it's a great one because i think you know what we've seen in the early days of data science is the potential power of of of using data for for frankly good uh you know the one that i think about is you know given this recent fire in new york city you know one of the early cases in the bloomberg administration was how they were cleverly using data to identify which buildings were likely to have lots of problems because of complaints and calls to you know the the city and they just it wasn't like they did machine learning there they just took a list and they said who's got the most complaints and let's look at the the places nearby and so they could prioritize the the police inspector or the the fire inspector's time to be most efficient and effective it almost seems like government is not and i also think about like medical errors in hospitals like is is it purely economics is it because like i think about all these do-gooders who want to go into these fields what's holding them back from from taking giving agency to to um forces of good for lack of a better return right right and i i do think i think it's money i mean i just think it's money it is uh it is much much harder and less profitable to do what you were describing and i and i that is not to say that people aren't going to to be able to do that kind of stuff you know like there's fantastic you know i i i meet people every so often who against all capitalist instincts have decided you know i'm going to make a business that does x you know i'm going to work my brains out to do a you know i met a kid the other day who um wanted to subvert the cash bail system by creating a whole alternative sort of you know i mean and and it was a it's a you know it could be a huge money maker but his whole purpose was to try and do away with cash bail because it is a deeply unethical and uh unequal system and falls most heavily on the poor and all of this stuff right but those people are very few and far between and i think that that you know i also you know i i one of the difficulties and i know that you have encountered i assume you have encountered this too and i and i wonder i'd love to hear your thoughts about as well is you know there is a tremendous culture in silicon valley and in technology in general of convincing the people that work in that industry that the work they are doing is for the better is for the you know a greater good it doesn't really matter what they're doing making chairs making vape pens the line from the the you know hr people who tend to be referred to now as people and culture kind of people right is is you're doing good works in this world and so i think a lot of people actually spend a huge amount of their time being convinced of that also you know some of the biggest companies they refer to uh you know the people in that line of work as as scientists and as you know the the the core you know they don't call to the headquarters anymore they call the campus right there's a there's an academic veneer to how some of this is is done that i think draws people in it makes it very difficult for me as a journalist to have a frank conversation with them and that's putting aside the fact that this is the most secretive companies in the world that you know when i talk to my colleagues at nbc who cover the military you know or cover the pentagon right they they people will talk to them endlessly not necessarily on the record but they will talk to them endlessly it is incredibly difficult for someone as a journalist to to speak to people inside one of these companies and by the way anyone listening who would like to speak with me i'm very good at keeping a secret i would love to speak with you but you know what i'm saying like like so i think there are many many incentive structures built here that are going to make that that make it difficult for people to make good choices and i would also like to point out smart very you know much smarter people than me people like you you know there's a guy a um a brilliant uh uh woman named meredith whitaker who was inside google she's now a consultant to the ftc she's working with lena kahn and she made you know she was one of the very first people to say it is not okay i mean thinking here about what you're saying about like trying to push back trying to raise your hand say this is this is a you know not okay that it is not okay inside inside certain companies for someone in the position of being a data scientist to even ask what their work is going to be used for you know i was talking to somebody the other day uh at one of the big five tech companies four six whatever there are now uh who said um that if you ask too many times you immediately get fired that it's a it's a if you ask you know what is this going to be used for too many times they let you go you know so i think it's a really complicated landscape in which to create any kind of groundswell of resistance to this kind of stuff um and i i worry about that right yeah so so for those that are just kind of joining us we're in conversation with uh jacob ward and his book uh new book uh the loop and we're gonna start taking questions in a bit so um i can encourage you in the zoom chat to ask them here and you should definitely be following jake on twitter and other social media uh at by jacob ward by jacob ward uh you know let the the one that you're talking about in these corporations and transparency i think is a great jumping off point because uh jumping into point uh around national security and you know some of these companies are talking about you know the need and the desire to work on on national securities given the complex landscapes questions about western values and what's being put into these systems china is aggressively going after surveillance technologies we've seen ai being used to break into systems from israeli companies it's a very complex landscape and if we the our classic argument is if we don't do it somebody else will that's right how how how do you square all these things as you look across across the complexity of the world and honestly preserving our lifestyles i know i know it is such a vast and important subject and maybe that's the next book right but but i think that the so first of all that thing of if i don't do it someone else will is one of the great traps of this world whether you are someone who you know i've i've had people say that to me who work in national security related products making surveillance systems uh you know i've and i've heard that from people literally who make uh you know uh highly addictive casino simulators uh designed to ensnare old ladies uh you know that way of thinking that someone's gotta do it it might as well be me i can do it a little more ethically than somebody else will is is a real complicated thing and that's common thing now national security so you're actually it's amazing that we have to start with the side oh my god i mean it's so complicated right like right before the pandemic i was in this i was just about to go to china to try to do a whole series for nbc about this because it's and and one of the conversations that we were having about it is the language that you have to use in order to speak to chinese officials about this stuff and what i wanted to do essentially was go there and say okay you have the exact same technological capabilities arguably even more sophisticated technological capabilities and an entirely opposite world view political worldview and and what does what would our lives look like if we were living in an authoritarian environment in which um stability and control was the priority as it is in china and it's so fascinating to to talk about that then with people here in the libertarian west uh you know west coast i was at a dinner where someone was giving a presentation on the chinese social credit system where you have to register your you know look at by jacob board you got to turn that to those credentials into the government so they can monitor what you do online and if you don't behave properly if you post something about 10 square or worse if one of your friends posts something about tinny square your credit score goes down and you can't get a loan and eventually you can't ride trains and eventually you're basically a prison a prisoner in your house you know they were describing that system and the effect it's having on social cohesion and this that and the other and after the presentation the room was divided half of people said uh wow that's a nightmare and the other half people said we should totally have that here in the united states you know there is an there is a way in which the most textbook form of communism you know and where and authoritarianism and where it goes and the most wild uh form of of libertarian tech fused capitalism meet in the distance in this weird way that i haven't really figured out for myself yet but i do know that over and over again i i bump into people you know who who essentially say we have always assumed in the united states that our model is the model and and you know there's that phrase theodore parker originated it but martin luther king said it right the the more like the universe is alone but it bends toward justice obama's say it a lot you spend some time with behavioral scientists political scientists people actually study this stuff and they'll say no we don't know that to be true the moral arc of of the universe is so long and we've been on it only very briefly we don't know which way it bends we don't even know if it's an arc you know they are not optimistic about this stuff it is all an experiment and so i do not think it is in any way a foregone conclusion that our way of life and how we deploy technology and the sorts of conversations that you and i are having tonight are a natural thing you know that's going to take place all around the world you know it could very easily go the other way so you know i don't know all i know is that yeah that is such a minefield i'm it's so complicated to think about and i i i wish i was asking you the questions about it because i don't know the answers to that one well that's why i get to ask the questions because we're all trying to figure it out right but at the same time you know some of the stuff that i struggle with honestly here is you know i i think like let's just take a look at the google maven project the one you referred to before which is also this one of surveillance and on one side in the surveillance systems you have a human who's sitting in the screen who's trying to track a person and if they get distracted because somebody asked somebody and now they're following the wrong person that may result in somebody deploying you know i don't know in a drone or something that that takes out somebody who's innocent and so could the computer is a computer there also to fire us out of jobs that we find so boring that we are bad at them and results in an error i'm thinking about the nurse who mistypes the diabetes the insulin number or fat fingers it just because she's so tired because she's working taking care of 50 coveted patients and and now this person is getting the wrong thing yeah yeah i you know for me i think all things being equal you know i was talking to an organizational psychologist who was basically saying yeah if you you know i was asking are you in favor of using automated systems to find candidates for jobs and screen them and the rest of it and she said you know if it replaces the racist instincts of some you know uh long-standing hiring manager who's been bringing his racism to it for years yes absolutely unfortunately i she was saying i don't think that you know the it is it is as simple as swapping the judgment of one out for the judgment of the other because along with this is this blindness to all of these different things and this is me talking now i mean you know the unique human vulnerability to believing systems we do not understand right there and uh there was a recent study of all of these um high-level ctos uh at these some of the biggest companies in the world and they were asked um you know how often are you used relying on ai to make important decisions and they were like all you know all the tests like 75 or something crazy and then they said you know how many of you can actually explain how the systems work almost nobody has any idea how these systems are arriving at the decisions they are and to what extent are you worried about that not even slightly you know like less than a quarter of these guys and it's mostly guys of course uh you know care so it's the the problem i think is that in the case of the exhausted nurse who's going to be replaced by the system excellent that would be great if the pattern of history were that we then gave more time to that nurse to do her job better to do his job better that's not how we tend to use this stuff we cut that nursing staff down to two right and then we use those automated systems so we i think we have to build some values and emotions into how we make these decisions as opposed to just doing them for efficiency and better accuracy because you know it is it is going to be necessary and i will say sort of take it more time here but just you know i will say there have been some instances in which we have done that as a country so for instance you know uh backup cameras um there's a physician who accidentally uh backed over his child's most terrible story and he gave all of this congressional testimony about it and eventually after a long battle it is now the case that if you buy a new car in the united states you have to the car has to have a backup camera and you're paying about 1200 more per vehicle for this and that is because about 60 kids a year were being run over typically by their parents the efficiency model and the you know the math would suggest well that's not very many kids but we as a country can agree that that is totally unacceptable and that we can agree that we can out we should solve that problem and send the senators could for whatever reason they got together it was a bipartisan effort they made it happen and today that doesn't you know those numbers have dropped to almost nothing so we can make decisions on the basis of things other than efficiency and we're really going to have to because the temptation to just do it because it saves you a few nurses is going to be the thing that drives the value proposition for so long and we need to stop thinking about it that way i think it's it's so it's so good it brings up two sort of memories of mine one is the uh president obama's precision medicine initiative and the argument that i think really got the president president obama over the line on it and realizing why he wanted a data scientist to really run this was the only way we're going to go after long these long term genetic issues that sort of cropped up in the in the that we call n of one these rare diseases is if we make this a data problem and we're able to kind of instead of just having hypotheses out there and going and building these very complex tests kind of going wait isn't that interesting we're seeing this correlation here of cancers uh um a population just having cancer we should go study that you know this other one that comes to mind is early in my career uh with national security was i was talking to this general we were building these detectors after 9 11 to to you know basically tell if somebody was bringing in something that had a like a dirty bomb and you know we had all this pretty sophisticated stuff and and graphs and charts and everything and the general pulled me aside said son you ever work with a marine i was like no sir he said the light is either red yeah right or a green yeah he just turned and walked away now my apologies to my my marine friends but but it raises this other point that you're talking about and so i want to use this as a jumping off point because we're getting a number of really great questions here uh and taking those two examples back to the issue of racism and the racial reckoning we have because we got a couple dimensions there that i'd love your take on one side we have some communities that are saying yeah put license plate readers all up my over my neighborhood because we want to do i have to hide we might have to hide but there's there's clear bias and bending that that moral arc there we have people using facial recognition in fair protests we also have the ai systems that are who designs these but also has commented in one of these com somebody pointed out here is thank you so much for speaking on this issue as a lawyer i noticed that most judges are irritated when the defense counsel even suggests that our systematic way of doing these things should not be challenged and then the judiciary is starting to use bail calculators as you talked about that have been proven to be racist so i i bring up all these things because it's such a wide area how do you how do you get your head around this yeah yeah so there are so many dimensions to it as you as you say and you know and thank you so much for that question whoever brought that in so the law is such an interesting one because it that is i mean there is there is no better example of that slow sorry as i said slow thinking brain right the slow thinking brain in which we are policed by people we've never met and we try to create these laws we all agree on and you know i mean our instinctive snake detection systems were not capable of that and so it is a credit to our species that we're able to do that my dad always makes the point you know that when we when we drive on the highway that we managed to keep our lane and not kill each other it's just a miracle and it's absolutely correct so the fact that in the law we're thinking here about this is so interesting so there's a a superior court judge uh named tino cuellar um who is now running an institute and as a very smart guy he was at stanford when i was doing a fellowship there and he said to me you know you could make the law so much more efficient it would be possible to do it you know he said for instance entering a guilty plea is such a pain you've got to fill out all these forms and you've got to do this you've got to do that you've got to you know think it through and we could make it a swipe left thing you could make it you know such an easy thing he's like but we have a principle that we call weak perfection and is the idea that you build a system intentionally awkward so that people have to think about it because with a guilty plea or not guilty plea that's going to change your life you don't get to take it back you know it's one of the biggest decisions you'll ever make and so we cannot make it as simple as ordering from grubhub we need to keep it hard we need to do that so that people bring them their best selves to it so in this particular question of you know you know judges don't like having something like systemic inequality brought up i know i mean i have this whole section of the book that for me was the really mind-blowing one for me which was getting into the world of online casino simulators i've mentioned them a few times here because they're the big bugaboo for me these absolutely cynical predatory companies that that use the circuitry of addiction and and geo fencing and targeting and you know distressed lines of credit and all the other stuff you can find through data to pinpoint people they think are going to get addicted and what's so interesting is that for years i was talking to this one lawyer who for years has been trying to sue on behalf of these people who lose their life savings to these companies like 499 at a time and he was left out of court for years because the judges would say i'm not calculating this is you know what are you talking about loss you know losses these people are suckers you know and i've had many people in my reporting on this for nbc also say you know these people are suckers and they get whatever they deserve because they're playing a fool's game but recently they've started winning these lo this law firm has begun winning they just did a 150 million dollar settlement with one of these companies because they were able to show in the data that these companies know exactly how addicted these people are and are finding them on that basis and so i think if you can get better if we can as a society get better at deploying the same pattern recognition systems that we're using to ensnare people to instead prove the existence of systemic racism the way that redlining and systemic discrimination has made its way into all these other things right i think we could get to a place where you could actually make a case in court you know i know people are very cynical about the role of of trial attorneys and litigation and this stuff but that's why we don't smoke cigarettes anymore you know and i think that that suing the bejesus out of people is part of how we're going to get through this and and that is that's for me i i hold i'm clinging to that one because i think that's going to be a big part of this um so please make sure to get your questions in here in the chat uh this raises another good one that was on my list of things to talk about and i'm glad a number of people have brought this up also which is misinformation where we are as a country in people going down into conspiracies and getting sucked into things whether it's vaccinations or january 6th in the insurrection and people's head getting turned around what's the role of technology in this and you know one side of it is people are like you know we have this culture in america of you know if you're not tough enough addiction is your fault and then we have this other side of you know oh it's not it's not them right yeah yeah yeah that's right you know so so i i've been so lucky to be in touch with people who you know to work alongside and have for this book interview people who really specialize in this in in in thinking not just about misinformation which is itself such a problem but also the grift which is how i thought to be taught you know is how i've taught to be thought to think about it involved in misinformation the small time grifters who make money off of misinformation so around january 6th for instance i spent that day monitoring all of the online streaming folks typically on youtube who were streaming from the capitol and they're pulling in the feeds the live feeds of on people's phones which of course have gotten so many of those people arrested um uh and at the top of the youtube screen if you have a certain number of subscribers you're allowed to institute what's called a super chat which allows you to charge money for pinning somebody's comment to the top of the window for a few minutes and you can set whatever price you want youtube of course takes a percentage of that and it's 20 bucks 50 bucks whatever you know 100 bucks and i just watching people bing bing bing you know they're making you know a few thousand bucks a minute now is that a few thousand bucks a minute live streaming live streams of the seat of democracy the an insurrection you can't make this up right you cannot make it up exactly you would not believe it if you saw it so it is it is it is uh yeah it's idiocracy but not funny right it is it is and and and so that profit incentive is a huge driver of this when i go cover um you know when i was covering a lot of the stop the steel protest rallies at nbc i would go see these people there who are streaming live and it's so interesting because they look sort of to a space alien they might look like their job's the same as mine they've got lights their hair is done they're doing their makeup right and then they go live but these people are leading the chant and when you look at their their instagram feed or you look at their super chat on youtube you can see they're making money in that moment now i am also being paid to cover this but i don't get paid more per comment you know what i'm saying like you have a you have an industry behind you ideally called journalistic yeah that's right that's right and i get fired if i make it up if i lie i get fired so so there's some gargoyles right but anyway that that for me learning the grift was really powerful and then there's a very brilliant woman named nandi gemini who who runs something called check my ads uh and uh she was the co-founder of sleeping giants you're probably familiar with and she has been doing all of this research about the ways in which online advertising funds um all of these uh very scary publishers of all kinds of scary stuff and um she turned me on to the research that that really sort of set her on her path which showed that there are all kinds of blacklist services that will spike certain published articles against basically make it such that advertisers won't be you know who don't want to be publishing or advertising next to sensitive topics won't be published next to certain news articles and she discovered that in fact with the research actually discovered that it is people covering really important stuff that are being blacklisted off of these lists such that some pulitzer prize-winning brilliant people at the new york times for instance were not no online advertising was appearing next to their work so it's actually costing the new york times money to run that kind of really important journalism so for me right when i when i think about misinformation again i'm thinking okay there is a system here both of pattern dumb pattern recognition that nobody is equipped to question and incentive structures that is fueling this stuff it i you know i also blame as much as the next person our tendency to just try and be tribal and crass and get attention the attention economy is a really important part of all that stuff but there's some specific machinery in there that i think we should be starting to think about how we're going to take a hammer to it i mean yeah one of the ones i will highlight because this is how much i enjoyed the book here is i think it's in chapter two you talk about this experiments of what happened when kids are just basically told they're on the you know green team versus the orange team and what affiliation does as a powerful i mean psychological motivator you know and it gets me to one of these the questions that's in in here is and almost like the way i almost want to describe it is is on once you could you talk about this addiction of people have too much time on their hands because technology is freeing them up they can take drugs get into the stupor and detach from the world same thing happens with gambling you see a version of that you see a version of this with uh um as with people who don't have alternatives to spend their time on work or other things getting into these forums where they get they get radicalized not just here in the united states we see it around the world and it's almost this version of like we're using technology to free ourselves up from time but then you know when time comes together you know your free time plus despair the note i wrote is free time plus despair equals opportunity to take advantage of people and technology accelerates it yeah it is i'd love for your reaction to that yeah so i i yeah i i i absolutely i'm various about what you say i i'm not sure that i blame free time as much as i blame despair in in the equation that you have there and i also think that social isolation is a huge part of that as well so one of the common threads you know there in the book there's a 14 year old kid who lost his mom and was deeply isolated in uh florida who wound up going down this rabbit hole uh of of race quote unquote race realism and all of this stuff and wound up adhering to all kinds of white supremacists he's a muslim right and he turned out to be muslim that's right he was a his parents were bosnian muslims who escaped genocide and he nonetheless wound up down this rabbit hole and became you know somebody who believes in white supremacy so that kid couldn't have been sadder or lonelier than than he was he was deeply looking for connection and was not able to find it another character who you know fell prey to online casino uh simulators also a deeply lonely and and sad person and here's the thing is i you know what i'm starting to understand is that there are marketing mechanisms out there that find people who exhibit those conditions you know i don't know about you but a lot of my pandemic as soon as i turned in the book anyway i went hard at tic-tock for a while and would doom scroll my way through hours of it until and here's what happens when you get to a certain point in tik-tok until a video comes up that says you've been scrolling really fast you should slow down there's like a little little warning that says you've been going too fast slow down and then eventually it'll say you've been looking at videos for quite a while you want to take a break meanwhile every ad i get is for adhd medication right and i'm sure anyone out there listening to this who's been on tick tock recently has gotten these ads as well huge amounts of adhd medication now maybe everybody's getting that maybe that's just a blanket kind of advertising campaign i don't think so i think that inside that company there are there is a pattern recognition system that says this guy is exhibiting the classic signs of x y and z serve him an adhd ad you know it is not just the affinities that we have and the hobbies we exhibit and what we post about it is the way we behave that is showing our inner state and i think that we are being analyzed in that way that is the loop right that's what's starting to grab us um and as they get better at noticing that i'm adhd i'm not actually a diagnosed adhd people and there's and there's a whole problem with with advertising adhd people who have not been clinically diagnosed but putting all that aside the qualities they have spotted in me and are feeding me information as a result i have to basically the way tik tok is for me it's like doing drugs i do it for a couple of months and then i have to erase it off my phone because i cannot control myself with that app and so yeah there's an inner state being analyzed here um that i think is a is a huge part of this and you know maybe it is extra time on our hands but i don't know about you like half of americans can't put together an extra 400 an emergency right now i don't think time is our problem you know well i've i've tried to layer in a number of the questions there's so many more that have come in that are they're amazing i hope so many people will uh keep putting them in here the uh maybe to capture a couple more of these is it's almost like i might describe these as are you optimist or a pessimist no i know i'm a weird i mean it's almost like this what do we do about this what are we going to do jake i know here's what we're going to do here's what we're going to do i think we're going to first of all need to look deep inside these companies and make them civilly and maybe even criminally liable for the ways in which they have tried to manipulate our behavior i think that it's going to start costing these companies money right now human attention is treated as this kind of ephemeral thing there's it's endless you know um but you know people smarter than me have been you know saying no no it is like mining and we need to regulate it not that we do a great job of regulating mining but we need to get into to these companies i think probably through lawsuits and begin showing what they are what knowing and doing now that's that's for me the the first step but i also think there needs to be a recognition that all of that it's like star wars we're watching you know when i watch star wars these days and han solo is being told by c-3po never tell me the odds you know leave me alone nerd right he's always saying you know don't tell me you know oh you know captain solo the chances of survival are 10 56 to 1 right and he says never tell the odds listen to c-3po c-3po should be the hero of that movie because he's right should not do this you know and our whole culture is geared it has been since the 19th century on this idea of rugged individualism growth at all costs is good we're going out to the west and and you know uh pioneering our way out to a better life you know as opposed to thinking as a community about how are we going to support one another what if it all goes wrong and and and for me a big part of that is going to have to be making it socially acceptable to say here are my mental predilections so for me i try i tell anybody who who you know wants talking about it like i am uh i no longer drink i'm a uh i don't i think it's unfair to people who suffer from alcoholism to refer to myself as a as an alcoholic i'm not sure i fall fully into that category but i absolutely cannot drink i've learned that about myself and i have also learned as a result that when people say to me hey let's meet up and go to a bar i say to them no i i would love to take a walk with you i would love to do this other thing but i cannot go to a bar with you i used to drink and i don't anymore and that's going to mess me up right being able to say tick tock has got me right being able to say i'm having trouble with this thing you know making it socially acceptable to look at the odds right to listen to c-3po i think is going to be a really important thing and then the last thing is i think we need to stop letting culture the modern culture as it's being dictated by some of the biggest companies um tell us our norms so for me right now i'm in the process at the school that i'm that my children are at of creating a pact with all the the parents in the grades that we are in to not give our children personal smartphones until they enter high school at the very earliest and i can't tell you how complicated that conversation is it's a very hard thing to have that conversation because it involves admitting to your own difficult relationship with smartphones the you got to sort of admit as apparently you don't have any idea what your kid is really doing with them and what that might be and that you may not even know your kid fundamentally at all you know it's a really hard conversation but we have managed to get through it and in fact i'm i'm on the hook right now for being the guy who's supposed to write up the new revised pledge after a huge amount of really smart input um it makes my palm sweat to realize that i am i'm on the hook for that right now but you know it's going to require communities coming together and saying nope nope i'm not going to do that because you know the statistics show that the vast majority of parents get their cues about what's an appropriate use of technology from the ads for technology from a cutesy alexa ads you know in which the kid and the dog you know trigger alexa by accident is not adorable you know they're normalizing behavior that we have not actually signed off on and i think that we should we need to start coming up with some civic structures for saying no it's too and we're too quick to say oh don&
2022-01-29