Bruce Schneier: "Click Here to Kill Everybody" | Talks at Google

Thank. You. So. This is the. Talk. This is the book. So. It has one a my first ever clickbait, title. And. I like the cover line cover because for two reasons one there's only one button that says okay when. It's clearly not okay, and. It. Looks like this thing's been throwing error messages, in the past hour and nobody, has been paying any attention to, them. Alright. So. This is a book. As a computer, and what, I'm writing about is. Security. In a world where, everything is a computer, and I, think this is the way we need to conceptualize. The. World we're building. This. Smart, this phone is not a phone it's. A computer that makes phone calls and. Similarly. Your microwave oven is a computer, that makes things hot and your refrigerator, is a computer that make keeps things cold and. The ATM machines a computer with money inside, and. In a car is now a computer, with four wheels than an engine actually, that's wrong a car, is about a hundred plus distributed, system with, four wheels in an engine alright. So, this is more than the internet it's. Again out of things but, it's more than that as well in my, book I use the term internet plus I hate, having. To invent a term but, there really isn't, a. Term. We have for. The internet the computers, the, things the. Big, systems, like power plants the data stores the processes. The, people and. It's. That holistic. System that. I think we need to look at when. We look at security. So. If everything is becoming a computer it. Means, two things that are relevant here. That. All that internet, security, becomes everything, security. And all. The lessons and problems, of, Internet's. And computers become. Problems. Everywhere so. Let me start with six sort. Of quick. Lessons, of computer. Security which would be true about everything in everywhere. Some. Them are obvious in computers, not. So obvious elsewhere, are the first most. Software, is poorly written insecure, right, we know this basic. Reason, is the market, doesn't want to pay for quality software right, good fast cheap pick any two we, have picked fast and cheap over good.

Right. With with very expensive, exceptions. Like avionics. And the Space Shuttle, most. Software, is really, lousy. It. Kind of just barely works now. For security, right. Lots of vulnerabilities, some. Of those bonus, lots. Of bugs some of those bugs of vulnerabilities, some, of those vulnerabilities are exploitable. Which. Means modern software has lots of exploitable, vulnerabilities, and. That's, not gonna change anytime, soon. The. Second lesson, is that, the internet was never designed with security in mind that, seems ridiculous, today, but. If you think back to the late 70s, and early 80s there were two things that were true one. The internet was used for nothing important, ever, and. Two. You, had to be a member, of a research institution, had access to it, and. You. Read the early designers, and they talked about how. Limiting. Physical, access. Was. A security. Measure and in. Fact you could exclude, bad, actors. Meant. That you didn't have to worry much, about security, so. Decision was made deliberately, to. Leave security, endpoints, not, put in the network. But. Fast forward today and we are still living with the results. Of that indem, them domain, name system, and routing, and in packet security, email addresses, sort of again and again the, protocols, don't have security and we. Are stuck with them. Third. Lesson the. Extensibility. Of computerized, systems, means they can be used against us extensibility. Is not something, that non computer people are used to, basically. What I mean by that is you can't constrain, the functionality, of a computer, because, it's software. When. I was a kid I had a telephone. Big, black thing attached to the wall great, device but. No matter how hard I tried I couldn't make it be anything other than a telephone, this. Is a computer that makes phone calls it can do anything you want right there's an app for that. Because. This can be programmed, because it's a computer, it can, do anything you can't constrain this functionality. Might. Mean several things for security, it's hard to test this thing because. What it does changes, how. It's configured changes, and. It. Can get additional, features you don't want that's, what malware, is. So. You can put malware on this phone or on interconnected refrigerator, in, a, way that you can't possibly ever, do, it in an, old electromechanical. Refrigerator. Because. They're not computers. Fourth. Lesson is about complexity, a lot. Of ways I can say this basically the, complexity, of computers means attack is easier than defense I could, spend an hour on that sentence. But. Complex, systems, are hard to secure. Units. The most complex machine mankind, has ever built by a lot, which. Makes this incredibly, hard to secure. Hard. To design securely, hard to test so, that everything about it it. Is easier, to attack a system than, to defend it a, fifth. Lesson is that there are new vulnerabilities. In the interconnections, as we. Connect things to each other vulnerabilities. And one thing affect other things with. Lots of examples the. Dyne botnets, and. Vulnerabilities. In interconnected. Was. Digital, video recorders, and webcams primarily, allowed. An attacker to create a botnet, that dropped, a domain name server that. In turn dropped a couple, dozen real popular websites a. 2013. Target, corporation. Attack. Through a vulnerability, in the HVAC. Contractor. Of several of their mid Pennsylvania stores. Earlier. This year is a story of a casino in Las Vegas we don't know the name of the casino they.

Had Their high roller database. Stolen. And. The hackers got in through and I'm not making this up they're internet-connected, fishtank. So. Right vole, nobilities. This. Can be hard, because. Sometimes, nobody's, at fault I blogged. A few months ago about a, vulnerability, that results, from the way Google. Treats. Email. Addresses, right. The dots don't matter for your name and the, way Netflix, treats email addresses, the dots do matter turns. Out you, can play some games with that. Who. Do we blame, I'm. Not sure we blame anybody. There's. A vulnerability in PGP, which is actually not really vulnerability, in PG speeds vulnerability, in the way email is handle PGP, which. Everyone just convinced everyone else is a fault and. These. Kind of thing is going to happen more and more. The. Last lesson is, that attacks. Always get better. Attacks. Always get easier, faster. Cheaper, right. Some of this is Moore's law computers. Get faster so. Password, guessing, gets. Faster as computer gets faster or not because we're, smarter about it but, we also get smarter. Attackers. Adapt, attackers, to figure out new things and, expertise. Flows downhill. What. Today is a top secret NSA program, tomorrow becomes a PhD, thesis the. Next day is a common hacker tool and you. Can see this again. And again, an. Example might, be, IMSI. Catchers, fake cell phone tower stingrays. Right. Which when I mean. The cause of them is that, cell. Phones don't Afeni Kate to, tower they they automatically, trust any anybody says I'm a tower so. If you put up a fake tower you can now query phones and get their addresses. That sort of know who's there, now. This was something that the. NSA used the FBI used, big, government secret for a while. Expertise. Flowed downhill, a few years ago I think was motherboard. Did. Looked around the DC found, a couple of dozen of them run by we don't know who around, us government buildings I mean. Right now you can go in alibaba.com, by. One of those things for about a thousand, dollars in China, they're used to send spam to. Phones. You. Get a software defined radio card, you can download. Free. Software and make your own, right. What started, out as something that was hard to do is now easy to do. So. Those are my sort of six lessons. That. Are gonna be true for everything, and. None, of that is new but. Up to now it's been basically a manageable, problem. But. I think that's gonna change and. The. Reasons, are. Automation. Autonomy. And physical, agency, computers. That can do, things. So. If you do computer security, you've heard of the CIA triad, confidentiality. Integrity, and, availability three. Basic properties, we, deal with insecurity by. And large most. Of what our issues are our confidentiality. Someone. Stole, and misused, our data. That's. Equifax. That's, Office of Personnel Management. That's Cambridge analytical, that's all the data thefts, ever. But. When you get, when. You have computers. That, can affect the world in a direct physical manner integrity. And availability become. Much more serious. Because. The computers can do stuff there's real risk to life and property so. Yes I am concerned than someone hacks my hospital, and steals, my private patient on medical records but, I'm much more concerned that they changed my blood type I. Don't. Want them to hack my car and. Use. The Bluetooth microphone, to listen in on conversations but. I really don't want them to disable the breaks I. Mean. Those are data integrity and data, availability, attacks, respectively. So. Suddenly the effects are much greater and. This. Is cars medical. Devices drones. Any, kind of weapon systems thermostats. Power. Plan. Smart, city anything, appliances. I. Blogged. A couple of days ago about an. Attack, where, someone, just theoretical, if you could hack enough, major appliances. Can. Turn, power on and off in synchronization, and. Affect. The load on power, plants and. Potentially cause blackouts, now. Very. Much a side effect but, once I say that you say well yeah duh of course you can do that. Very. Different sort of attack. Right. There's a fundamental, difference between my. Spreadsheet, crashes I'm gonna lose my data and my, implanted, defibrillator. Crashes and I lose my life and it, could be the, same CPU, the same operating, system the same vulnerability. The same attack software. Because. Of what the computer, can do, the. Effects are much different. So. The same time we're getting this increased functionality. There's. Some long-standing security.

Paradigms, That are failing, and. I'll give three the, first one is patching. Attaching. Is how, we get, security and it's. Now. Having, trouble, actually. So there are there's two reasons why our phones, and computers as secure as they are the, first is that there are security, engineers. At Apple. And Microsoft at, Google that are designing and as secure as they are in the first place and those. Engineers. Can, quickly. Write, and push down patches, when, vulnerabilities, are discovered, that's. A pretty good ecosystem, we do that well. The. Problem is it doesn't work for low cost embedded, systems like, DVRs, and routers. These. Are designed and built offshore, by, third parties, by ad-hoc teams that come together design, them and then split apart I mean. There aren't people, who. Can write those patches when. A vulnerabilities, discovered. But. Even worse a lot of these devices have, no way to patch them I. Mean. If your DVR, is vulnerable, to the to. The to. The hack that. Allows you to because you a botnet, the, only, way you can patch it is to throw it away and, buy a new one that's. The mechanism we have no other. Now. Actually. Throw it away and buy a new one is a reasonable. Security measure, we. Do get security the fact that the life cycle of of phones. And computers is about three to five years. That's. Not true for. Consumer. Goods, you're. Gonna replace your DVR, every, 10 years your. Refrigerator, every 25, years I. Bought. A programmable, thermostat last, year I expect. To replace it approximately. Never I. Think. Of it with think about it in terms of a car, where. You buy a car today let's. Say software is two years old you're. Gonna drive, it for 10 years sell, it someone. Else buys it drives are 10 years they, sell it someone. Else buys it puts on a boat said to the South America where. Someone else there buys it dries there another 10 to 20 years and. You go home find, a computer from. 1976. Try. To boot it try to run it try to make it secure. We. Actually have no idea, how. To secure. 40. Year old consumer, software we have the faintest clue. And. We need to figure it out, so. What does Christ will maintain a testbed of, 200. Chassis x' for. Vulnerability, testing and for patch, testing, is, that the mechanism I, mean. We're not gonna be able to treat these Goods like we treat phones and computers. You. Know we start, forcing. The computer the computer life, cycle onto all these other things, we're probably literally gonna cook the planet, so. We need some other way and we don't have it. Second. Thing that's failing is authentication, all. Right it's we've, always been only okay on authentication. But. Authentication. Is going to change right now Fenne occation tends to be me, authenticating. To some, object or, service. What. We're going to see an explosion in. Is thing, to thing authentication. We're. Objects, need to authenticate to objects. And, it's. Gonna be a lot of it imagine a driverless. Car or even some kind of computer assisted. Driving. Car it'll, want to authenticate to, thousands. Of other cars road. Signs. Emergency. Vehicles. And signals. Lots. Of things and we, don't know how to do that at scale, but. Or you might have a, hundred.

IOT. Objects, in your orbit to. Authenticate to each other that's, ten thousand authentications, wait. A thousand objects a million authentications, i mean right now, this. Is tends, to be our IOT, hub if. You have an IOT anything, you likely control it via. Your phone I'm not. Sure that's scales to. That many things, and. Well. We can do thinking, of Education it's very much. Deliberate. So, right now when I get into my car this. Phone authenticates to the car automatically, right, that works Bluetooth. Works but, it works because I was, there to set it up and I'll. Do that for. Ten things for, twenty things I'm not, doing it for a thousand, I'm not. Doing it for a million. So. We need some way to, do. This automatic, thinking thing of dedication at scale. And we don't have it. The. Third thing that's failing is supply. Chain supply. Chain security is actually insurmountable. Hard. Now. We've seen it you've, seen the papers in the past year it's been one of two stories it's. Kaspersky. Right should we trust, a russian-made, antivirus. Program and, Hue. And ZTE should. We trust Chinese, made phone equipment, but. That really, is just, a tip of the iceberg, there. Are other stories not, just the u.s. turns, out in 2014, China, banned Kaspersky. They. Also ban Symantec, by the way. 2017. I start from India identifying. 45, Chinese, phone apps that they say shouldn't be used. In. 1997. None of people remember. There was uh worries. In the US about checkpoint, and, israeli-made. Security. Product you know should we trust it. Also. I like, to remember, 2008. Program. Called Mujahideen, secrets. Which. Was an isis created. Encryption, program, because, of course you can't trust Western, encryption. Programs. But. You, know the country, of origin, of the product is just the tip of the iceberg. Where. Are the chips made where. Is the software, written where. Is the device fab. Where. The programmers. Are from. And. I mean this, iPhone, probably has one a couple of hundred different passports that. Are. Programming, this thing it's not made in the US and, every. Part, of the chain is a vulnerability. They. Are there a paper showing, how you can take a you. Know a good, chip, design, that the masks, and.

Maliciously. Put in another layer and. Compromise. The security the chip without, the designers knowing it and it, doesn't it doesn't show up in testing. There. Was another paper about two years ago you. Can hack, an iPhone through a malicious, replacement, screen. Right. You have to trust every. Piece of the, system. The. Distribution, mechanisms. We've. We've seen backdoors, in Cisco. Equipment. Remember. The NSA intercepted. The. Cisco. Routers, being sent to the Syrian telephone company as, one of the great pictures, from the Snowden documents, we've. Seen fake apps in the Google Play Store. We. Know that Russia, attacked, Ukraine. Through. A software. Update mechanism I. Think. My favorite, story it's. A this is a hard one 2003. There. Was actually a very clever, very subtle, backdoor, that almost, made it into Linux, we. Caught it and we kind of just barely caught it we. Got very lucky. There. And, you look at the code, it's. It really is hard you have to look for it to see the back door. Now. That could have easily gotten in we, don't know what else has gotten in in what, and. Solving. This is hard no. One wants a us only iPhone, is probably a impossible, and B it'll cost 10 X. Now. Our industry. Is at, every, level, international, it, is deeply, international. From. The programmers. To the to the come to this to the objects, to the cloud the services. We. Will not be able to solve this easily so. A. Lot, of ways this, is a perfect, storm if things are failing, just. As everything is becoming interconnected. And. Well and I think we've been ok with a unregulated. Tech space because, it, fundamentally didn't, matter and that's. Changing, and. I. Think this is primarily a policy, problem and. In. My book I spend most of the time on policy. And. I talk about a lot of different policy, levers, we have to improve this. Talk. About standards, regulations. Liabilities. Courts. International. Treaties I, think. Is a very hard political battle and I, don't think we're gonna have in the US until a catastrophic event. You. Know I look more to Europe to lead. And. I could go through all of this if I want to want to give two principles. I want to pull out the first, is that defense must dominate I think. We as a national. Policy need, to decide that defense wins. That. No longer, can we accept. Insecurity, for offense purposes, that. As these computers, become more critical. Defense. Is more important. And. I gone to the days when you can attack, their stuff and defend our stuff everyone. Uses the same stuff we. All use tcp/ip. And Cisco routers, and Microsoft, Word, and PDF, files and. It's. Just one world one network one answer either.

We Secure, our stuff. Thereby incidentally. Securing. The bad guy stuff or. We keep our stuff vulnerable, in order to attack the bad guys thereby. Incidentally. Rendering. Us vulnerable and that's, our choice. And. It means that it means a whole bunch of things, to. Disclose and fix vulnerabilities. To. Design for security not for surveillance, encrypt. As much as possible. To. Really separate. Security. From spying. Make. Law enforcement, smarter, so they can actually solve crimes, even though there's security, and. Create. Better norms. One. Other principle, is that we need to build for resilience, when. You start designing systems, assuming. They will fail. And. How do we contain, failures, how do we avoid catastrophes. How. Do we failsafe or fail secure. Where. Can we remove functionality. Or delete data how. Do we have systems, monitor other systems, to try. To provide you, know some level, of redundancy and. I. Think the missing piece here is government. Now. The market, will not do this on its own. But. I have a problem. You. Know handing us to government because. There really isn't, an existing, regulatory structure, that could tackle this at a systemic level, guys. Because there's a mismatch between the way you a government, works and the way tech works the. Government operates in silos, the, FAA, regulates. Aircraft, the FDA, regulates, medical, devices and. The FTC, regulates, consumer, goods someone. Else does those cars and. Each. Agency, will have its own rules and on approach and own systems, and that's. Not the internet right, the internet is this freewheeling. System. Of integrated, objects, and and networks. And it grows horizontally. And it kicks, down barriers. And it. Makes. People, able to do things they never could do before and. All of that rhetoric is true I mean. Right now. This. Device. Logs. My health information. Communicates. With my car. Monitors. My energy, use and makes, phone calls, right. That's four different probably five different, regulatory agencies, and, if, this is just getting started. Right. We're. Not sure how to do this. So. In my book I talk about a bunch, of options and, what. I have. And. I. Think we're gonna we're gonna get eventually is a. New. Route in a new government agency that. Will. Have. Some jurisdiction over, computers. This. Is a hard sell to a you know low government, crowd. But. There, is a lot of precedent, for this in, the last century pretty much all major technologies. Led, to the formation new government agencies like ours. Did planes. Did radio, did nuclear. Power did. Because. Government, needs to consolidate, its expertise. So. And that's what happens first and then, there is need, to regulate I. Don't. Think market solvus markets. Are short-term markets. Are profit motivated markets. Don't take society, into account, markets. Can't solve, collection, I have collective action problems. So. Of course there are lots of problems with this my. Governments are terrible, being proactive. Regulatory, capture is is a Ria's a real issue I. Think. There are difference between security and safety that. Matter here it's, safety against things like a hurricane and secured, against an adaptive. Malicious, and tells an adversary or, sort. Of very different things and. You. Know we live in a fast-moving technological. Environment, and it's. Hard to see how government, can stay. Ahead of tech. There's. Something that's changed, in the past couple of decades their, tech moves faster, than policy. The. Devils in the details, and and, III don't have them but. This. Is a conversation, thing we need to have. Because I believe, that, governments, will get involved regardless. That. The. Risks are too great and the stakes are too high. Governments. Are already, involved, in physical systems they. Already regulate, cars and appliances, and, toys, and. Power. Plants. And. Medical, systems. So. They already, have. This. Ability. And need and desire to, regulate, those things as computers. But. How do we give them the expertise to do it right. My guess is the, courts are gonna do, some. Things relatively, quickly because. Cases will appear and that the. Regulatory agencies, will. Follow I, think. Congress comes last, but. Don't, count them out nothing. Motivates a, government. U.s. government like fear. Let. Me think back to the terrorist acts let's head up September 11th we, had a very small, government.

Administration. Create. A massive bureaucracy kind. Of out of thin air and. That. Was all fear motivated, and. When. Something happens, there. Will be a push that something. Must be done and, we. Are passed the choice of, government. Involvement versus, no government involvement our, choice now is smart. Government violent, versus stupid government involvement and the. More we can talk about this now the. More we could make, sure it's smart. My. Guess is any good regulation, will incent. Private industry, but. I think the reason, we have such bad security, is not, technological. It's more, economic. There's. Lots of good tech and. You. Know while some. Of these problems are hard they're, like send demands the moon' hard they're not faster, than itravel, hard and. Once. The, incentives, are in place in. This, we will figure out how to do it right I'm a good example it might be credit cards. In. The early days of credit cards we were all liable, for for, fraud and losses. That. Changed in 1978. The Fair Credit Reporting Act. That's what mandated. The maximum liability for credit card fraud for the consumer is $50. And. You ever seen what that means that means I could take my card fling, in the middle of this room give. You all lessons, on forging my signature and, my. Maximum, liability is $50, right it might be worth it for the fun. But. What that meant right that changed. That even if the consumer, is at fault the, credit card company is liable, that. Led, to all sorts of, security measures, that. Led to online. Verification. Of of. Credit. And and card validity, that led. To anti, forgery. Measures, like the Holograms, and the micro printing blood, led to mailing, the card and the activation, information, separately, requiring you to call from a known phone number and, actually. Most importantly, that, enabled, the backend expert systems, that troll the. Credit. The, transaction, database looking, for chill and spending patterns none. Of that would, have happened if. The consumers, were liable because the consumers, had no ability, to implement, any of that you. Want the entity that can fix the problem to be responsible. For the problem that is just smart policy. So. I see a lot of innovation that's, not happening, because the incentives, are mismatched. So. I think Europe is moving in this direction right. The, EU is right now the regulatory, superpower, on the planet and they, are not afraid to use their power we've. Seen that in the GDP are in the privacy space I think. They're gonna turn to security next I. Mean. They're already working on what responsible, disclosure means. There's. That you, ever seen on manufactured, goods there's that label, called C II that's. An EU label basically, means.

Meets. All applicable, standards. They're. Working on standards for cybersecurity, and. You. Know you still see them get incorporated trade, agreements, into GATT and. There's. An interesting rising. Tide, effect. It's. Not necessarily obvious the, the, car you buy in the knighted States is not the car you buy in Mexico right environmental laws are different and the cars are tuned to the different laws. But. Not. True, in the computer space. The. The. Facebook you get is pretty much the same everywhere, and. If. You can imagine there's some security regulation, on a toy, the, manufacturer, meets it they're not gonna have a separate, building United States they're gonna sell it everywhere because it's easier. That, and even there'll be times and that's not true I think, Facebook would like to be able to differentiate between, someone. Who is subject. To GD P R somebody was not because. There's more revenue to be gained through them the greater surveillance. But. When you get two things I think it's more likely that it'll. Be a rising tide and we all benefit. The. United States look, to the States more especially. New York Massachusetts California. Which. Are more aggressive, in. This space. But. I think this is coming. And I, want. To close with I. Guess, a call. What. We need to do is, to, get involved, in policy. Technologists. Need to get in policy, get involved in policy as Internet, security becomes everything security. The. Internet security technology, becomes, more important, to overall, security, policy, and, all. Of, the. Security policy debates will have strong, technological. Components. We. Will never get. The policy right if the policy, makers get the tech wrong right, it will all look like the Facebook hearings. Which. We're embarrassing, and. You see, it even in some of you see it in the going dark debate you, see it in the equities, debate you see in voting machine debates in, driverless. Car security, debates. That. We need technologists. In the, room during, policy, discussions, we. Have to fix this. We. Need technologists. On congressional, staffs and NGOs. Doing, investigative, journalism, in the, government, agencies, in the White House. Right. We. Need to make this happen, and right. Now. You. Just don't have that ecosystem, so. You think about public, interest law. 1970s. There was no such thing as public interest law there. Were actually wasn't it. Was created, primarily by the Ford Foundation oddly, enough that, funded. Law, clinics, funded. Internships, in different. NGOs and, now. You. Want to make partner at a major law firm you, were expected, to do Public Interest work.

Today. At Harvard. Harvard. Law School, 20%. Of the graduating, class doesn't go into, corporations. Or law firms they go into public interest law and the. University. Has soul-searching. Seminars, because that percentage is so low. Percentage. Of computer science graduates is probably, zero. Right. We need to fix that and that's, more than just. You know every. Googler needs to do an internship because, there aren't spaces, for those people. So. We got to fix a supply gotta, fix the demand, ecosystem. To link the two I mean. This is of course bigger than security, I think, pretty much all the, major. Societal. Problems of this century have a strong tech component, a climate. Change future. Of work farm. Policy. And. We. Need to be in the room or. Bad, policy, happens to us. So. That's my talk there's, of course a lot more in the book that I didn't say and I'm, happy to take questions. I. Do. You imagine, that, some of the like, socio-political. Things. That we're seeing crop up fit within this framework or. Do you think that that might be an entirely. Separate. Problem, that needs an entirely separate, set of solutions, I think it's related I, mean, this problems I'm talking about a pretty purely, technical the. Problems, of, Internet. As proper as a propaganda vehicle. Are I think. Much more systemic. And societal, I do, blame, surveillance, capitalism, for. For a bunch of it right the business model that, prioritizes. Engagement, rather. Than quality. Has. Learned that if you're pissed off you stay on Facebook bored so. So. I think there are pieces. That, fit in, so. Some, related some different um, you should be talking a lot about policy. That the United States and that's to some extent that you can do but I wonder what do you think will happen as policy. Everywhere. Policies. Local the arnessk global and how is that going to play so I think that never goes away and. Some. That's going to be the rising tide I talked about that. Especially when you, know less, about privacy, but when you get to safety I think, it's more likely that we. Benefit, from a, European, regulation. That, ensures, that you. Know the smart vacuum cleaner you bought can't, be taken over by somebody and then like attack you and trip you right, we're. Likely to data from that more than. Look you can't have a microphone, on the thing. We. Have to assume, that, there will be malicious, things, in, whatever. System, we have so. If we have a US only regulation it'll, clean up a lot of the problem because Walmart, won't be able to sell the bad stuff but, you can still buy it in mail order from Alibaba comm right so there will be some, stuff in the network that, is malicious much. Lower percentage easier, problem, we're. Still gonna have to deal with that and I think it ever goes away because.

We're Not gonna have world government, there will be a jurisdiction. Or. There will be homebrew. Stuff that. Doesn't meet whatever regs, we have that that'll always happen, you, talk about the need for intelligent. Technologists, to get involved with making policy but, there only so many hours in a day and we. Probably. Muslims would be taking a huge pay cut to go work in government and lend our expertise there so what how can we fix the incentives, there so, some, of it is desire, I mean, I know ACLU. Attorneys that are making a third of, what they would make at a big law firm and they. Get more resumes, than they have positions, so. It works in law the. Desire to actually make, the world better turns. Out to be a prime motivator so, I think once we have the ecosystem. We will get the supply I think. That enough of us will say you, know we've had great careers we're gonna take a break or. We're. Gonna do something before we go work at a start-up or a big company or maybe. That mean there will be a use, for sabbaticals, like, you see in law firms or you, know you bits of pro bono work you. Know like a 20%, project, so. I mean yes you will people will be making less money I don't, think that. Is going to. Harm. The, system I think we just need to get a system working the. Most jarring thing I saw. You write as a Googler, was that, data. Is a toxic, asset. What. So. The. Promise, of big data has been save, it all figure out what to do with it later and. That's been driven by the, marginal, cost of saving it has, dropped to zero is basically, cheaper, now to save it all and to figure out what to save and. Dis storage is free processing. Is free transport, is free but. It turns out that data, is, a toxic, asset, that. For most companies, having. It is an, enormous liability because someone's. Going to hack it's it's going to get sold and you're going to lose it and I. Think. We need to start talking about data, not as this sort. Of magic, goodness. But. It no. Decays, in value. Than. That there are dangers and storing it or the the best way to secure your data is to delete it and you're. Going to delete it if you don't know if, you know you don't need it, okay. So I've seen lots of studies on data. On. And shopping preferences and it turns out some, pieces of data are very valuable and a lot of it just isn't very valuable so, is it worth the extra quarter. Percent of accuracy to have this day that is potentially, dangerous, and we'll get you find, or embarrassed, and just take a stock takes a hit if it gets stolen so. I think we need to make more, of those decisions, that the data is radioactive, it's toxic right, we keep it if we need to, but. If we don't we get rid of it we figure how to get rid of it safely and securely I. Mean. Take I don't know ways right ways is a surveillance, based system.

Very. Personal, data but. Probably only available for like 10 minutes. Or. At least you know it, can be sampled I mean. Lots of ways I can, treat that data. Understanding. It's a toxic, asset get my value, and, much. Less risk to my organization and. That's. What I mean by that it's a interest. In that there are ways to anonymize, stuff, but, there seems to be no demand. And no supply, there. Are marginally, more, expensive. To do federated. Machine learning than do everything in the center but, okay companies. Don't care and consumers. Decidedly. Tony consumers. Don't care that's why me. You need these decisions, made not by consumers, but by citizens, many. Consumers don't care right consumers, are buying, the Big Mac at 10%. Off Zoomers truly, don't care the point of purchase nobody, cares, at, the point of reflection people. Care a lot and that's. Why, you. Don't want the market doing this you want us, as our best selves, doing. This. So. And. About. Anonymity it is harder, than you think in, most of our ways of an anonymizing. Data. Fails. It. Is a very, hard, problem we don't, the. Anonymity research is really, I mean X is the breaking enemy research is very good these days and. And outstripping, the anonymity research the. Next one, of the things that I'm thinking about is a lot of times when you see like a big vulnerability. So say there's a big operating, system vulnerability, it's actually a genuine mistake of it's not that someone you. Know put it in there on purpose it's they missed something so how does regulation, solve that problem, of sure, you could have some great regulation, in place that something's supposed to be done a certain way but oh the implementation, was slightly off or slightly broken how. Do you fix that you know genuine, mistake even. If they were trying to do what the regulation specified, as this would be a secure system so, you'd be surprised, but.

Financial. Motive fighting up money, motivates, companies, if. Companies, will be fine a lot of money if their employees make a mistake they, figure out ways to their employees to make fewer mistakes but, doesn't that only take effect after the mistake, has already been like, the. Evil has already been dying, there's. A deterrence effect okay. So so yes I mean like arresting. Someone for murder only takes, effect after he's done murder but, the goal is that the threat of being arrested for murder will keep you from hurting someone tomorrow right, and so so we want this deterrence, effect. Exact. Result. For mistakes. We, actually know a lot of techniques, that pretty much all software manufacturers, never do because it would be slightly more expensive, but. If it's a lot more expensive not to them but. Suddenly the math changes. And. I need, the math to change I need security, to, be cheaper than insecurity right, now the market, rewards. Let's. Just take the chance. You. Know let's hope. For the best. Ok. I'm in it no industry, they say this already, remind. Me yes no no ok no industry. In the, past hundred of something years has, improved security to state without being forced to. Cars. Planes. Pharmaceuticals. Medical, devices, food. Production. Restaurants. Consumer. Goods. Workplace. Most, recently financial, products. The. Market, rewards. Doing. A bad job hoping. For the best and. I. Think, it's too risky to allow that anymore, hi. So, you mentioned that you were embarrassed, by the Zuckerberg, hearings, and I was embarrassed by the questions. At the Zuckerberg yes ok. Look. Congressmen embarrass me so I so. I so. I assumed correctly I assumed you were embarrassed, by the. Senators, yes yes. Whereas I have, the opposite, problem I was embarrassed by Zuckerberg, what, would I do me fair there's a lot of embarrassment to go around yes no, but we can both be right of saying yeah but I have, a serious point here though is that while. The Senators don't know about tech I think, the tech doesn't, know about law. Ethics. Political. Science. Philosophy. Like, do, you think Mark Zuckerberg, can even teach an introductory college. Course on free speech like.

Has Even read like. What anyone has ever said about it so like shouldn't we all be learning, about. The. World yes this, happen this, has to go in both directions yes right I want, techies, and politicians. I want, policy, people in tech. Companies so yes I think we need both you need both sides talking to each other right, and so. So I agree. With you 100%, good. Ok. All. Right so so, right now I teach, internet security, at the Harvard Kennedy School at a public policy institution. So. I'm trying to go push, people in that direction at. The same time there are people at Harvard the computer science department trying. To teach policy. Issues let's, go in the other direction I think, you probably know I. Know. But I'm not in charge, you. Mentioned like sort. Of shock events, as things, that drive government, policy, and I thought, the example, of 9/11, was like, instructive, maybe in a way you did or did not intend in that like, the government response to 9/11, was to launch two illegal, wars and create a surveillance state that violates. Our civil liberties a. Good. Like I guess I'm curious tell you it's parable right so how do you see the reaction, to the, reaction to the mounting threats in technology, as being different what's going to prevent the same sort of thing from absolutely, nothing okay that's, that's, my fear that something. Bad, will happen. Congress. Will say something, must be done this. Is something, therefore. We must do it, all. Right so, my goal of having this conversation now. Before. This happens, is that. We will as a, community. Figure. Out what should be done when. We have the, luxury of time. And. And insight, and patience. And because. I agree with you that there is a disaster, we. Will get a disaster. As a response, and it, will be just as bad so, let's get ahead of it this time, let's do better, how. Do you, envision. Preventing, everything degenerating. To the lowest common denominator, like you said client-side you can't really restrict people from doing what they want you, know even if we say okay any company that wants to make money in the US has to follow these provisions I'm, just going to encrypt my data and send it to Alibaba, translate, it's a third the price of Google Translate but they steal all my data like. How. Do we prevent this. Is. There anything we can do you know some of it is the answers gonna be no some it's gonna be yes so. If you think about other. Consumer, goods we. Do make it hard for consumers, to modify something it's that she's hard to modify, your car to. Violate emissions control right. You can do it but. It's hard and then. We try to have spot checks you. Can imagine some, sort of regime you. Can imagine some system, that tries. To maintain security anyway because it will be able to the minority doing that I, think. Once we start. Hitting. The problem, for real we'll.

Come Up with tech solutions, and, ways the system to self to wash itself other. Systems to watch each other can. We do this non invasively I think, we have to figure it out so, I don't have the answers here but, you mean these are certainly the problems, I. Really. Liked your phrasing. Of the problem. Of we need to give, up on offense so we can go all-in on defense and I. Think. It's it's pretty clear to me where a lot of the offensive, focus is in, terms of law enforcement but. I think one thing that sort of remains. Mostly. An unknown is on, the military side and how, there's a ton of investment, in military, offensive, stuff and we kind. Of know you know a little bit more maybe about like what Russia and China are using. Offensively, against, us, and. We have seen the good stuff yet. Anyway. But. But, do we do we have a sense of what, the military did. The the you know US military, let's say would be giving up to, give, up this this offensive. Idea. And I. Don't. Know how willing, they would be to go, with that direction they wouldn't be willing but it's not their job to be willing it's. Why you don't want the NSA in charge of your privacy policy because. It's. Not their job but you know we need people. Above, the. Military, the NSA, to, make these trade-offs, because. They are security. Versus security trade-offs, right. Is the security, we get from, being able to spy on and hack the bad guys greater. Or less than the security we get from. The bad guys being unable to spy on and hack us, that's. Right. So. I mean security versus surveillance is the wrong way, to describe it security, versus security so. Someone above this the the, military needs, to decide that it can't be the military because, the military is not in charge of overall, policy, they're in charge of the military part and what, we know but the capabilities, is very, little I mean we get some, shadows of it here or there and it. Seems to be you know we you know on the one hand cruder, then and then we'd like it to be on.

The Other hand you know Stuxnet, was pretty impressive you. Know, in general the stuff you see is sort of them the minimum tech, it has to be to succeed, there's. Sort of this myth of this these. Uh you, know super. Powerful cyberattacks. That, basically. You know one iota more. Than just barely necessary, to succeed you. Know. You don't need to do more if you can take out the DNC with a you know pretty sloppy phishing, campaign I. Mean. Why bother using, your good stuff so. A lot, we just don't know. At. Risk of revisiting an earlier question I was. Interested, in what you thought about so, like one of the things that's often you see cynical in the finance industry is that they think that people. In finance can outmaneuver all, the people, who are regulating, them in part, because they're less are paid so I was wondering if you could revisit that because I think law has maybe, the exception because it's a little bit more directly related, to human. Rights and things like that that yeah. I don't know I mean certainly I, worry. About regulatory, capture. Regulations. Being evaded, I think, that all of those are real risks I mean this is not a great, answer I have it's just the best one I have. Because. I don't see any way, to. Put a backstop, against, so this massive, corporate power other. Than government power now, in a, sense out of kind of want either power, but. Tech. Naturally. Concentrates. Power at least as its configured. Today, so. That's. My missing piece I think you're right that that is a serious. Problem. And worry and something. We just have to deal with a policy. Is iterative, as. Techies, it's hard to to. Accept that we like to get the answer right and implement it whereas, policy gets the answer like slightly less wrong every few months. But. That's you, know that's the way it works I mean. The real question is can we do this at X speed and that. It really is I think it's an open question. So. With that I'm going to end, thank you all thanks for filling the room thanks for coming. You. You.

2018-10-17 21:28

Other news