[Music] Welcome to TecHype, a series that debunks misunderstandings around emerging technologies, provides nuanced insight into the real benefits and risks and cuts through the hype to identify effective technical and policy strategies. I'm your host, Brandie Nonnecke. Each episode in the series focuses on a hype technology. In this episode, we're debunking artificial intelligence. You cannot avoid hearing about artificial intelligence or A.I..
It's literally everywhere. What do you search for something online? An algorithm was used to provide you a response when you go to the hospital to get a CAT scan and a AI powered image recognition system was probably used to aid the doctor in spotting any abnormalities. When you apply for a loan, no surprise there.
Again, an algorithm is used to determine your credit worthiness. Last year, over 14,000 AI startups were in operation in the US alone. A PWC Global artificial intelligence study shows that AI actually has a 15.7 trillion with a T trillion
dollar potential contribution to the global economy by 2030. A.I. appears to be simultaneously the greatest benefit and the greatest risk to the world.
While it can contribute to greater efficiency and effectiveness, the technology also poses serious safety, security and bias risks. So what can be done to better ensure we realize the benefits of this transformative technology while mitigating its risks? Today, I'm joined by Professor Stuart Russell OBE, professor of computer science at UC Berkeley, director of CHAI, the Center for Human Compatible AI and also director of the Kavli Center for Ethics, Science and the Public. Author of Artificial Intelligence A Modern Approach with Peter Norvig, which is the standard text I understand in AI.
It's been translated into 14 languages and is the author of Human Compatible A.I. and the Problem of Control. Stuart, Thank you so much for joining me today for this episode of TecHype. Well, thanks for inviting me. Thank you.
I think it's really important that we first start with a definition. There's a lot of misunderstanding around what is artificial intelligence. You are the expert. What is artificial intelligence? So I think everyone understands it's about making machines behave intelligently. And that dream actually goes back thousands of years.
You can even find Aristotle talking about fully automated musical instruments that play themselves and things like that. So what does that mean as an engineering discipline? So that what I call the standard model, since pretty much the beginning of the field, has been machines whose actions can be expected to achieve their objectives. So this was borrowed from notions of rationality and economics and philosophy. And it really focuses on how it behaves.
And then, you know, if you want a thought process inside, well, that's an engineering decision. You know, is that the right way to achieve this kind of intelligent behavior? But now when we go online and we interact, which ChatGPT as many people have found out, it's really hard to dispel the illusion that you're actually conversing with an intelligent entity. And it's so in the same way that, you know, if you ever watched the movie Titanic. Right. Yeah, of course. It looks like there's water.
Yeah, there is no water. Right. It’s all computer generated water. There isn’t any water. And. But you can't look at the movie and think it's anything other than water, because, you know, it's splashing and foaming and people are getting wet and all the rest. There’s got to be water.
No, no water. So we we need to learn how how do we inoculate ourselves against this illusion And one one way to do it is to sort of think, okay, when I read a book that’s written, it's got all kinds of intelligent arguments and so on, maybe some beautiful poetry. Do I think that the paper embodies the intelligence? No, of course not. Right. I think. Okay. Yeah. There's a human over there somewhere.
And they wrote all this thing they created. They're very intelligent, and they create this. And it's just, you know, it's just a an artifact.
An artifact. A medium. And ChatGPT is somewhere in between. So it's on a spectrum from just a printed copy of human human sayings to, you know, it's something that actually is originating through a thought process. And I don't here mean a thought process that involves real consciousness, real subjective experience. I think that's a whole different story, but just a thought process where it's meaningful to say that it knows things.
It's meaningful to say that it goes through reasoning steps that when you ask a question, it's referring to its knowledge to answer the question right? You know, so if if I asked you, you know, where is your car parked? Right. You have an internal picture of the world, you refer to it and you said, yeah, it's on the fourth floor, the parking lot. So, yeah, so that's how humans mostly answer questions, but sometimes we don't. Right. If, if someone says to me. Hi, Stuart, How are you today? I say, fine thanks.
How are you? Right. I'm not really referring to an independant model, and if I did, I would go on and on complaining about this, that and the other. Right. So sometimes we just respond in this sort of automatic reflex way. And as far as we know, that's mostly what these systems are doing. Mostly except Bing Sydney, which is completely unhinged if you ask it a question.
Well, but.. it's the people believe and again, we don't know it because we don't have access to the training sets that they probably trained it on a lot of emotional conversations between individuals. Drunk texts at 2AM I think. That's sounds like it's just. Like a psycho girlfriend or psycho boyfriend.
I think so, yeah. You’re trying to dump and they're trying to convince you that actually, no, they're the right person for you. There was a lot of that going on. A lot of red flags.
But you know, it's talking about its feelings for the person who is interviewing it or asking questions. Of course, it doesn't have any feelings as such. So this is just fictional. It's not referring to any internal model or state.
It's all it really does is it takes the previous 3000 words of the conversation and based on training, on trillions of words of text, it outputs the most likely word that comes next. Right? So in that sense, it's like a, you know, a parrot or a book. Yeah, right. That is. But it's a transform transformational process from this vast corpus of training data. Exactly.
And we actually have no idea what that transformation process is, how it works inside. You know, it's theoretically possible, that actually ChatGPT really does have internal knowledge. states, really does has developed internal goals solely for the objective of becoming better at predicting the next word. Because where are those words coming from? They're coming from humans, Right.
Those humans have internal goals, and that's why they wrote the next word. Right? They didn't write he didn't write the next word because the previous 3000 words were on the page. Yea. Right? They wrote the next word because they're trying to tell you something because they want something.
They have their own internal drives. So it might be that the best way to predict what the human is going to say next is actually to sort of become like a human right, to actually develop internal goals and knowledge structures and reasoning and planning and and all the rest. But again, we have no idea because we didn't design these systems, right? We just train them. And what's happening inside, we have no control. Yeah. Now I've played with ChatGPT a bit.
Now, I don't know if I should say this aloud, but I, you know, I gave it a prompt, something like write an op ed on this topic and it spit it out. And it was almost exactly how I would have written the article. And then I thought, okay, maybe we're not so creative as we think. When this is creating something I would write or I said one time, you give me a syllabus for an artificial intelligence governance course, and it pumped out everything that I would think to put in the course. But there are probably many, many such syllabi already on the way. Exactly.
And that just shows and we look at each other as an individual, we're not necessarily that creative, but we're following a norm of the profession. So maybe this can reveal to us how uncreative we actually are. Yeah. And pushed us to be more creative. I think it could.
It could actually have a positive impact on how we think about educating. As actually we don't we don't want to train a lot of human ChatGPTs. No. Right? Well you use in the classroom? Will you have your students use ChatGPT? Uh Not for, not for what we're doing. And I think there's a, there's a debate going on right now in the mainly in the media. Yeah. And you've got the,
you know, some educational experts saying, you know, anyone who thinks students shouldn't use ChatGPT is just one of these dinosaurs the same kind of dinosaur who said, well, we're back in the 19th century, no even long before that. So in the 19th century, people were saying, you know, if students ever start using these mechanical, calculating devices, they're going to be the end of civilization or something like that. Right. And I have the sort of two responses to that. Right.
So what a mechanical, calculating device or an electronic calculator does is actually automate an extremely mechanical process of following an arithmetic recipe game. Right. I bet you you said most of listeners, including me, I, I don't really understand what's going on when I'm doing long division. It's just a recipe. I know you're supposed to bring these numbers down and keep things in the right columns and and carry. Carry the one. Carry the one, and do this and that. Right?
And you write down the dividend and the this that that and the other. But what's actually going on? Why does that give the right answer? No one ever teaches you that it's purely mechanical and it's not really about the understanding of number and arithmetic. But if we were to give people calculators but never teach them what do numbers mean, what does plus mean? What? What's multiplying four. Right?
What is this sine function about? It would it would be an incredible disservice to them. Yeah. So if we give them ChatGPT to answer all the questions that we sent them, then they will never learn how to write, how to think coherently for more than one sentence, how to put together an argument, how to marshal facts. And facts is very important here because ChatGPT marshals fiction as much as fact.
I know I've seen that in the syllabus I was mentioning earlier is cited journal articles that don't exist, just made them up. Yeah, I mean you can there was an example, someone asking it what's the most cited paper in economics? And they it made up, I think it's called a Theory of Economic History, which just doesn't exist yet. You know, it had some real authors. It sounds lofty. But they never wrote anything like that.
And and so it's just complete fiction. You can ask it, you know, what's the largest even number? And it says 9,999,998,998. And that's like. Yeah, exactly.
It's like obviously silly. And that's because as far as I can tell, it doesn't have an internal reference model. It doesn't actually know things in the same sense that a human knows things. So one of the things we do with our internal knowledge of the world is we try to make it consistent, right? Because we know there's only one world. So if our internal model is locally inconsistent with itself, then there must be something wrong and you have to resolve, try and resolve that that internal contradiction.
But there's no such internal structures in ChatGPT. It clearly couldn't care less about contradictions because, you know, it can say in other examples from my friend Prasad Tatapali which is bigger an elephant or a cat. And it says an elephant is bigger than a cat. And then you can say, which is not bigger than the other, an elephant or a cat.
And it says, neither an elephant nor a cat is bigger than the other. It contradicted itself In the space of two sentences on a pretty basic thing. Yeah.
So in TecHype, every episode we debunk three misunderstandings and I think in our discussion so far we've touched on a few of them, but let's solidify those. So one I think is about this internal consistency, internal logic that when you interact with this system, it feels so human that you think it's smart. Yes. So a part. Is that number one misunderstanding?
Well, I think that's that's one of many misunderstandings in the media. So I think probably one of the most important misunderstandings and this is this is filtering even into very high level policymaking, for example, in the European Union and in the UK government and other places where they are in the process of making laws that are going to regulate A.I.. There's this misunderstanding that the A.I.
and machine learning, and particularly former machine learning called Deep Learning and AI, which became popular around 2012, are the same thing. It's surprising to me because we think that we're in this new state. Are we in a new state right now? Is it actually different? I think some interesting things are going on.
So if you stand back and say, well, what is deep learning and why does it work better, we obviously we had machine learning methods before that, and there's an entire field of statistics which thinks of itself as is in the same business, namely taking data and training predictive models in order to predict things from the data. So what changed? So I think if you look at the models that we were using before, the two primary categories would be decision trees, which you can think of as sort of long, thin decisions. So each branch tests some attribute in the input. So, you know, if you're trying to fix a car, you say, okay, well, does the engine turn on when you turn the key? Yes or no? Okay. Well, if the engine turns on and the car still doesn't work, right? Well, you know, is is the gearlever engaged or are you in neutral? Yes. No, when you follow that sequence and then at the end it says, okay, your you know, your fan belts broken or, you know, you're out of gas or something.
Right. So it the diagnosis is that the means of the treat. So that's kind of a long skinny. Yeah but but in that are you telling you what to do. Are you telling it. Check this, check this, check that. This or those those trees are generated by a machine learning process, right? Well, I mean, they could be built by hand.
In fact, that's one attractive characteristic of those systems is that you can look at them and understand what they're doing. You're right. But so machine learning developed decision tree methods, as did statistics, and they're widely used in industry. And then the other you might call instead of long and skinny, you might call them sort of short and shallow or short and fat ones right there, methods like linear regression and logistic regression, which sort of test all the attributes at once and then just apply some simple function like, you know, add them up. And if the sum of all the attributes is more than this, then you know, then you have the disease. Otherwise you don't have the disease or whatever it might be.
And those methods are used. For example, in credit scoring, your FICO score is exactly the output of a logistic regression function applied to a bunch of attributes about your payment history and all the rest on. So we had long and skinny and short and fat and deep learning, basically long and fat. And then as we wrap up this show, I would like to hear from you quickly.
What do you think are the greatest benefits and risks of AI? And then I want to turn to some strategies, technical or policy strategies that you think we need to implement. So greatest benefits? Greatest risks? Well, so the benefits of AI are in a sense unlimited, because if you if you think about, well, what is the current level of intelligence that we have access to? What does that buy us? It buys us our entire civilization. Right? Everything that we're able to do in the world is the result of our intelligence. And so if we had access to a lot more, we could have a much better civilization, you know. I hope so. In a in a simple sense, right. So I did in the human compatible book that that you mentioned, I did a little back of the envelope calculation saying, okay, suppose we have general purpose A.I., which means A.I.
systems that can do anything that human beings can do. Including embodied in physical robots and so on. So by definition, those systems would be able to deliver all the benefits of civilization that we have learned how to create so far and deliver them to everybody at basically negligible cost. Ok. I like this utopian thinking. Right? So no science fiction, right? We're not inventing faster than light travel or, you know, eternal life or any of those things.
We're just saying deliver what we already know how to deliver except do it in a fully automated way. And so that would raise the standard of living of everyone on Earth to a respectable level that you would you would experience in a developed country. And and that would be about a tenfold increase in GDP, which translates in terms of net present value, like what's the cash value of that technology? Yeah, it turns out to be about $15 quadrillion.
Okay. So what's the other side of this coin, though? Right. So, so those are some of the benefits and that creates an enormous momentum. Right? So if you start talking about risks, you might you know, people very quickly go to, well, there's so many risks. Maybe we should ban. Brakes, put the brakes on.
Ban the technology, put the brakes on. Slow AI. Like “slow food.” Guardrails, all the words.
But those kinds of thoughts, I think, have to be tempered by the knowledge that the momentum towards achieving general purpose AI is fast and it's going to get bigger. Right, and if you think the tech companies are big now. Right. As we approach general purpose AI, they will be the economy of the earth, So, so the momentum is, I think, probably unstoppable unless we have a very serious and very obvious accident.
So think of it as like a Chernobyl on steroids that we attribute to AI going wrong. What are those guardrails? What should we put in? What do you think we should do? Right now? With our technical or our policy. Well, it depends on which risk you talking about. And there are a bunch of risks that are already in play lethal autonomous weapons, where the risk, the primary risk there is actually not accidentally killing a civilian. The primary risk is that because they're autonomous, they can be scaled up, that one person can launch a thousand or 100,000 or 10 million weapons and wipe out an entire country. So that's that's a very serious risk.
And it's been very difficult to get governments to even acknowledge that that's an issue. You know, there are risks from the way social media operates. So social media algorithms control what billions of people read and watch. Right, they have more control over human cognitive intake than any dictator in history has ever had.
And yet they are completely unregulated. Perfectly targeted propaganda. Yeah. So but yeah, it's individualized and sequential propaganda. Right.
Because the system sees whether what it tried to get you to do worked and if not it'll try something else. So it's kind of like a reinforcement learning system. The main concern, you know, and Alan Turing I think put it very succinctly once the machine thinking method had started, we should have to expect the machines to take control. Well, because, you know, our power over the world is our intelligence. And if these systems are much more intelligent than theoretically, they're much more powerful. They should hold the reigns.
And how do we. Well not should. No, they shouldn't. We should hold the reigns. But. At least we think we should hold the reins. But how do you have power over something more powerful than you? Forever. That's the question.
And that's that's what I spent the last seven or eight years trying to solve as a as a technology problem. It's a very thorny question. And I think with that, I'd like to thank you so much, Professor Stuart Russell. Thank you for joining me today on TecHype. It's clear that artificial intelligence has transformed society in fundamental ways, providing greater efficiency and effectiveness in a variety of domains while simultaneously posing serious safety, security and discrimination risks.
It's clear from our discussion that in order for us to move forward to realize the benefits of artificial intelligence, we must debunk its misunderstandings. TecHype was brought to you by the CITRIS PolicyLab and the Goldman School of Public Policy, at UC Berkeley. Want to better differentiate fact from fiction about other emerging technologies? Check out our other TecHype episodes at techype.org.
[Music and credits]
2023-12-31