Cade Metz, author of "Genius Makers", on the rise of AI

ai is a weird field right it's it's this combination of various fields and it's always been like this right since the 50s when the term was coined it's this blend of computer science and neuroscience and psychology that has always been the case and it continues to be the case you're listening to gradient descent a show about machine learning in the real world and i'm your host lucas beewald kade metz is a journalist who's been covering technology for the past few decades and he recently wrote a book genius makers which is kind of a historical document up until the present about artificial intelligence and the people that built the technology behind it i have so many questions about this book i can't wait to talk to him so you were the first non-ml practitioner to to ever appear on this podcast so i'm excited to do this and we might take things into different direction than um normal but i was really excited to talk to you i kind of procrastinated on on reading your book and then i actually really enjoyed it i was kind of afraid that i wouldn't you know the name made me a little worried that it might be a bit over the top or something like that and i also felt like typically when you know when you read journalism on topics you know really well it's hard not to be critical or feel like you know the the person didn't you know get something exactly right but then i actually it's it's you know it kind of reminded me of that show silicon valley just in its like incredibly accurate details like i feel like i've been in this world of of machine learning and i've been in a world of venture capital which are kind of the two main topics that your book covers and just all the little anecdotes and details they really just rank true to me like i felt like you know you do these things where you explain math you explain sort of like when someone makes fun of somebody for you know differentiation by faith what that means or like you know you describe what a tpu does and you really actually go into technical detail that i'm not even sure i would necessarily do you know if i was writing for a mass um audience and i actually think you are remarkably accurate in that and then you sort of describe these like very vivid scenes that just seem like you know sometimes i feel like when you read sort of descriptions after the fact of the scene of like an acquisition or a fundraiser or something it's like i don't think this journalist really you know it was getting accurate information or transcribed it the way it just didn't doesn't feel right sometimes but your book really felt accurate to me and it was like a really interesting lens for me just on a world that i've been sort of adjacent to you know some of the folks have been on our podcast some of them are customers of ours now so i know i know a lot of the characters in your book but i don't kind of get to know them intimately in the way that that you clearly got to know them so i actually you know the question i was kind of dying to ask you which has really maybe nothing to do with ml is just how did you get so much access and what was your process for researching this book because there are some details i'm just surprised you got someone to tell you and it's not like you're sort of recanting interviews that you did it's like somehow you just it seems like you must have actually sat down with jeff hinton for a significant amount of time to be able to write this or or maybe of some other process that i don't understand no i mean well well i will tell you but like you know it's really interesting for me first of all to hear what you thought you might get and then also what you might have gotten in in the end after reading it like in in a way there was a there was a time when i started you know work on this book and really got into it when i realized it was a really dumb idea because on one once that your audience hopefully is going to be machine learning professionals and researchers who are really steeped in this stuff and if you if you venture too far outside that world you're going to to get them angry with you and you're going to lose them but ultimately the the goal of the book should be to have any reader pick this up and enjoy it and that should be the goal as well and if you move too far towards the machine learning researcher you're going to get those people angry and they're not going to take up your book and the trick becomes to to find the sweet spot right in the middle and and that's that's very difficult and then on top of that within the machine learning community we act like that's a monolithic thing it's actually this huge spectrum as well and this is what you know i get to at the end of the book really is that you know you have some people who believe this is just math and you have other people who where this is something more right this is this is sort of this almost it's almost like a religion and this is going to create agi this is going to create a machine that can do anything the human brain can can do so within the ml community you have this spectrum where people really really disagree and and and the goal is to in somehow you know get all those people interested in your book it seems like a mistake um to even try that but here here is what i really believe in and this sort of gets back to your question ultimately this is a book about people right this is a book about some really interesting people and they are interesting in incredibly different ways from jeff hinton to demisabus at deepmind to jeff dean i can go on down the list tim neat gebrew who is in the news recently because she really clashed with people at google including jeff dean you know these are these are really interesting people who are relating a lot of different ways and ultimately that's what this book is right it's a book about the people and what i realized is that i if i can just show who these people are and who they and what their stories are and how all those stories fit together then that that's what makes it successful right and what what that's about what what finding what those people are about ultimately it's about spending the time with them you know as you indicated right and that takes a lot of doing right you just some people because they work for these giant companies you can't really get at them initially so you you try somebody else and you get some good stories from them and you go back to the first person you say hey i've got this what else can you tell me and you develop in some ways a relationship with them you know i tend to like even as you get close to these people you know keep a little bit of a distance as a journalist i think that's important too because again you've got to have an objective view and be able to to really appreciate and rope in you know the beliefs and the experiences and the points of view of all these different people but it's about years and years and years of gathering information um and understanding it yourself and and taking it back to people and say can we talk about this more and then somewhere along the way you you know you get them to talk well you know i can only speak for myself but i thought it was a really interesting book i mean just my i you know i couldn't put it down once i started it one thing i was wondering about is the ending i thought was very understated like you know it's it sort of ends with with jeff hinton who's kind of the main i mean i feel like he's almost like the main character in your book kind of saying like well maybe agi isn't that important how about spoiler alert i guess well but you know but you know i and i thought i thought it doesn't even like like he's kind of like well what like would you really want a vacuum cleaner that's this was my takeaway i'm curious like if i if i got it wrong or different than your intention tell me but you know i was thinking does he really like a vacuum cleaner that could like you know navigate my house and and be smart about like when to turn on and off and stuff i mean it doesn't really exist and i actually i think i would want my vacuum cleaner i think to be you know reasonably smart and and you know bordering on you know if it could you know reason about the world that that seems like it would actually be kind of better than than a roomba that that you know can't yet i was kind of surprised that jeff hinton thinks that and it sort of felt different than what most of the people in your book were thinking i was just kind of curious and and then you actually say you sort of say but you know he kind of invested in you know some crazy reinforcement learning company so maybe he doesn't even really think that you know what i'm kind of just kind of left with i wonder like what's the what's the takeaway here and i was wondering what's what's kind of your takeaway because you really noticeably never seem to take a position on this stuff but i mean you've been watching the field for for decades i'm sure you have um opinions on this right well you know it's interesting that you would in in some ways have that takeaway that the ending is understated and sort of questioning agi almost you know my first book review has come out there's this trade publication called kirkus which which reviews big books and their takeaway is completely the opposite like if you read the review it is kade metz is making the case that agi is possible right and so it is it is that they completely other ending spectrum in you and and i completely understand why the two of you have come to different conclusions because and it makes me happy because that's my aim my aim is not to judge my aim is not to make a call my aim is to show what is going on in the world and what has gone on over the past 10 years and you have someone like jeff hinton who is one of the most respected people in the field at the same time there are people who who can't stand him because they feel like he has gone too far there are people who can't stand him like because he doesn't go far enough and doesn't say that agi is around the corner and you know you're right the book ends with him you know in a way questioning agi but it also ends with him changing right him embracing some stuff namely reinforcement learning that he hadn't embraced in the past and he sees the value there and he sees it accelerating and in a way he's just not going as far as some other people and you know i think the book ends with some people who who take a a very different view and who do think a agi is around the corner and it states them very explicitly right and you know i i think it's i think it's about showing where all these people are coming from and letting the reader make their own decision about what's really going to happen it's kind of interesting i mean one of the things that struck me about your book is you're sort of describing in a historical like a story and feeling way something that's like completely in progress like there's a whole bunch of things that happen you know right after your book stops right like you know timnit and jeff dean and all the stuff that happened at google and then i was actually thinking it's funny like joshua i think in your book he's not really doing a lot of commercial stuff in contrast to some of the other characters but then you know i think element ai you know recently kind of sold and that was like a bit of a you know controversy if that was a good outcome or a bad outcome and then you know open ai went you became a private company i did you have a sense of like the book needs to stop here or was there other stuff that you kind of wanted to include could there be a sequel to this there could but you know what's interesting about all the stuff you mentioned and i would argue like almost everything in the book it is completely in tune with what happens in the book right now timmy gabriel is a character in the book and the same things happen in the book it's just with a different company right it's with amazon right okay it's not with google then it happens with google yahshua bingio you know he has stayed outside or more outside of the commercial realm than hinton and lacoon but as the book goes into like he dips his toes certainly right in the book you know it's you know it's his partnership with microsoft he's also had one with with ibm and with all these people it's sort of a balancing act and i think that's what the book is about is that you have these very idealistic people whether it's to me or it's joshua or hinton and they all come into contact with these forces that are frankly much bigger than them these corporate forces these government forces and you know when that happens there's going to be conflict and all those conflicts that have come since i finished the book it's all happening in the book as well and you know what you know open ai you know how many times have they gone back and forth you know as far as what they're going to do and what they're not going to do are they a not-for-profit are they a public company do they believe in withholding the technology or sharing it right these things will continue to go back and forth but the the constant is is that clash right of of belief and then you know those corporate forces which are about money and about attention and promotion you know i think that those are the constants and and that's that's why i really believe in the book is because all those things are just going to continue to play out in the years to come i mean one thing that i really was curious to ask you about is you know you kind of set up these kind of dichotomies that you personify right and like sort of gary marcus versus on the maybe or like elon musk versus zuckerberg should probably see what those are people listening to probably guess what the you know what the dichotomies are here sure i i like i was curious where do you land on the stuff now that you've kind of talked to everybody like do you feel like for example like do you feel like we are you know sort of like overstating the future progress of ai it sort of seems like if you take a historical view like you're taking it seems to me like ml has just kind of made this sort of steady incremental progress and people keep moving the goal posts of like what you know what it means to do agi like first you have to win at chess and then you have to let it go and you know you know then it's like you have to pass the turing test but then that doesn't even you know that's not even hard enough and so like when i take for me when i take a historical view i sort of imagine steady progress extending out into the future then when for me when i look at these algorithms it sure seems like a stretch that they turn into you know kind of agi just with with more compute so i actually don't even know where i land but i'm curious where where you land on this topic well i i think you're right you you have to look at this historically and that's what the book does is that a lot of the claims that are being made now about agi and this sort of pervading our lives and sort of taking away jobs all that has been around since the 50s right and i showed that in the book and in a way it's just a repeat of that now that said there has been a huge amount of progress over the past 10 years which is what the book really covers right we've had a huge amount of progress what what i really believe firmly in as a journalist particularly as a as a new york times reporter is i feel like what has happened and what is possible in the public consciousness is way out of whack right and a lot of that just has to do with the term artificial intelligence you know which has been thrown around so much over the past 10 years that alone gives people the false a false impression right about what is happening and what will happen and then you know frankly most people write about this stuff they you know for whatever reason they don't really understand what's going on and and they exaggerate and maybe they exaggerate consciously maybe they exaggerate unconsciously maybe they don't know that they're exaggerating but if you sit down and you read most of the stuff is written you have a false impression and what that that is one thing that i really want to at least in my small corner of the universe try to correct and show people what is really happening right and the fact the matter is none of us knows what the future is and you know as much as you know someone who really believes in agi might get on this this you know call with us and you know get angry at me for not saying agi is around the corner the reality and i think the book shows this is that none of us know what the future holds and when it comes to agi it's an argument it's a it's a religious argument right and i show that in the book people with the same experience the same knowledge the same respect across the industry really disagree on this but go ahead no no it's funny i guess like one thing that's sort of it's almost in the water so i don't think to question it because i kind of swim in it right is why do you think it becomes such a religious argument like why do you think people feel so passionately frustrated that you know other people don't agree with them on this on this particular topic of like is aji possible or coming or coming soon well i think that people are just coming from from a very different place when they start talking about these things and one of the things you realize about silicon valley is if you're going to be successful you've got to really believe in what you're doing right that's again either consciously or unconsciously that's how you attract the money that's how you attract the talent that's how you get these things to snowball okay whether you're building you know a tiny little app that does something simple or you're trying to build agi so what has happened is people have taken a rule book that has worked in silicon valley for certain things let's say facebook right and they're applying it to this notion that they can build a machine that can do anything the human brain can do so in their mind they're just doing what everybody else has done right but agi is different than facebook right that is a goal that is far far bigger and so you know in their world they're just doing what everyone around them is doing has done for the past you know however many decades in silicon valley but for someone else like they're taking a huge step and they they just do not see that right how can you extrapolate from from you know a machine that can play go to a machine that can do any anything human brain can do and if you ask people to to describe to you how that's going to happen right that's at very least that's hard for them to do right describe how that's going to happen right you know the path that they see is a path that they painted very broad strokes and you know saying i can build a facebook you know there's a path there to building a social networking app we know how to do that we don't know how to do this and we don't know how to build a self-driving car right that alone is an astronomically difficult project that we don't quite know how to complete yet but people still talk about it in terms like it's already there and on one level you see why they do that but on another level right it misleads the public it misleads people about what's going to happen soon so i guess i sense that one opinion that you kind of hold is that there are a lot of over-inflated claims and therefore the public feels like the public does not have a good sense of what's possible and not possible that that'll that at the very least is true right you know who knows you know tomorrow we may have a new technology that really blows us out of the water but what we've seen over the past um 10 years with this are repeated over inflated claims just in the sense of they don't give think about your mother right or my mother when when they read stories even in the new york times over the past a few years where they assume that tomorrow we're going to have cars that can drive by themselves all over the place right they can't help but have that assumption because that's the way it's written about and journalists write about that way because people like elon musk and so many others just say it's around the corner and they take them at face value right so i think that's really where the problem is that you're you're misleading the general public and and i do think that that's a real real problem right in a at a time when our society is grappling with what is true and what is not let's let's make more of an effort to to say what is actually possible now and show people what the reality is now and and try to do that in a way that's separate from what might come right the reality is now is that self-driving cars aren't up to the task but it's kind of interesting you say that because well i wonder if maybe journalists are at fault then because like certainly elon musk has a pattern of over stated claims but i i think he might be a little bit of an outlier i mean you would know better than me but i feel like when i talk to ml researchers they tend to be fairly understated or almost like maybe a little too reticent in their claims and maybe the ones that rise to the top aren't like that but you know we've done like 30 40 interviews on this here and i almost feel like i'm trying to push people to you know like extrapolate what you're doing like it seems like a big deal i don't know like when you talk to like jeff hinton or actually let's go way back right in your book you talk about like rosenblatt and then the you know the new york times i think or yeah it seems like a lot of journals the journalists kind of write about what he's doing saying it's going to get consciousness soon when he's basically like you know doing like you know like a perceptron without even like a second layer exactly so what happened there like do you think do you think rosenblatt has a responsibility to communicate better what's going on like was he making over-inflated claims at that time well yeah he clearly was right i mean you know he's telling these reporters that you know we're going to have systems that can walk and talk and and recreate themselves and you know somehow venture into space right like and so the reporters are just going to report that right right okay and in a lot of ways it's not that different now and you talk about elon musk being an outlier that's true and it's not like again you talk about ml researchers that is not a monolithic group like that's the other thing i want to show people is that even the new york times has written stories ai experts say x right well ai experts that's not you know one group it's you know it's this like spectrum of people and if you you got to remember like deep mind and open ai are founded on the notion that they are going to build agi and there are people at those companies who really really believe that and they're at the top of those companies and they may not be as cavalier as elon musk they may not have the megaphone that he has but they really believe that and those are important companies right they have a lot of serious research talent particularly deepmind has had some really important breakthroughs you know just recently the cast contest breakthrough that that's really important research that in in some ways is separate from this you know notion they're going after agi so these are important important labs that are founded on this this belief right and you know i i've known demus hasabis you know the co-founder of of deepmind for a long time now and whatever you think about that belief of agi you got to take that guy seriously right he you know he has a track record he is he is a a serious serious person and you may have a problem with a lot of the stuff he has done or said but you have to listen to him right and i mean similarly the work coming out of open ai it'd be hard to argue it's not super impressive like you know so i feel like some people claim that it's a little so there's a little bit of publicity stunt but you know you know like you talk about the robotic hand manipulating a rubik's cube and that's really impressive and maybe the rubik's cube makes it more fun but you know i i still think it's an amazing breakdown i agree i completely agree it's both right it is super impressive science on the one hand and it's a stunt it's both and and me as a as a new york times reporter as a book author my job is to show you that it is both right and give you a really real sense of what's going on there it's very easy to see that hand right this five-fingered robotic hand solve um a rubik's cube and think agi is gonna happen tomorrow right if you're if you're not educated in the field it is super easy to think that and so my job is to say there is an advance here right and you can see it but like there there are some chinks in the armor and and the other thing that i've seen is that not even everyone at open ai is aware of the chinks in the armor right and that that hand while the result is super impressive there are some caveats there that show you even the science isn't quite where you think it might be you know let alone sort of the stunty nature of it right you know my point over and over again is that these things are complicated i guess you know maybe this is inserting myself into it it's a your story but you know i was kind of there throughout it and i couldn't help but i keep having this thought you know i was at the stanford ai lab in like 2003 2004 at the sort of like nadir of interest in neural nets and you talk about this in your book and and you know i felt like the zeitgeist there was kind of like ah these neural nets are kind of like the name is too good like you know we use support vector machines not like neural nets that's like you know that's not serious and it's like these people just start trying to like hype these things and yeah they sort of work but they're like kind of tweaked to the point where they're like overfitting and serious people wouldn't you know wouldn't make a system called a neural net and it's been kind of interesting to watch it turn out that the neural net strategy actually really works you know like the perceptron is like the base thing that that now is like you know used everywhere and so i actually kind of feel like maybe the the folks i was working with at that time you know weren't dreaming enough like it's you know i think it's great that angering you know kind of you know when he saw it working really you know invested into it but i you know i remember like you talk about some stories about like the skepticism of the progress of neural nets and i like vividly remember that just like everyone says they have a better algorithm especially neural nets but then but then they were right and i kind of wonder if you feel like there's any lessons to that because it seems so remarkable that something would get all this attention and then sort of like you know be thought of as bad and then kind of come back as like the working technology like i wonder if there's other technologies out there that that have followed that same path well i think it's i mean it's an incredible story right i mean that's like it's amazing that some people kept working on that this stuff and that that you know again is at the heart of this book and it's something that i have always really um been amazed by and impressed by is is someone who keeps working on something even in the face of everyone telling them it's not it's not going to work right that is the basis for any good story and that certainly happened here and it will keep happening and in fact you know in some ways you've already come full circle where you have this sort of the let's call them the gary marcus crowd you know you know who are saying the same things like you know neural nets don't do everything these guys say they're going to do they're limited and you know and and so in a way they're still fighting the the the same battle right but but you're right there are other technologies that will come along have already come along that people are skeptical of that you know that are going to work in the face of that and and it takes that right it it takes that belief and and that determination and and just sort of years and years of hard work to make this stuff you know do what it's what it's ultimately going to do it seems like a lot of the characters in your book i was kind of struck by i don't have like a good stat in this so i could be wrong but it seemed like a lot of them didn't come from a computer science background like it's like a remarkable number kind of came from biology and neuroscience and and things like that do you have any any thoughts on that i i agree and that's another thing that i'm fascinated by is ai is a weird field right it's it's this combination of various fields and it's always been like this right since the 50s when the term was coined it's this blend of computer science and neuroscience and psychology that has always been the case and it continues to be the case and and this is embodied by again my main character jeff hinton right who he is he is someone who didn't come at this from the computer science angle and he's still like one of the running things in the book is that he loves to downplay his skills as a as both a computer scientist and a mathematician and you know he doesn't think of himself as either he he you know he comes at it from that direction and and sort of gives this what is really just math you know a perspective that you wouldn't necessarily um expect it to have and that bothers some people and and some people don't understand that perspective that he gives it but that you know that is how he thinks and it has a real influence not only on on how this field has progressed but it does have an influence on how people perceive it right people don't understand when he and others as much as they explain it and re-explain it they don't understand them calling a neural network you know a facsimile of of the human brain they don't understand that's just a metaphor in some ways right but that's but that's part of the way this field works well i guess from a historical lens maybe the the takeaway is that you know being an outsider is an advantage in in some ways absolutely absolutely and and that's that's sort of the story of silicon valley as well right but that doesn't mean that just because you're an outsider that you're going to be right you know not not every outsider is right some are and and and some aren't and i think that's the story of this book as well probably everyone else is going to ask you this question but i felt like i had to ask it do you have any kind of like fun stories that you couldn't fit into the book because they didn't quite fit or any any good anecdotes in all the research you're doing that's a good question let me let me think that over most of it's in there to tell you the truth i mean like all the good stuff some of it is just unbelievable and and it took a a long time to get and it and it's you know once you have it from one person you got to get it from another so there were a lot of things right including like the lead story in the book and the prologue like that i wasn't sure i was going to be able to get in there and and thank goodness i did talking about the auction of the the company dnn exactly and in particular the price right that was one of the hardest hardest facts to nail down you know i have to tell you that's the only anecdote in the book i don't totally believe it was the one where it just maybe it's because it's actually true it just feels unbelievable it is 100 true and including wait to the part that i felt like it might have felt that way to the people involved but it's hard to believe it actually happened like this is they like literally got google and baidu to like bid at a particular time like they're running like a sotheby auction or something is it are you sure that's true that's amazing no but it's true because i love i've talked to i can't tell you the number of people i talk to who are involved in that like directly involved in that it's absolutely true and it's i guess that's that's how it goes right like the thing that's really true is like actually unbelievable exactly and but like so many parts of that story are amazingly you know improbably true because it encapsulates everything right at the very beginning of this movement let's call it a movement you know like the or what is it like the very beginning of this explosion in ai height in neural networks starting to work all the players there you know who would be involved are already there right from china and baidu to google to microsoft to deep deep mind is there right they're all there in this competition that would play out over the next 10 years like i you know and and that's that whole story came to me in bits and pieces right over the course of it was really you know months or maybe even years and as each peach pops into place you're saying this sounds too perfect to be true but you know it's true because it's coming from multiple people and you know and it's verified by multiple people and and all the perspectives kind of come together and some people say well i won't tell you that and then you get it from somebody else and they say okay yes it's true right that's the what's most fun about being a journalist is when you when you get those those nuggets that just show you so much you know about human nature and also just help your story just fit together in ways you never expected i never expected the book to begin with that but it had to begin with that because it's just it's just the greatest story it's a good story and you go back to it a lot and yeah it is a great story i guess one more just thought that i had reading your book is i i hadn't quite had the timeline in my head of like when neural nets started taking off but i feel like one thing that's kind of impressive is i feel like you know elon musk and zuckerberg and and larry page i feel like they noticed that neural nets were working really well before most academics even noticed it like i feel like they they like i was thinking about the timeline i was thinking about when you know and i'm in mlm selling to ml companies for the last 15 years and and i feel like actually they were really early like how did they figure this out it's remarkable isn't it and i think one of the things you can do is contrast the way they reacted and you can you can criticize the way they were at you could say they went too far of course but contrast the way google and facebook reacted to the way microsoft reacted right and microsoft did not jump on it the way that those two other companies did they didn't see it the way that the leaders of those companies did you know part of the narrative there right in my book is that you know jeff hinton was in microsoft's lab doing this stuff with speech and it worked in a way that nobody thought it would work nobody you know in the in the ml community nobody at microsoft and it works and they're all shocked they're all blown away but they don't jump on it the way that google and facebook did that's really really interesting and you you do wonder you know is it about the age of the company is it about the the general area that the company plays in like google had a real need for that speech recognition system that hinton and his students built in a way that microsoft didn't right because it had android it had a place to put it now it was also a company that and this is talking in broad strokes that that would take new technologies and put them into play far faster than microsoft would especially in those days right that's part of it but you know in the end it's a combination of these things right it's the way the leaders think is the way the company is built which in some ways is a reflection of the leader it's about the age of the companies right once these companies get to be a certain size like microsoft it becomes harder to jump on on something but like you see in the book the way that google jumped on it and it's astonishing right you know there's that conversation between larry page and alan eustis you know where he says you you're you got to bet big on this and this is you're right this is before even the ml community at large really understood what was going on and larry page is is telling allen uses to basically bet the farm on it it's astonishing it really is i guess my takeaway is when i see something working i'm going to jump on it but but even then like like you know it's unclear where it's going to go right like you know it works for speech and then it works for images and that imagenet is such a big moment but then people in the ml community are still like is this really going to work with natural language i mean years later they're saying that is this really going to work with natural language and then it does right you know these these large language models a lot google bert gpt-3 you know it really started to work and there was real doubt there and you know it's it's hard to see these things even when you're close to them and and you know we could go on down the line robotics it's not clear even when this stuff works with multiple different areas whether it's going to work with the next one one theme that also comes up in your book of course because we're talking about academics is sort of like who gets credit and who doesn't get credit and where's credit deserved and actually one anecdote did i i never knew that you have in your book despite voice being a pretty good friend of mine is that alex that was originally called wagner is that do i have that right i can't believe he never told me that i feel like if i was him i would it's a great story right you know why do we call it alex day you go you go to the paper and paper doesn't really call it alex now it's like everybody calls it that well the way it worked was and you know this is in the book you know in you know in a much more elegant way but like google has started to build its own version basically and it was voice check who who did it and the and the the way it worked at google was whoever built the thing you named it after them and so that's what they called it and then you know hinton and krashesky and zeus cover show up and they're like why do you call it that right it's krashesky who built the thing so they just start calling it that and that's what propagates right all over the community i think that that's it's a testament to those guys right that you know they're rightfully so in a lot of ways revered in a way they had some capital right but it's also just funny how those those things work in the in the tech community and sometimes those those those things are are corrected so to speak sometimes they're not right well who do you think so is there something that stands out to you as kind of not getting the credit they deserve because most of people that the heroes of your book i think are really really well known at least of people listening to this but do you feel like someone really do people talk about someone when you when you interviewed them that that doesn't show up so it's in such a big way well you know you know i think jurgen schmidt humor is is is the classic example right he's been written out a lot a lot he's written about in my book you know the reality i don't know that he comes across so well in your book interesting okay i don't know i i think you know i what i was gonna say is with all of this stuff it's complicated okay and and let's take let's go well before we get to jurgen let's start let's start with alex net you know the reality is is although alex krishevsky and hinton and ilya sutscover you know did the work on that and really made it happen they are building on the work of john le right they're using a modified version of his algorithm and he's building on the work of so many others everybody's building on everybody else's work and and on some level they all deserve credit right and you know what schmidt huber is saying is you know these guys who work for these very big companies are getting this credit and and i'm not right and you know i re i really like jurgen and and and i and i feel for him at the same time he is out there saying give me credit give me credit right and that's that's part of this too right right some people do that some people let the the credit come to them right and that's going to be viewed in different ways right some people are going to criticize jurgen for saying give me credit give me credit but but you know i know him and and and you can't help but feel for him as well because you know the reason that these others have gotten so much credit in large part is because they had these giant companies behind them right and and you know these companies are good at you know at producing and driving narratives and you know some of the narratives that have been out there aren't necessarily true right there have been you know published stuff a lot of it came from the companies that don't necessarily give the real view of these things and the real view is that you know it's it's more complicated than you think do you think there's a topic in ai that the press should cover more than they do i think it's more about and i guess i'm going back to what i've said before is is the press needs to cover this in a different way right you know and with more skepticism i guess with more skepticism and and it's look it is hard like again we're talking about you got to strike a right the right balance between you know showing people what's really going on but not going too deep in the weeds like you don't want to lose people and and and that's that's a very hard thing to do but you know when it comes to topics what i will say is that you know a lot of people have written about this this clash at google between timmy and and the company you know she's saying that she was fired and some people at google saying that wasn't the case and and you know in a way it's you know it it's it's a very specific argument but i i think this is really representative of a much larger clash that is is going to have to happen in this field right these language models that are being built these giant you know gpt-3 style language models they are inherently biased right that is just that is just a fact because human language is biased and these things train on this enormous amount of text they're biased and and and they spew hate speech and other toxic material that's just that's just the reality and that's what tim knee and others were saying in the paper that was at issue at google that battle is going to if these models are going to have to co if those mods are going to continue to progress and they really get out into the world that battle is going to happen it's going to have to happen in on a much larger scale at so many different companies right and what's the battle like what are what are the two like visions of the future well on the one hand you have a company like microsoft who put out you know a much simpler conversational bot years ago now called tay right yeah of course i remember that yeah it was rules based for the most part chat bot and it started spewing hate speech and it created this huge you know backlash and they and they took it away okay microsoft ostensibly you know is going to put gpt-3 out in tandem with open open ai that is a clash waiting to happen right microsoft's got to deal with the fact that these things are biased and that's going to offend a lot of people right how do you deal with that that's an open question it's an open question for microsoft for google for facebook for open ai on the one hand you have science really progressing and doing amazing things but you have this problem it's a problem for a lot of people right and some people don't see it as a problem they just think we need to release this stuff and you know get over you know your issues with the bias and the hate speech and but a lot of people think it's a real problem and and to the extent where you know that clash is going to have to happen if those models are going to continue to progress and to get out in the world right you got to find a way to deal with it whether it's technically or or by other means right and you know that's why i think that that situation at google is so important because it represents something much larger that's going on here and it's something that that the press is going to have to look at as well as all these companies okay one one more question why is it so easy to demo a thing that's evocative and so hard to turn that into a complete product that we engage with every day i think it's you know it's just about aligning the technology with the need okay that open ai rubik's cube hand right that is not a line with any need right we don't need that the trick is is finding you know where there's real gain and applying it right and i think that's where people you know often sort of miss the point right and they and you know these neural networks have worked and worked really well in particular areas right they don't work well in other areas you know there's all this hype around ai and sort of remaking how your business operates that sort of thing but that's something different right it's you know there's not always an alignment there's an alignment with that deep mind result right that is something that is a real need and they're going after it and and in one sense you know they solved it there's still a lot of work to be done but that's what we're talking about protein folding protein folding right the cast contest right that's something that the world needs and they're going after you know gpt3 it's not hard to be impressed by it but it's really hard to see where that's going to have you know the the practical application when you find where it works you know becomes much easier to show people right you know i think the difficulty is often just sort of a misalignment if that makes sense yeah no that totally makes sense all right well i think it's a good note to end on thank you so much that was that was a lot of fun thanks for answering all my questions thank you uh glad to do it and uh um really good talking to you as well yeah real pleasure thanks for listening to another episode of greeting descent doing these interviews are a lot of fun and it's especially fun for me when i can actually hear from the people that are listening to these episodes so if you wouldn't mind leaving a comment and telling me what you think or starting a conversation that would make me inspired to do more of these episodes and also if you wouldn't mind liking and subscribing i'd appreciate that a lot

2021-03-21 11:35

Other news