Will ChatGPT take our jobs?
- Well, I can tell you, you know, years ago if you had asked me, I would've said, well, I think we're years away from having something useful in term of bot. And we all have this experience, you know, trying a bot and say, oh gosh, you know? - Yeah. - That's not super useful. I must say that what was done, you know, after GPT-3 which was another layer of learning that made it, you know, very useful and, and yeah, it's gonna change the way we do search, but it's still a search engine and it's just an engine that does compute probabilities to find the next word and it does a great job. And as shocking as it might sound, we haven't equipped the internet with the ability to learn. And you need to think very hard on what you're trying to achieve. I can tell you something, if you don't repeat it, of course, right? - Oh, no, no, no.
Well, we're not filming, so don't worry. - Oh, okay, okay. (upbeat music) - Everyone, David Bombal coming to you from Cisco Live again. Got a really exciting guest with me today, JP welcome. - Thank you, David. I'm glad to be with you today as well.
- Yeah, it's great. I mean, we are just talking offline and then I stopped you, chat GPT, it's gonna take our jobs away. - No, I don't think so, David, I think that it's a tool. - Yeah. - Right? And when you think about it, you know, the technology is not that new, but what open AI managed to do is to make it very useful, and for me, this is just a tool.
It's true that the tool will be impacting, and it's exactly like, you know, doctors like radiologists, for example, right? So we have new tools now. - Yeah. - They can do a better job, but is it going to replace doctors? No, I don't think so. - So, I mean, I've heard, like, some people say, "This is crazy, it's gonna take all our jobs away." And then other people say, "It's not that good." You were saying offline, you've got people on your team that are mathematicians and other people.
So, you know, what did your team think about it and what did you think about it? - Well, I can tell you, you know, years ago if you had asked me, I would've said, well, I think we're years away from having something useful in term of bot. And we all have this experience, you know, trying a bot and say, - "Oh gosh," you know, - Yeah. - "That's not super useful."
I must say that what was done, you know, after GPT-3, which was another layer of learning that made it, you know, very useful. And yeah, it's gonna change the way we do search, but it's still a search engine and just an engine that does compute probabilities to find the next word and it does a great job. Now when I hear people say, "Look, are they sentient? You know, of course not, they're not sentient in any way, but it doesn't mean that the tool is not useful. So yeah, the guys in my team were pretty impressed as well, as I was.
But is it going to take our job? No way. We have a lot of stuff to do that machine can't do. - That's great to hear. I mean, so it's not the Terminator, it's not gonna kill us, it's not alive. - [JP] No, not at all. - And it's not gonna take our jobs away, but it could be useful as part of our job.
- Yeah, it could be useful, it's gonna make our job even more interesting because we can focus on, you know, what is more interesting as well. So I think this is just amazing and we need to progress and technology will never stop. - So you've been at Cisco a long time, you've done many things. You gotta tell us patents, you've worked on BGP, MPLS, and then you're now working, well for the last few years you'll have to, you know, give us the details, AI machine learning with Cisco, right? - Yeah, absolutely. You know, I joined Cisco, oh gosh, some time ago. - [David] Yeah.
- We don't wanna think too much about this, but 25 years ago to be more precise. I worked on MPLS indeed, you know, for many years with service providers. Then I led the IoT, which was a lot of fun, then I moved to security and then wifi.
And for the past decade, you know, I've been focusing with my team on machine learning artificial intelligence, and some people may not know, but we've been working very hard on ML/AI and I think we have an amazing technology, you know, and for many products, by the way. So I can tell you more if you want. - Yeah, I wanna hear about it. So, I mean, because the, the concern for network engineers, I think, or a lot of people is, "Okay, my job's gonna be taken away." You've said, no, that's very unlikely. But how's machine learning, how's what you doing changing the way that we run networks or, you know, operate networks, troubleshoot, et cetera, et cetera? - Yeah, I'm going to tell you something that might be a bit shocking.
You don't mind? - Yeah, no go for it. The more shocking, the better. - Oh, really? - Go for it. - Okay.
But I'll tell you, you know, I don't have a passion for technology. - Oh, wow. Okay, that is shocking. - Yeah. You know why, I have a passion for solving problems. - Yeah, okay. - And the technology is a side effect of it, if you will.
- Yeah. - So you're right that, you know, I got a chance to have hundreds of patents, but my goal has never been to invent new technology, it has been to find the best technology to solve a problem. And I think that makes, you know, my job absolutely exciting because I spend a lot of time with customers. I'm trying to find what will be the most interesting problem to solve in a few years with vision, and I'll tell you why in AI in a minute. - Yeah.
- And that's how you come up with your technology. So the way it started, it has been an amazing journey, you know, almost 12 years when you think about it. And we started with the internet of things. - Yeah. - And at the time, you know,
we had to find optimal routing at the edge of a network, and we didn't have the chance to push all of the data in the cloud. You remember, you know, at the time there was no cloud. - Yeah. - Pretty much.
- We are old, yeah. - Yeah, yeah. No, stop saying that. - Sorry, sorry. - But anyway, we had to do some local processing. - Yeah. - And machine learning
was a very attractive technology at the time, very new for networking. So we started with that and it was pretty amazing. We managed to do optimal routing, then we worked on DDoS attack detection at the edge of a network. And then, you know, security was top of mind, still top of mind for all of us. And we started to say, Hey, can we learn on-premise at the edge of a network and use machine learning to detect zero-day attacks, and it was a big success and we made it a product.
And so in 2016 we thought, okay, what's next? And we started to build the first cloud-based solution, ML/AI, and the first use case was wireless. So we started to anonymize the data to run in the cloud and we came up with a bunch of very interesting technologies to solve concrete problem. We managed to reduce the noise to make it easy to troubleshoot and we continued the journey, and then we worked on security again. You know, for example, when you have an iPhone and you want to detect if it is a spoofing attack- - Yeah. - For example.
Right, how do you do that? Well, we used to use static rules and thresholds, and of course you can't do that at scale. So we started to train a model that will recognize the behavior using net flow. And it was again, a success in 2009/19, we started to work on predictive networks, which is a brand new product and technology we've been working on, but you have to tell me if you want to know more. - Yeah no, of course.
I mean, you said you were using NetFlow, so was it supervised learning, unsupervised learning? Just take us, like, take us through the journey. - Yeah, yeah. - Because I think the thing is how does AI recognize this stuff? - Yeah, so it's actually fairly simple to explain. And you are a trainer, so you probably do a way better job than me. - No, I doubt that, no I doubt that. - I think you do.
But it's fairly simple to explain. So let's take a few use cases, right? - Yeah. - So the first one would be about wifi.
So years ago, what we used to do was to send a bunch of telemetry data and you would run with multiple rules and with static thresholds. And then you would just say, if X is greater than Y, than do this, right. That's how we used to manage wifi networks. It depends, right? It depends on the noise in the building.
You know, the quality of a signal, the number of users, right. So when you do that, you can continue to edit rules at which point you start to stop scaling. - Yeah.
- Or you can train the model and say to the model, your job is to learn what is normal, this is abnormal. And you mentioned about supervised learning could be unsupervised learning as well. - Okay. - So for wifi,
we started to predict to, you know, compute what should be the normal throughput, for example, for users. - Yeah. - And that was done, you know, fully automatically. Now if you look at spoofing attack detection, the one that I was mentioning, you would just show multiple examples and say this is the behavior of a camera, for example right.
So we take NetFlow, with the NetFlow record, you keep training and say, this is a camera, this is a camera, this is not a camera. And so the job- - You would tell it that or it will figure it out? - Yeah you absolutely. - Okay. - All right, absolutely.
So that would be supervised learning in this case for example. - Yeah. - And so you would keep training. And so at some point, you know, the classifier would look at the behavior of any device that claims to be a camera. - Okay.
- And either, you know, the classifier says, "Well, it's unlikely to be the behavior of a camera, "that might be a spoofing attacking." So that's one example. The next one was even more interesting.
So I can't resist to tell you more about predictive. You are ready? - Yeah, yeah, go for it. Go for it. - Okay, you sure? - Of course.
Yeah, that's why you here, that's why you here. - Oh okay, great. So predictive. So predictive was that we are in 2019. And so when you look at the internet, as much as you and I love the internet, it has been reactive for 35 years.
- Yeah. - So upon detecting a link failure, it would take minutes, 20 years ago, and then seconds, and then milliseconds, with then get MPLST fast reroute, IP fast reroute and stuff like that, but you still react. And as shocking as it might sound, we haven't equipped the internet with the ability to learn. - That's true, yeah, yeah. - And so we thought, well, we've been developing products ML/AI for almost a decade. Is that a way to start learning so that we can predict issues, and if you can predict issues, guess what? You can avoid issues. - Yeah.
- And you can increase the SLA. So I'm sure you're gonna ask me the question, which is- - No, go for it, if you're gonna preempt it, predictive networks, here we go. - Yeah. But, and so when we started, a lot of skepticism around us. - Yeah, I can imagine, yeah. - A lot of resistance as well.
- Of course. - Which is always a good sign when you're trying to innovate, right? If there's no resistance, you're not disrupting. You know, one of the mistakes we did years ago, and I strongly believe in the ability to learn when you make mistake, that's how the human brain works, right? Was to jump on the model. - Jump on the model. - Yeah. - Okay.
- So you would just take a ton of data. - Yeah. - And the first thing you would do is to train the machine learning model. - Yeah. - And jump on it
and refine the model and stuff like that. - I see what you're saying, like train it. - Absolutely, and of course you don't want to do that.
The first thing you need to do is to look at the telemetry, the raw data, and do a little bit more of data science as opposed to machine learning modeling, if you will, and try to see if there's a little bit of a signal in the data that may be used by a model to interpret it. And so we spent about one year looking at, I think it was 2000 service providers, millions of path across the internet and say, "Can you find some signal "that will allow us to predict issues before they happen?" You may say, "David, hey, can you predict a power outage?" And the answer is, I wish I could, I would be very rich. - You would be. - Oh yeah, for sure. Or fiber cut. - Yeah.
- And well, no, it's very difficult, why? Because, well, we don't have, you know, first of all, when you train the model, you need to have a lot of data and there are not so many fiber cuts and so it's very difficult to predict rare events. So in our case, the sort of issues you can predict is, and you know, we all experience these kind of issue. You take the road at 6:00 PM on the highway, you know there will be a bottleneck. So this kind of seasonal effect.
But that's not the, the only thing that we can predict. So the idea was to say, if I take a massive amount of data, can I start predicting some SLA violation, for example, so that if I have a high accuracy when I predict, then I can start avoiding this kind of issues, if you will. So we are not predicting fiber cut, no power outage.
Still, we can predict a very big chunk of issues before they happen. And if for the things we cannot predict, we fall back to reactive. That's the story. - When you say you didn't want to jump on the AI, was that because we as humans, especially engineers perhaps, we want to grab control and try 'cause we know better or we think we know better, you just had to let it learn, is that right? - Yes, you know, I'll tell you why, because it's not a hard to do a prototype.
It's actually fairly simple. Now you can use a bunch of libraries, you know, TensorFlow and sci-kit learn and all these things. - Yeah. - And so taking some data and jumping on the modeling is very easy to do. But it's a mistake because if you do that without looking at the telemetry first, without even thinking of the model, but just look at the telemetry, you will find out first of all that most of your time will be spent on the quality of the data, which is the major issue. We spend about 70% of our time only on the telemetry.
- Yeah. - And you want to remove the noise and all of these things, and you need to think very hard on what you're trying to achieve. I can tell you something, if you don't repeat it of course, right? - Oh no, no, no. Well, we're not filming, so don't worry. - Oh, okay, okay.
All right, so I'll tell you that. You know, one of the biggest challenge, and that's a question I've got all the time, is are you guys coming up with your own algorithm? And say, of course, sometimes we do that. And so then people say, "Huh, so it's not very hard to do, right? "You're just using some libraries and you just, right, "auto learning it's called." And I'm like, "Well, not quite." So first of all- - It's only you've been working in it for 10 years. - Yeah, exactly.
- Just pulling some Python libraries. - Exactly, exactly right. And so what are the main challenges? Well, the first one is look at the telemetry.
And I keep repeating that, sorry about that. But, so if you guys start in the machine learning AI field, you know, look at data science, quality of telemetry, because if you have a poor quality of data in the first place, people talk all the time about the volume of data. The more data, the better, which is not always true, by the way. It's often true, but not always. But you look at the quality of a telemetry. And you know what is the second trap? This is the metric that you're trying to optimize.
- Okay. - Because- - It's like a threshold, right? - Well, the objective metric, you know, the performance metric for example. - Like is there a problem or isn't there a problem? - How do you assess the efficacy of your algorithm? - Yeah, okay. - And that's very hard because with many algorithm, you're not coming up with one metric, but 10.
And you never know which one to really optimize. Well, you do know when you have some expertise in the field. If you want to do ML/AI for a networking industry, don't think that you can give a telemetry to a data scientist or machine learning engineer and say, "Go and just optimize the model." You need to have deep expertise in networking, first of all.
- Yeah. - So our job are not going away. - Yeah. - One. And second, you need to understand the metric you're trying to optimize, and that's the most difficult task.
Is it a business metric? Is it a performance metric? For example, if you take some algorithm that are doing, you know, prediction for example, are you trying to do optimize what we call the log entropy? Or are you trying to optimize the performance, you know, the percentage of issues you can catch. If you're doing false positive, is it an issue? Is it an issue to do false negative? And I can give you many examples. Look at healthcare. In healthcare, we don't want to do false negative.
- [David] Yeah, yeah. - And when you work in security, you hate force positive because you at some point you don't even pay attention to. - Yeah, yeah, exactly. - So it depends
how you tune your model and what you're trying to achieve. So I could go on and on about these problems, but this is very important. - No, but it's good. I mean, I think the two things I wanna ask you is, number one, when do we see this in Cisco products and what products are those? And then number two, if I'm interested as a network engineer, like this sounds like this is something I need to start learning now. How do I become like you? Or like what's the roadmap to like what should I study so? - Oh yeah, thank you very much for the second question because I'm as passionate as you are about training people.
- Yeah, yeah. - I think this is so important. So, but let's talk about Cisco first if you don't mind real quick. This has been production for a long time. - Okay. - So if you look at, you know, Cisco AI network analytics for wifi, we have thousands of customers in production at very large scale.
If you look at Cisco endpoint analytics, Cisco AI Endpoint analytics, the name is so long, it took me two years to remember it. But we have millions of devices being, you know, monitored. If you at predictive, guess what, we at Cisco Live 2023, we announce it this afternoon. So we have two products, one is is called ONE Site from thousand eyes So that was an internal innovation at Cisco predicting networks. And we have two products using it, one is thousand eyes which the acquisition we've made a few years ago, which is awesome, by the way. When you do internal innovation and you use it for a company that we acquired years ago, it's a great feeling. - Yeah.
- And the second one is SDWAN Viptela Analytics. And we're launching two products, you know, next month in GA. Now let's go back to the other question about how do you start. I think the best answer is, watch your YouTube by the way. - Oh, thanks. - [JP] It's a good way to start, right isn't it? - Yep. - But seriously I do mean it, you have really good videos there.
Then I would go to Coursera. - Yeah. - Which is always a great start, Andrew Ng, you know, he did many, many things like that. Udemy, many, many excellent courses over there. And when you think about it, I don't want to look like a grandpa, but look at that, 20 years ago you had to buy books everywhere. - Yeah, yeah. - Right?
Now you have a ton of information on YouTube and that's the best way to start. But there's a little bit of a trap, which is that don't be stuck with books and anything like that, you need to have hands on, you need to try. There are many data sets now available, publicly available, that you can, can play with and in a matter of a few hours, you can easily have a new account and you start to play with sad sci-kit learn or whatever.
And you need to try an experiment. Now that's the best way to start, I believe, and be hands on, and then join a team like our team, that's the best way to learn. Because the scale is the issue, by the way.
It's not, you know- - It's like you said, prototype's easy. - Yeah, prototype is very easy. And you can do that in a matter of a few days.
If you are passionate about it, it's very easy to do. When you think about scale with millions of devices as we do at Cisco, it's a bit more complicated and you need to join a team that has been doing that for a long time, yeah. - What skills do you need? Is it like Python? Do you need to know pandas? Or is it just like, jump in and just play? - Yeah no, Python is well, one of the best, if not the best language.
There are other languages of course, but yes, you're right, you need to know PyTorch, and you know, pandas is a great one. Spark if you want to go at scale and do parallel computing, these sort of things. So absolutely, you need that.
Don't forget networking. - Yeah, because the problem, I saw this when we, sorry to interrupt, when we moved from like traditional CLI like configuring devices, and then we went to like automation. The problem I always found was, Okay, there was a lot of Python training on stuff that wasn't networking. So I mean, the problem is like, how do I learn AI in a networking context? Any suggestions? - Yeah, no that's a very good question.
I think that you need to do both in parallel, I don't think there's any workaround. There are a lot of great specialization like CCNA and up to CCIE. - So do like normal networking, but then do also separate track like learn AI.
- Absolutely, and apply your knowledge, networking knowledge, you know, and and use ML/AI to do that. Start with a interesting use case. And I keep saying that, I'm sorry to, and there are many, many use cases, I think we're just scratching the surface. I told you about, you know, security, security has been using ML/AI for a long time.
- Yeah. - Collab. You are using ML/AI, but networking as well. So I mentioned wifi, LAN switching, you know, WAN I was mentioning about predictive. So there are many, many use cases. And I think it's fairly simple to take some dataset and with NetFlow, look at NetFlow, right? Yeah, with the main example logs and you start to apply to very simple use case, don't start with something too difficult.
Another advice that I would take is if you do follow the ML/AI, you know, research, these guys will tend to jump from one thing to another. - Yeah, yeah. - And so the risk is to take the most sexy and latest algorithm, don't do that.
I mean start with simple stuff like trees, decision trees, you can do amazing stuff with decision trees, and sample techniques, and GPT and all these things. And if you want to do a little bit of deep learning, do that, but don't start with the latest and greatest transformers, for example, and go incrementally. - You know, the problem is, I might know networking, like learning python for a lot of guys is really tough. - [JP] Yeah, it is.
- And now this is like just another step of like feels like craziness that I have to learn. Is it really difficult? - No, no, let me tell you why. Because of course if you want to become a core ML engineer- - Yeah. - You need to have a deep background on statistics, obviously, in maths. - I was gonna say in maths. Yeah, yeah. - But as a practitioner- - Okay, - You don't need to do to know that.
You know, backprop for example, if you want to go into all the details of backprop, you need to know a little bit what the derivative is and how to compute these things, but why would you need to know what is under the cover? - Yeah. - You don't need that. You need to understand a bit how it works and how to interpret the results, and keep your networking expertise and you'll go very far. - So do you see, some people have been saying this like as a network engineer in the old, old days, we used to have to try and memorize everything perhaps, but like then you start using Google, it's like, don't try and memorize everything, just use Google, know how to use Google. Do you think it'll go the same way, people are talking about we'll become AI engineers, like we'll use AI more and more to help with our jobs.
- I think we will. Does that mean that we gonna have to remember less? I don't think so. - Okay. - I think we won't have to remember non-interesting stuff. - Yeah, so that's a difference rather than doing all the mundane stuff- - Exactly. - We're gonna do like the clever stuff if you like.
- I think so. And I think that our scope is going to be a bit broader. - Yeah. - So instead of being pure networking, you know, is going to be a mix between networking, programmation, you know, as you said, Python, these sort things, and a little bit of machine learning AI but, you know, again, for me ML/AI is a tool, it's nothing more than a tool, but a very useful tool. - So if I am 18, and I always use that as an example, but let's say I'm maybe older but I want to transition into this.
CCNA? - Yeah. - And maybe DevNet CC and like DevNet associate. - Absolutely, absolutely, that's a very good path.
On the side, you know, you do a little bit of programmation, you know- - Andrew's course on Cousera. - Yeah, or Udemy or whatever, you know, some other courses and you try to be very much hands-on. If you are interested in the math, well why not? I mean if you want to understand deeper, you can do that. You know, my son is 17, he's at EPFL, he's just starting his educational path if you will at EPFL, and I keep saying, "Look, it's good you love math, that's great, "but you need to to be a little bit hands-on," and try and still to convince him to do networking career, but I'm not there yet. Maybe he heard too much of internet- - Exactly. - But yeah, it's a combination.
I think this is very important to not be on just one technology, but you know what, David, you've been in that field for a long time as well. If you want to have a broad scope, would you, you know, be stuck with one area like wifi? No, no. - No, no. - Exactly the same thing. So you would do some networking and in addition to that, you know, Python, you know, you need to know more about the cloud, it's unavoidable. - Yeah. - Especially with machine learning. You can do on-prem, but there are so many tools that you can use, why not using these tools, and on the side you start doing some training on ML/AI.
But my fear is always that some people when they start with ML/AI, they think, oh it's not funny, you know, there's too much math and stuff like that. - Exactly, 'cause you get confused with the maths. But it's the same with programming, 'cause you might get like- - Exactly. - Programming is not for network people, but it's a different type of programming. - That's exactly right.
You're absolutely spot on. And again, you don't need to know what is really deep inside the algorithm site. The one thing that I would say is, look at statistics. - Okay. - Because there are many people, you know, they go for an interview and we give them some basic stuff to do like a your percentile and they say, "Percentile?" - Yeah. - Okay.
So basic stuff, you know, it's not rocket science, but at least you need to know a little bit about statistical tests and stuff like that, but it's not rocket science. So still, I'm super optimistic about the future. Seriously, I'm very optimistic because it has never been so easy, seriously, to learn new stuff and I think that we can go even further in networking by using this kind of tools.
- Is it too late for someone to get into this? If I'm a bit older or, you know, has the industry already moved on? - Ah, not at all David, I think this is quite the opposite. And you can be 40, 50, or 60 years old. If you are curious and you want to deep dive, years ago it was the case, you know, years ago we did not have really the tools and you have to understand the deeply the mechanics and all these things, now there are many tools now. As long as you understand that you, and I think this is very important for our career as well, to have a broad scope and it's no longer about the specific technology, but if you have all of your knowledge about networking, and you have some programming skills, and you don't need to know so much in there, but still a bit of programming, you understand a bit of the statistics, there are many tools that will help you. And I think we have a bright future, I think that's going to be even more exciting than the past 20 years so I'm very, very optimistic. - JP thanks so much for sharing and thanks for the encouraging words because it's like people are worried, so thanks for sharing. - Yeah, we should not,
I think we should be excited instead of being worried and I really mean it. - Brilliant, thanks JP, cheers. - Thank you, cheers. (gentle music)
2023-02-23 13:36
