RustFest Barcelona - Sebastian Fernandez & Ryan Levick: R-Evolution: A Story of Rust Adoption
All. Right so, we're we're both very happy to be here come on a little bit closer so you can see it all right, we're, both very happy to be here to talk to all of you today. We. Are. Going to be talking about a story, of rust. Adoption, at the company that we work at and. This. Has really been an evolutionary, process and a revolutionary, one evolutionary. And the fact that it is taking a very very long time but that's to be expected. And. Revolutionary. And the fact that this represents, for us really. A new. Opportunity and, one that does not come along very, often. We. Are a company that normally, invents, programming, languages, and if, we do it's usually in the order of one per decade that, sees the light of day and. So bringing, in a new programming language is something that doesn't happen every. Few months but really. Every. Ten years or so so, this is a very exciting time for us but. Before we get started talking about that real quick a little bit about ourselves. My. Name is Ryan you, can find me on twitter at a Trion underscore Loic say. Hello. I've. Been riding rust for quite some time now before I joined Microsoft before. 1.0, came out and. I, was the MC, the very first press fest so, it is extremely, exciting, to see so many people here we're, excited about rust. Back. In the day used to be I would mention that I like trust and people would go what's, rust and. That's happening, a lot less often now which is really really, cool. Hola. Barcelona and, my name is Sebastien, I'm. In security. Software engineer it. Means that I work with when it ability some patching software, those kind of things and I'm, we. Know so right. In rut or, six. Years now I actually. Checked yesterday or, how long I've been writing rust and I I found that my first contribution, to the ROK compiler was in 2013. And, it was a tiny, but four lines but it was a best way of blaming rust for me so, yeah, yeah, yeah. We. Both work for Microsoft opposite. Sides of Microsoft, to be honest we don't even work in the same country. Ryan. Works from Germany, in the US. The developer advocate and I work in the Microsoft. Security Response Center and. Yeah. Not. Yeah. Really, in the same team but we. Teamed. Up for, pushing. For rust adoption, in Microsoft, and. Yeah. So. We are part of it but being that we like Pauline the safe system programming. Language that you might have seen in some of the blog posts that we have been publishing. It. For, sure yeah. So. We're gonna be talking about adoption. Of Russ today at Microsoft, but before we get big. Before, we begin with that the, first thing to talk about is, the problem that we're actually trying, to solve obviously. When we don't want to adopt. A programming, language just, because it's cool just because we too like to write it as. Much as I wish that were possible you. Don't get very far with that kind of reasoning and. So we're really trying to address a problem, here. And this isn't really a problem that affects us as a company, only it's also a problem that we believe it impacts. The entire industry the entire software industry, and, because. Of this situation we've gotten ourselves into as the software industry that, really impacts, the entire world for. Better or for worse. And. This, is we. Should say a billion. Dollar problem really, impacts, us in an extremely. Deep. Way and I know money is perhaps, not the best way to measure this but you can think of it not only as a problem. That causes us to lose. Money but also causes, everybody. In this room everybody in the world really a, lot. Of headache and pain. And. At. The core of this problem, at the, very very core of this problem is a technology that we've been using in the software industry, that. Has gotten. Us very very far and. Been really, really great at a lot of. But is really starting to show. For. Lack of a better word its age. And. That. Technology. Is C, and C++, C. And C++ are, extremely. Great at writing low-level, systems. They. Use, very. Little resources on the machine. They, are in, fact really, the basis on which we create. Our systems, today. But, the issue with that of course is that they are very. Very unsafe, and when they were developed, did, not really have. Safety. In mind and when we say safety really we mean the ability to write secure. And. Correct. Code with. It. So. I'd, like to introduce a little about, and what, the Security Response Center does and the. Security Response Center is basically, an. Organization, in, Microsoft. Centralizes. The management. Of vulnerabilities. It means that we work with external researchers. Internal. Researchers as well to find vulnerabilities in, the Microsoft. Product. We. Work with the product teams then to fix them and also, try to avoid from. Developers, from repeating. This mistake. We, also work in different mitigations. To. Prevent, the, exploitation of, the owner abilities and, we have all this data about. The.
Vulnerabilities. For. The sake of the argument today. This. Is a binary, classification of, the ordinary. Table I've seen that Peter. Mentioned, the, is. 70% before, so I don't think it requires much, introduction, but. There are around. 70% of the vulnerability, that we patch in our software are related, to memory safety. As. You can see this has been maintaining, time so, the, problem is not getting any better. So. Naturally. The question comes, up then okay this, seems to be a problem that you have vulnerabilities but how, much does this actually. And how much does this actually cost the industry as a whole. Well. This is a very conservative estimate. And. This. Is one. Hundred and fifty thousand dollars per issue. You. Can see this it's a lot of money and this is only for Microsoft. It's, not accounting, for the. Work, and time that our customers, have to spend rebooting. Machines is telling the parties and doing. Everything else that. Every. Tuesday. Okay. So one hundred and fifty thousand, dollars per issue but. Okay, if there's only one issue per year then perhaps that's not too, bad. How, many issues are, we actually talking, about. And. Only. In, 2018. We had four hundred sixty-eight issues you. Can see that this is a lot of money if you made Emma. If. You made a if you made a Matthew and x, times, the amount of money with Perisher. Yeah. This is this, is a lot and. We. Can see the. The. The, issue is not getting any better it's actually getting worse year, over year and. Only. In 2019, we had over. 170. Issues and Annie, there, is not even over yet. Sorry. About the meek. All. Right so the the issue is getting worse over time that's, that's not so good, but. Is, it possible that the, cost could be even, higher than what we're talking about here. This. This, cost is only for, for. When the good people find the issues they reported to us then we patch them and then everyone is happy but. What. If. My. World also finds, the bug, and write an expiry for it. This. Has happened like many many times in history but. There. Are a few occasions that we've got out of hands I. Don't, know if you remember a SQL. Slammer. Back. In 2003, it was, ah a. Worm. That was exploiting, a vulnerability and, SQL, Server then, was, infecting. The Machine and using that machine to infecting other machines. Then. We had configure, that was exploiting, also. Us to. Your surprise buffer. Overflow in one of the RPC. Servers, of Windows. In, fact in the machine and then also, using the machine to distribute, the, virus. To the other machines and most, most, recently, and when. I cry I know if you were affected by it of your, organization. But you probably know more, than one person that was affected. By it what is one, was doing was exploiting. And the, SMB server off of Windows. Then. Was infecting the machine and crypt in all the day all the fights in the disk and. Asking. For I think was hundred, dollars yeah three hundred dollars and to decrypt a device. This. Worm. Distributed, to over 200. Countries only, in the first week. Well. And, to give an, example, the, national. Healthcare system and, PCs, of the of, the UK where. We're affected by this ransom. Release. An estimation that it costed the organisation. That on, that organization, only 96, million dollars. The. Global, estimate for. For. This ransom, was around. 4 billion dollars yes, that's. What you're seeing these nine. Zeros. Some. People will be the estimate even over four eight million dollars so as. You, can see this is a lot a lot of money. Alright, so we have these memory safety vulnerabilities. That are causing lots. Of head ache and pain upwards. Of four billion dollars at a time, that's. Really not a good thing but. How could we actually fix. This issue, and. There's been many, ideas. That have occurred. Over the years some, better than others and we're going to talk about a few of them so. The first bright idea that was had um and, there's an idea that you often see on a particular, orange, website nowadays. Is that, we. Need better programmers, all right we're done thank you know.
This, Please. Stop this, does not work, we. Need better programmers, is not an answer to this, training. Does, help you, can try. To mitigate some issues by making people better at. Writing secure, software so. We're not saying that you should not train people, but, this is absolutely, by no means a way to address this issue you. Can have extremely, highly, talented highly. Trained individuals writing code and they will still make mistakes so, no. Now. That's that, is out of the way that's good what, about another great idea and, this one does, work a lot better. That. Is the need for better analysis. Tools and is something that has been happening over, the years quite often both static and dynamic analysis. Tools that, allow you to look at a program and. Determine if it has vulnerabilities, before it ever reaches, anybody's, machine, to actually run and. This works fairly. Well over. Time and. Is, something that we still, want to actively, invest. In because no matter what happens after. We leave the room today the, world will still continue, to have plenty of unsecure. Code in it people will continue to write and insecure. Technologies, and so we need the ability to take a look at that code and determine, if. It has a vulnerability. Inside. But. These. Tools are not perfect, and we are, convinced, that they can never be perfect so, there's, really only one idea. Left that we can actually use. And. This is really the only way that we can address this issue fully and, that's. The idea that we need to make these issues impossible. To introduce in the first place. And. Over time there, have been there's, actually, been a lot of active, research into, this area how to make these issues, impossible, and. There's a few, things that have been introduced, in software that allowed them to not actually occur. Okay. So I, think. To understand, the issue we have to understand. The role we have to understand the program to find a solution so we. Can classify the memory. Safety vulnerabilities. In, into. Big classes of vulnerabilities and. One. Is related to. Spatial. Memory, safety what. Does it mean that when you have a pointer. Or a buffer. That that's, some type of memory and it. Usually has a size it's not infinite so when, you try to index out of that, buffer. Or pointer. And you. Go to C. Developers called undefined, behavior so. It means that everything, is wrong and it's. Not doing what you are expecting, it to do. For. Fixing, this issue. There. Is a, clear. Way that most languages are half. Now that. Runtime, check when, you allocate memory you subscribe you, can embed, that, information, is in the pointer in the buffer so now you have.
Rich. Pointer, or. Fat, pointer. So. When. You, try. To index, that pointer, now, you, have a random, check that, you. Don't have to do anything if. Is. The indices, outside, the bounds it it, will fail the, the. Memory access and nothing. Bad will happen, and. The. Second. Type of. Say. The memory safety we can talk about this and spatial. Memory safety, this. This, means that, when. You claim memory or when you allocate memory that, memory will be valid only for some, time. Once. It gets freed you shoot them you shouldn't be using that memory but. Sometimes. In the languages you don't clear the references, and then you can still access it and, there. Is an, easy. Way of solving. Des that a lot, of languages use. That's an using. Garbage collection what does it mean that you can clean memory but you cannot free memory manually. You, expect, the runtime. To frida memory and. This. Is, basically. Always. Leaving. The memory they are and then. You will have these. Program. Inside, the runtime that will go and find the memory that's not used anymore and free. But. Of course garbage. Collection does come with its downsides, there. Are a whole class of programs, out there where, a runtime, like, a garbage collect at runtime is not, really acceptable, so. If you're writing an operating, system or perhaps a really low level system, like the database or something like that. More often than not you don't want to pay the penalty of having, a garbage, collection there collector, which comes with performance, penalties and also comes with penalties. For not really knowing when. Certain things will run and. That's just oftentimes, unacceptable. But. We. Believe that there, is a technology out, there that allows us to kind of have our cake and eat it too and spoiler. Alert it's. Rust. Rust. Is not particularly. Exactly. Perfect, in this regard and we're going to talk a little bit about the caveats to this but, rust does allow us to write performant, performant. Systems, programs, in. A safe way and. To. Look into what. Rust actually, gets us and practice, we'll. Take a look at this. So. I. Mean. Right, I was talking before about making, these issues impossible, and. That's. Kind, of a hard thing and, but. What, if we could isolate, what. Unsafe. From. Not. Unsafe. What's. Unsafe from safe and when. We are looking at the C++, code base. We. Find, that, hundred. Percent of the code. Base is, written. In unsafe, super bad you have no way of isolating. What. What's safe from, and safe. For. Example we are talking about. Software. We have to pull to, make thread. Models about the software what does it mean that you, have to. Make. An analysis, of how the software can, be attacked if we are talking about a browser, for example a browser, the thread model is that every, website can, be malicious, if. We are talking about the kernel we, have. To, start. From the ground that each. Program user running in user space will, be also malicious, so that. Has to protect it. Has to protect from. Every. Program and. After. You have this suite. Model for these two. Examples you, come, to the conclusion that almost. All. This, all, the code, is exposed, to the. To. Their malicious. Symbols. When. We have and. When. We look at rust, and. There. Is a clear, boundary of what. Safe, and unsafe. And this. Is a big advantage for, software. Security software engineers, like myself that we have showed it and these, code bases and. We. Made an analysis, of the, most-used great and came. To the conclusion that the, amount. Of lines that were. Unsafe in the, top 200, great, was. Only, 1% and. I. Remember. I was wroth. Kampf watching, and sharing, means Jeremy's. Talk and he, was talking about the rest adoption in Facebook and he was saying that the recessed ratio of, data. Technology technology. Has to overcome to. Be 10 times better and done. Than. The next, technology available and. We. Can see here Rusted not. Over that threshold but. We. Can consider out in this aspect it's a hundred, times better than the next technology available. But. Of course we're a truss fests so we're kind of preaching to the choir here. Most. Mostly, everybody in this room probably already, understands, this you, may not have selected, to use Russ or this property, but you've probably heard about it before and so let's just say for the sake of argument, that, we're. Going to use rust we we, have chosen it we've been able to convince ourselves that, this is a good thing to adopt, what. Does that actual, adoption, process, look. Like and this is really where it, actually begins. Because. While Russ might present a very, interesting opportunity for us as an industry, we. Have a whole ton, of C. And C++ code, written out there and we, can't just snap, our fingers and have it written and rust it's going to be a very very very very, very long time until, that goes.
Away Even if we were all 100%, convinced, that that's the right thing to do so. When. We talk about rust, adoption. The first thing that we should take, a step back at and look at is actually, how do languages, get adopted and particularly. How do they get adopted in existing. Large companies, because. If it's a start-up it's, quite easy you can just choose to use a new technology because you haven't written any code before but. If you haven't a huge, code base that already, has seen C++, in it what. Is the process actually like and, the. Issue with this is the. Following. You. Have costs, and you have benefits, and I don't know if you can see it but down, there it says benefits, but they're very, hard to see the, costs are quite clear you, have the costs of actually introducing the, code inside, of your code base of, introducing. The. Compiler, inside, of of, your build system. These. Are all very clear costs and are actually very easy to reason about how much it will and end up costing your organization, to actually adopt it but. The benefits, of introducing, a new language may, not be so clear if you're introducing. A new language because, it has a great type system, and. You think that that type system will not introduce as. Many. Bugs as the current language you're using well. How do you actually prove that it's extremely. Difficult, to actually measure the, impact of something. Like a more, advanced type system. But. At the end of the day this. Is what makes rusts, more adoptable because. For, rust that picture looks more like this where. The costs are still quite clear but, at least this one benefit, that we've been talking about of memory safety is extremely, clear we. Know that rust will get us much, further along to, the writing memory, safe code and we, know that memory. Unsafe, code leads to billions, and billions and billions of dollars of damage so it's. Quite easy to go to people and say how. About we stop spending billions of dollars on this stuff and they go sounds, good. And of course I've we've only written benefit, here that's. Not to say that rust doesn't have other benefits it's just to say that those are harder to talk about and harder to convince others, in a large organization, to bet. On rust because of them so. Really, when it comes to adopting. It in a large organization, it. Really is enough, to just talk about its memory safety. Properties. In order to convince. Others, that. Is worth adopting. Yes. So maybe. Just maybe, and we. Shouldn't be writing, security, critical software a in, C++. Because. Of the things we were mentioning before so. Now, the debate is and what. Do we use for for, writing the Security's critical, software and this is very. Rough I mean rust. You. Can write performance. Performant. Code. And. Having. All these extra, things that are, related to the memory safety of the language. But. We, cannot just, ditch everything. That we did till now and start. Over with rust. So. What. What. We can do now and, do. We just keep using continue, using what the, technology that we were using before usually, see LastPass or we. Can maybe try and decorate in rust with our exes, cold bases and technologies. This. Means that we have to integrate rust. With with. Our infrastructure, and that's. Not easy because rust. Is not just a language rust. Is a, set of tools that you are bringing. To, your infrastructure, and. This. Especially. For for. Windows people and. Rust. Comes, with this beast that the. Compiler. Back in that rust. Uses that is LLVM. And. VM, has topped. It with great, support for, much, BSD. Than Linux, but. Windows. Support is not really there and. We. One. Example is for example IBM didn't have a linker. For for, Windows minor is still only, a few months ago. Continuing. On this line of bringing, all the, all these rust. Tools to your infrastructure, is. That. We. Have a lot of tools that we have to run in our pipelines. To generate. The binaries, and to check that the binaries are okay if an, example, is that. We. Check all our all. The binaries I will ship to our customers, and for. For performance, that, means analyzing. All, the binaries, in, at. Runtime checking, that they, are doing well and sometimes. Rewriting, those minor is it means. Rearranging. The basic blocks that are, an. Inside, so they are more performant, and more friendly. And. The. Problem that we have with those tools is that those, tools are, used. To. To. Analyze binaries. Produced by ms PC and. It. Obviously you. Cannot just expect that LLVM. Enemies. You see are generating, the, same. Type of binaries, they have different, name, mangling they. Have different, types of symbols and, the arranged information, in the riff way and they, have different heuristics for, generating, the code so, this is a whole new set of tools.
That. We have to adopt. And. Continuing. On. This line of tooling. We have also build system that have been optimized for many. Many years to. Perform. Well and to, comply with a, set of rules, as. We have our. Business and just shut for a second in machine and, theoretically. That you want to include a rot component, inside Windows, and. These. Challenges. Are that when. You when. You want to use Roscoe there and. You. Have. Obviously. Cargo. Which, people. Used to build and. To. Build rust binaries and cargo, is a great tool I mean. Who. Can imagine using rust, without using cargo you. Are losing half of the half of the features. Cargo. Is a great, will. Build tool and a great package management tool but. It. Has to be integrated in a in. An existing the system we cannot just let. Cargo. Manage our whole bin system and obviously. For cargo, there is rust. Code and not. Rust code that is nothing else for for. Cargo and. Convene, on this line we have a. We. Must be able to interoperate with the. Existing, software as as I was mentioning before and, the. Software is usually. Written in C and C++ and. Sometimes. You have a clear, boundary between and is. This. Different. DLS. Maybe one, is written in in. C and the other will be written, in rust in the future. Rust. Fortunately. It's very good at interoperating, with other technologies, because it supports, most. Of the ABI is supported, by C. Also. Another great. Tool that comes with rust it banshan mention. Lest you take a seat header and generated. Rot. Binding for it but, the downside is you. Have now all these unsafe, binding that you have to make, a save, wrapper around it, so, maybe. The idea is that in the future we, can kind. Of guess. If. We. Can if we can make a safe wrapper around, these C, headers. And. The last one is the. It's. Crazy. But and there, have been like many technologies, to make interoperability, language agnostic this. Labor. Is like, common. And, we have T these are well, define an API, standards. Because. What severity then have a well-defined ABI e back then. And. These. Are things that the communicative working, on. We. Analyze most, of these most. Of the things. Available for calm and, we. Decided to to. Also write another an hour comradery that we released, this. Month I think yeah. So. It. We are getting there with interoperability. Then. We. Have also we have problems, that, common. People don't have and, what sorry. Common. People is about, description, body and individual. Developers. Maybe don't, don't, have or a small companies, but the, company's food processes, in place to, prevent errors from, from, happening. It. Means that everything that you, bring into the company have, to. Must. Must comply with a set of rules and one, of them is having a trusted tool chain when you are developing with, rust you. Don't. Know the stop you executing, in. The PC and then after two minutes depending. On your internet connection you. Have rust available, to to, be used. But. When. You are bringing, rot. Into an, urban system, and I like, the windows one you. Just. Cannot pull a binary from the Internet and run it to there. More. Even, even more if these binary is connecting. To the Internet at putting even more binaries, from there so. You, seen rust up there or putting, the rat compiler, directly. Into the build system is not an option it means that we have to build our own tools. And. This is a big challenge because we have to start from a trusted. Source code, and a trusted binary and then build the, trusted binary that we can use in a wearable system. The. Following, one is about, these. Are all examples and. There. Are many more but having. A we have binary, security, policies all the all the binaries, that we ship to our customers, comply, with a set of rules, to. Make to. Make them more secure so. For. Example an only. Miner is that our sheep have. The. SLR, and a few other and memorialization. Mitigations. Enable, in them. An. Example of it is a control. Flow gap and control. Flow here is something to protect a control flow of the program so malicious. Input cannot hijack it and then well do something that the program was not programmed, to do and. This was a mitigation I wasn't available in LLVM. Yeah. Now. We can happily say that we in. A madest, mitigation, and contributed, it back to LLVM and it's, been in master, or, I. Think since last week and. We. Hope we can enable, it in rust in, in. The following months as well so, super, happy about that. Then. We have an, also. The humans, challenge we, we, can call it like that and this is probably the most interesting to me that I used, to deal, with machines. But. Not really with with people. This. Is a first. Period you have is convincing, people of using a new technology, and fortunately. For us and rot, is a very adoptable technology, and but. When you, try. To tell, someone hey maybe we should use rust and because.
It's Memory safe the first thing that they think is okay we, already have seizure for that why why are you suggesting this and. Then. You have to start explaining that Roth doesn't have a garbage collector and it. Doesn't have a runtime and it's. More. Similar to C++, than. Than. Actually, seizure. And. The last point is you, have to be able to train people who might be, used to writing C and C++ for, 5. 10 30, 40. Years and. Be able to get it to show, them, how to write. Rest code that, they might not be able or used, to writing and. The good news about this is that when before. When we've introduced, rust, to seasoned. C++, programmers, they, generally, are able to get it rather. Quickly, because. It kind of just or malaises things that they already have in their head when. People are coming from other backgrounds, it might be a little bit more difficult, but. While, the learning curve is quite steep generally. People get through it and once they're through that learning curve they're. Quite productive which is which is really nice to see but. We've been talking for a while now about the challenges, that we're facing as a company, and okay. That's great we're going to face these challenges but. There are also challenges that, we as an entire Russ community, also have. To face when Russ becomes more and more adopted. And. One of the first things to talk about is the idea of governance. Really. We want to as a community, ensure. That, rust is owned by the community and not by any one particular entity. And as, more and more companies come in and. Want. To use rust there's always the temptation to start, taking control and that's something that we do not want we, want to ensure that that, rust stays, in as a community. Owned project, and directed, by the community, because frankly it's gotten here because of the community and the community is the best one that we'll be able to take it forward from, here on out. And. Really. What this means is that we want to take, the community principles, that. We as a Russ community, have established over time and really established. Them in. More, or less a formal. Way and of, course, there's great work going on already, from the governments working group and. A lot of talk happening, in many directions around, this and. We're. Really excited to see this go even. Further. And. With, that over time Russ will continue to change we just had a sink weight come out yesterday which is really exciting the, language that as it looks today although, it's still compiles, looks. Different than it did and and 1.0. And. The, language will continue, to change over, time. But. We want these changes to continue to benefit the majority of people. We've. Already shown that we have some rather. Interesting. And, perhaps. Esoteric. Needs that. Maybe. The most people in this room won't ever need in their language and. I'm. Sure that we'll be introducing or helping, to introduce some, changes to the language that make the. Job of creating Russ, software, at Microsoft, easier but, we want these changes to, at, the very least not disturb, anybody. Else who doesn't need them and at, the at best, make. Sure that they are actually helpful for everybody else and. So this is extremely important, and. Going. Back to the idea of community we, want this design, to be again, community, driven so, just. Like the RFC process, has happened in the past with input, from everybody in the community if. They're able. And willing to give it we, don't want some backroom. Community. Design happening, and then hope, there's changes, to the language all of a sudden we. Want these changes to be driven by the community as a whole and. The. Last point that we want to talk about is this idea of reluctant. Russ stations. Presumably. Everybody, in this room is here voluntarily. At. Least I hope so. But. In the, future that might not be the case I, started, writing Russ because I wanted to not because I had to. But in the future there, probably will be people who have to write rust and they may not want to, what. Does that do to our community, as we, introduce more people into it who are kind, of reluctantly, there. And. The. Real challenge with this is maintaining, what, I like to call the community spirit, or the, thing that we already collectively. Have as a community the thing that in my opinion makes. Rust awesome. How. Do we continue to have that as more and more people who are not necessarily there by choice, interact. With us and come into our community. Okay. So we're at the end now and this is really where we asked you to come talk with us, come engage with us even.
If You work for a small startup, or you're just a hobbyist programmer, maybe you're not a software, programmer at all you just like to use rust in your free time I'm all, the way to up to you're working at a huge company and have some of the similar problems that we have we'd, love to talk with you hear, from you work. With you and really. What, we want to be as Microsoft, is a part of this community, we're. Not here to take over we're here to be. A part of it and, work with all of you so. Thank. You very much.