Intro to Roc & Innovation in Functional Programming • Richard Feldman & James Lewis • GOTO 2023

Intro to Roc & Innovation in Functional Programming • Richard Feldman & James Lewis • GOTO 2023

Show Video

Hello, everyone, and welcome back to  GOTO Unscripted. My name is James Lewis,   and I'll be talking today, I'll be in conversation  with Richard Feldman. Welcome, Richard Feldman. I   think we're just gonna explore some ideas around  things like language design, because Richard will   introduce himself in a moment, but is the creator  of the programming language Roc, and author of   the wildly successful, I'm told, he paid me  to say that, "Elm in Action." But, yeah. So,  

we'll just have a conversation about languages,  and programming in general, I guess. But, welcome.   Richard, maybe you could introduce yourself. I created the Roc programming language, which   is still kind of a work in progress. But I wrote  the book "Elm in Action," and also spent a lot of   time with Rust. And later in the conference, I'm  gonna be talking about Rust and Zig, and, sort of  

together. Very interesting language design. You  were telling earlier, before the cameras were   rolling, about how you were there when Rich  Hickey announced Clojure, at the conference.  I was. I was at JAOO, I think it was still  JAOO at the time, in Aarhus, in Denmark. Rich   turned up and sort of blew everyone away with  this announcement of Clojure. And obviously,   it's been pretty successful since. We've had many  projects in Thoughtworks that have used Clojure  

over the years. And I know there's a consultancy,  JUXT, in London, that is very successful as a pure   Clojure consultancy, actually, in the fintech,  mainly in the fintech industry. What do you think?   I mean, it'd be interesting to get your take on  that. Do you think there are different domains   where different languages are more or less suited? Well, there certainly seems to be an element   of suitability, but there also seems to be an  element of just, sort of, cultural momentum. Like,   something will get traction in a particular  domain. Maybe it is, maybe it isn't, like,   especially well-suited for it, but then it just  sort of perpetuates. So, the example that comes  

immediately to mind is Rails and Ruby. Right.  I mean, if you were to zoom out and  say, aliens land, and they're gonna   pick which of the programming languages  are gonna become big in web development,   I don't know why anyone would say, "Well, it's  gonna be the one created by the Japanese guy   that's only big in Japan right now, that's,  the tagline is, 'Let's make programming fun.'   That's what's gonna be used widely in industry,  you know, and blow up in the next 10 years." I   don't think anyone would have predicted that. So, I don't think it's necessarily just about,  

you know, like, how well-suited it is, it's,  like, the perfect fit, as much as it is, like,   well, you know, one person, like DHH, made  Rails, that resonated with a lot of people,   and because Ruby was the language that he chose  to make that in, yeah, he could have made it in   Python. And he would probably say, "Nothing  else but Ruby would have inspired me to make   Rails." But I think you could pretty easily make  the case that someone could have made something   as successful as Rails in a different language. The thing, you mentioned Python. That's super   interesting. Because I remember when Rails  was massively taking off. And in North America  

in particular, and in India, Rails became a  huge thing. I mean, our founder at the time,   was very taken with it, and some very persuasive  people were talking about it. Obie Fernandez,   for example. And it seemed that we suddenly had  a load of projects in North America and India  

using Rails. And we still do. I think the world's  largest Rails project was a Thoughtworks project   in Atlantic City, maybe. Really? But the weird thing is, it didn't spread to the  rest of the countries that we're present in. So,  

the UK was, still, remains still very  much a Python shop. So, maybe it's not   just a domain-specific application. There's  also, like, a geographical thing going on   Well, I think you could generalize that to  culture. Certain pockets of culture might   be geographic or might be just other things,  that contribute to those things. I've spent an   increasing amount of time over my career, like,  learning about, like, why things get adopted and   why they don't. And the more I learn about it,  the more reasons I discover. And it seems like  

there's just an inordinate amount of variables.  And as programmers, we like to look for simple   solutions and simple explanations for things. But,  much like with...I would say another true thing,   where I, the more I get into it, the more  variables I discover, has been, like,   performance optimization, where, like, when I  was in school, you know, all the focus was on   big O notation, and, like, what's the asymptotic  complexity of this algorithm and stuff like that.  And now I'm like, "That's, like, this much  of it." That's the tip of an extremely large  

iceberg. Similarly, with adoption, languages or  technologies in general, it's like, you know,   I would have thought, you know, early on,  it's like, "Oh, well, people use that because   that's the best thing for it. What else  is there?" And it's all these cultural,   and, all these timing, factors that come into it. I remember doing a thing with one of my colleagues   about program language adoption. And certainly, it  was as much about culture availability as anything   else, it seemed. You know, are you better off  picking something where you know there's a lot   of people out there who you can just hire for it?  Or, another example would be, a counter-example,   you know, we had a publishing client who  deliberately chose Scala because it meant   that they could offer a fun, or more, potentially  more, "fun" programming environment for developers   to come in because they couldn't pay the same  rates as the banks. So, there's this almost, like,  

trade-off is, do you offer this more interesting,  exciting environment, versus, okay, just, yeah,   whatever it is, thousand euros a day for just  a standard job for the developer in a bank.  I think that's an underrated trade-off, is that  if you're a company and you're considering a novel   technology, and I've talked about this before  in other settings, but this is something like,   you know, at my previous job and my current job,  we both have used Elm on the front end as, like,   the entire front end, not just, like, a little  part of it. Embracing that means that you get   to be very selective about who you hire. We just  filled a front-end role, and the recruiter was   talking to me about, yeah, we, you know, hired  this guy, and it was close. We had to decide   between him and, like, you know, a couple of  other people who wanted this role. Usually,   it's the other way around, where employers are  like, "I just wanna find anybody who can fit   this description that meets our criteria, and  it's really hard to find people." But you flip  

the script when you're offering a technology  that people wanna use, but that not a lot of   employers are using. And that's almost sort  of like a, you know, self-fulfilling prophecy,   in the sense that if enough employers do  that, then it flips back around. But then,   by then, it's mainstream, so then it's not notable  anymore, and now you are saying, "Oh, well, I can   just find lots of people to do it, because it's  become mainstream." But I think most people are   aware of that side of the dynamic, but they're not  familiar with what happens before a language gets   mainstream and what the dynamic's like over there. I guess the big question is, did the person you   hired have 25 years of Elm experience? Well, let's see, Elm was created   in 2011, so yes. Of course. Which, I guess, when it comes to Elm,  

I'm gonna throw my hands up and profess to not  generally being any good at all at front-end   software development. That's fine.  It's not something I've done in my career, to  be honest. Any front end that I program tends   to look like a bad implementation  of Excel. That's pretty much it.   Well, Excel is not an easy thing  to implement, so, you know.  Well, funnily enough, coming back to Ruby,  the creator of RSpec, Nicholas Nielsen,   showed me an implementation of a spreadsheet  written entirely in the const_missing method.  Wow. Because if you think about  

having a spreadsheet, you have to be addressing  the, you know, by capital letter first. If you   didn't have that defined as a.. Oh, that's hilarious. Wow.  And you could do all the  calculations in “const_missing”. Have you been surprised at the adoption  of Elm and how successful it's been?  Yes, but maybe not in the way you might guess.  So, I would have thought that it would have been   sort of all or nothing. I would have thought  that either a language like Elm would just   take over the world, or it would just peter out  into nonexistence, and people would, you know,   walk away from it. Because I've seen that happen  with various languages. Like, TypeScript would  

be an example of taking over the world. That's  happening right now. And then CoffeeScript would   be an example of something that sort of petered  out and, you know, is not used anymore. Elm seems   to have sort of, like, found a solid niche.  There's just, like, a chunk of people who are   like, "Yes, this is how I wanna do front-end  development," but it doesn't seem like it's   on track to take over the world. It seems like  it's on track to be... Well, it already is, like,   a self-sustaining thing, and it seems like it's on  track to sustain. So, that's something that we've  

seen with a lot of backend languages. There are  plenty of backend languages that are not, like,   no language has taken over the whole backend.  There's just, like, people have preferences   on the backend world. Whereas on the front end,  it's very much been, you can use any programming   language you want, as long as it's a JavaScript  dialect. Like, it could be JavaScript or it could  

be TypeScript or it could be CoffeeScript, all  of which has the tagline, "It's just JavaScript,"   or implicitly in the case of JS itself. All of the other ones have been, like, kind of   niche players. But if you think about it, I mean,  like, on the backend, it's really common to have   a language that has, like, low market share, but  is, like, quite a healthy, active community, with   lots of people in it. It's just on the front end,  that's, like, a weird thing to be. And Elm being a   front-end-focused language, I just never guessed  that. I thought it was, like, oh, it's either   gonna take over or it's gonna peter out. I didn't  expect it to become more like a backend language,  

in that it's just, yeah, there's a chunk of  people who like to do it this way, and it's fine.  You mentioned TypeScript. So, that's  the elephant in the room in some ways,   right? So what would you ascribe to...can you see,  sort of, any particular reasons that TypeScript   has sort of eaten the world, or there's some  discussion about it at the moment? They're both,   on the surface, surface, fairly similar ideas. Elm and TypeScript? Or...? Which two things?  Well, so, essentially taking something that's  gonna be able to be used in the browser,   but offers may be a safer, allegedly more  productive perspective on programming front end.  I think, like, when I think about comparing Elm  to JavaScript and TypeScript to JavaScript, and   I guess also TypeScript to Elm, like, TypeScript  and JavaScript, I mean, TypeScript is really like,   "This is gonna feel like JavaScript, but with  types." Elm is like, "I am a programming language,  

and I run in the browser." It has no relation  to JavaScript other than, like, as a compilation   target. So, you mentioned, like, Clojure earlier.  I would liken Elm to Clojure, except, like,   even more separated from the hosts. Like, Clojure  is very much like, "I'm a programming language,   but I intentionally have some Java-like elements  inside," but I don't think anyone who's written   Clojure and has written Java would say, like, "Oh,  this is a Java dialect," you know? But they do,   like, share data structures and things. Whereas  Elm, it's even less than that. It's just kind  

of like, well, we use the same, like, string  representation under the hood and stuff like that,   but that's kind of about it. It's, like,  this feels like a different programming   language. Whereas TypeScript feels like this  is a new take on JavaScript, I would say. I guess that's maybe, it's a good comparison,  I think, with Clojure as well, because if you   look at something like two different JVM  languages, like Clojure and Scala, say,   I mean, most people's entry point into Scala was  programming Java without semicolons. That was the  

old joke, wasn't it? And, whereas Clojure is a  fundamentally different paradigm, a fundamentally   different way of approaching writing code. That's a good point. I've talked to people in the   Scala community who talk about there being sort  of three different ways that people do Scala. So,   one is, like, Java++, or Java without semicolons,  maybe. Another is, I want a hybrid OOFP language.   I want a language that has a lot of OO support  and a lot of FP support, and I'm gonna use them   together. And I can't get that from Java, so Scala  is the way to go. And then the third group is,  

I want Haskell, but my boss won't let me  use it, so I'm gonna use Scala as my Haskell   stand-in, and that's also a popular way of using  it. But I don't see the same thing in Clojure or   Elm. It's like, pretty much it's like, nobody's  using Clojure as, like, Lispy Java. Everyone's   using it as, like, Clojure. The same thing in Elm. Would you say... Maybe a bit random,   but so, I remember a few years ago, when Google  first, sort of, publish is the wrong word,   but created and then started talking about Dart,  the programming language. We act that in...we   have a thing called the Thoughtworks Technology  Radar, which every six months, we sort of take new   stuff and sort of think about it and assign  it, like, an assess or trial or hold. And,  

at the time we sort of said, we put Dart on  hold, on the basis that we were super worried   that adoption was gonna be limited by the fact  that other browsers weren't gonna jump on board,   right? Because it was very much a Chrome... With the VM part of it?  The VM part of it, yeah, yeah, yeah.  And of course, that's now come back,   right? I mean, it shows what we knew. Like,  some years later, we now have Flutter,   which is kind of, you know, very much being  adopted quite rapidly at the moment. I kind  

of find that kind of interesting, where you've  got something that sort of, at one point in time,   wasn't the right time for it to be adopted, but  then later on, it suddenly is the right time. Well, I think that's a great story. That's  the...I mean, Dart, to me, fits the same   category of adoption as Ruby, where it existed  for quite a while like Ruby was just big in   Japan for a while. Ruby was created to be, like,  "Let's make a language..." I mean, Matz was like,   "I wanna make a language that's fun to program."  That was the word he used. I mean, Dart, as I   understand it, was created, basically, because of  the VM, because Lars Bock, you know, had done V8,   and was frustrated by how difficult it was to  do certain optimizations around JavaScript,   and he was thinking, "If we just had a different  language that felt a lot like JavaScript,   but which was different in certain very specific  ways, we could make a much more efficient VM   implementation out of it," and that was kind  of the motivation behind creating Dart. And,  

you know, if you think about it, why would people  want to adopt that unless you're a VM author?   It's like, okay, but I'm over here doing my web  development job. What's the pitch to me? I don't,   you know, care about how easy it is to optimize  the VM or how optimized it can be. I just...   Especially since, you know, you and your team,  Lars, did such a good job making V8 a lot faster. Thank you, Eric, as well. What's in it for me to switch from JavaScript or  

CoffeeScript, which was big at the time, to Dart?  But then the answer comes with Flutter. And again,   you could make the point, Flutter didn't have to  be implemented in Dart, but it was, the same way   that Rails didn't have to be implemented in Ruby,  but it was. And that, I mean, if you look at what   percentage of Dart usage in the industry is not  Flutter, I would guess it's very small, similar to   Ruby and Rails. I mean, it's, like, overwhelmingly  Rails, it's overwhelmingly Flutter. So,  

the term I use for this is, like, this is, like,  the killer app adoption explanation, is, like,   there's some application of the language that's  so popular that it just brings the language's   popularity along for the ride because people wanna  use that thing, and that thing is implemented in   that language, and they want it so bad they'll use  whatever language it happens to be implemented in. That's quite a nice segue for me to  go to talk a little bit about Rust,   maybe. Because you mentioned about your new  language, Roc, that you're writing. We'll come   on to that maybe in a minute. Sure, sure, yeah.  But you mentioned the fact that the compiler is  written in Rust, and that's another... I mean,   I think, well, we are starting to see, in  terms of Thoughtworks, and our clients,   adoption in very specific areas, for Rust.  Specifically, there's lots of interest,   for example, in automotive, or, you know, sort  of safety-critical systems and these kinds of   things. What made you choose Rust yourself? This is going to bring a little of my talk  

into this conversation. No, that's cool. This'll   be published a lot later, so... Basically, it's important to me that   the Roc compiler be very, very fast. I want it to  run as fast as possible, and I certainly did not   want to get to a point where I'd built this whole  compiler out... I say "me," because that's what I   was thinking at the time. Now it's a bunch of  people working on it, and a lot of them are   better at this stuff than I am. But, you know, I  didn't wanna end up with a compiler that was very  

feature-complete and very done, and then we're  like, "And we can't take any more performance   out of it because of the language we've chosen,  that is, like, garbage-collected and whatnot,   and there's just this ceiling we cannot possibly  exceed, no matter how many hours of performance   we put into it, unless we rewrite it in,  like, a Rust or a C or C++ or something."  And I thought, "I don't want that to happen.  I want this to be as fast as it can be,   and I don't wanna hit that ceiling." So, that  meant one of a couple of different options. One   was to do C or C++, which I'd had some really bad  experiences with earlier in my life around, like,   getting memory unsafety-related bugs that were  painful to track down. And I was, like, well,   the pitch of Rust is that you have the maximum  memory ceiling, but, somehow, and I didn't really   knot the time, somehow, they do compiler things  to help you not run into those problems. And so I  

thought, "Well, that seems like kind of the only  game in town that fits all my criteria. There's   no performance ceiling, and yet I'm not going  to get these memory unsafety bugs that are a   nightmare to track down." So, I took the plunge,  and I'd done a little, like, toy thing in Rust,   or I'd built a command-line app that I'd never  quite finished before. So I had a feeling for the   language, and I was like, okay, I can get this.  I can stumble through it. And now I feel very   comfortable in Rust. But when I started, it was  like, just because I had this list of criteria,  

and that was the one language that fit them all. And you got to choose as well, which is   the nice thing, right? Yes. Very important.   I remember my colleague, Erik  Doernenburg, he's based in Germany,   he's a head of tech at the moment in Germany.  And he did a great talk at one of these events   on Rust. And it was back at the time when  not that many people were adopting it. So,   it was quite early on. [inaudible 00:17:54]  and it was a bit of an overview on why Rust,  

and, again, actually why some of the other  languages that have started to appear around,   which, you know, like Go and, oh God, I  always forget the Apple one, Swift, is it? Yes, Swift. And why they,   you know, what problems they were attempting to  solve, you know, which is around memory safety.   It's something like, I can't remember the exact  number, but some very high proportion of bugs...  Seventy percent of CVEs, yeah. Microsoft. There you go, right? So,  

yeah, I mean, this was around that. But he did  this lovely little thing at the end of it, where   I think he...he didn' wasn't Conway's, it  wasn't Game of Life, but it was a similar kind of   agent-based kind of implementation. And he always  uses that when he's learning a new language,   right? You need something, you know, you need some  framework to understand when you're learning a   new language. And he started running it, and he  was sort of running multiple iterations of it,   and he was looking at the performance. He  was like, this is a lot, lot faster than the,   I think it was a JavaScript implementation,  ridiculously faster, orders of magnitude   faster. But he thought, "Actually, I thought  it'd be better than this." And he realized  

he hadn't turned off, and I'm gonna get  this wrong, but he hadn't turned off the,   there's some kind of, like, setting in Rust,  I think, which you can turn off. It's, like,   "production mode" versus...does that make sense? Oh, yes. This is an optimization flag, yeah.  Right. And he'd forgotten to use  that. And then suddenly it was,   like, three orders and four orders of magnitude  faster. Which I quite like as an idea, yeah.  That sounds about right. That  flag makes a big difference. 

Maybe let's generalize it. There's Rust in  particular, which is spiky, I'm told. I've   only read some books. I've not made any serious  attempt to learn it. But I'm informed it's quite   spiky. There's quite an adoption curve. How do  you go about adopting or learning new languages?  Yeah, so.... Or do you know enough now that   you just go, "Oh, it's that sort of thing?" Well, it's funny, because you mentioned,   like... I know a lot of people who like to do  the same, like, "I'm gonna learn a new language,   I'm going to pick a project, like Game of Life,  that I'm very familiar with, and implement that   in the new language." I'm almost the opposite,  where I always need to have some specific project  

in mind first, where I'm like, "I wanna build  this in this language," or this, whatever the   new technology is, and then that motivates me  to push through whatever the learning curve is,   because I'm like, "Well, I can't get it any other  way, so I gotta do it." Whereas, so, I guess maybe   I don't tend to just seek to learn languages just  for the sake of learning them. It's more like,   there's some problem I wanna solve. This seems  like the right tool. All right, let's go. So,  

I don't think I've ever done the... And picking the easy things,   right? I'll just write a compiler. I'll just write a compiler, which I'd   never done before either. But, I guess, I don't  know, at least for me, the hard part of learning   something new is generally sort of finding the  motivation to climb over these obstacles that I   hit, whatever they might be. And I also am aware  of, there's an element of, if you pick a project  

that's too hard and a language that's too hard,  and, you know, like, those can kind of compound,   for sure. But I had previously done this  little command line app in Rust, where,   actually, the motivation there was, it was the  Elm test runner, and now there actually is,   somebody else separately went off and, like, did a  different Elm test runner implementation in Rust.   But at that point, it was mostly just frustration  with Node.js APIs, which is what the one I'd   written previously was in. And, I one day... Not because Node is blazingly fast.  No, it was nothing to do with performance. I wanna  write this in something that has a different set  

of APIs, shall we say. And I didn't really wanna  use Go, because I didn't have any particular   interest in Go. And I was like, "Well, I wanna  learn Rust, and I want to have a codebase that I   can maintain that's not Node.js anymore. So I'm  just gonna rage rewrite it in Rust." And I got,   like, 70% of the way through that, and I was like,  okay, I have a feel for this language now, and it,   you know, feels...I'm not great at it, but I at  least can stumble my way through doing things.  

And I have this code base that, as happens  with many projects at around the 70% mark,   I was sort of, like, okay, yeah, but do I really  wanna the rest of the work to get this over the   finish line, and then maintain that codebase  and then new contributors are not gonna know   what they're doing, and so on, so I ended up kind  of putting it on the shelf and not finishing it.   But somebody else separately went and did it. I definitely would agree with...the learning   curve on Rust is a downside. It's quite high,  and it's also, it's, some languages,   I think, have a high learning curve because,  like Haskell, for example. Haskell, I would say,   has a high learning curve, in part because  of a lot of the things, you're encountering   for the first time. I've never heard of these  concepts before, I don't know what they're about,   and there's just kind of a lot of stuff to learn.  In Rust, I would say the thing that's the hardest  

about the learning curve...and people often talk  about "fighting the borrower checker." So, the   borrow checker is kind of Rust's, like, marquee  feature. It's what sets it apart from other   languages. It's what gives you the memory safety.  But, at the same time, it's not so much, like,   you can just sit down and, like, once you wrap  your head around the borrow checker, you got it,   and it clicks. It's more like there's just a whole  lot of things that all fall under the umbrella of   borrow checker, but there are various scenarios. And I remember one time, it took me,   I'm embarrassed to say, like, I think it was,  like, two months or something, where I couldn't   figure out how this, part of the compiler was  blocked, and I couldn't figure out how to do the   thing I wanted to do. And, you know, the borrower  checker gave me an error, and said, "You can't do  

this." And I was like, why not? I know this is  possible. If this were in, like, C or something,   I would just be like, "Here, take this thing and  put it over there. Put it on this thread." And   it was like, "No, you can't do that." And I was  like, "Well, why not? Why can't I do that?" And  

I eventually realized, I was like, wait a minute,  do I just need to use, it was IterMut versus Iter.   And what Iter...the difference is, iterating over  these, Iter is, like, I wanna iterate through   these things, and IterMut is, I wanna iterate with  the possibility of mutating them. But it didn't   occur to me to use IterMut because I didn't wanna  mutate them, at all. But the problem was I needed   to use IterMut to prove to the borrower checker  that I had permission to mutate it, which meant   that it was safe to put it on a thread. So, in  this case, mutable was sort of a stand-in for,  

"is uniquely owned by this particular instance."  And I switched it to Iter and IterMut, and this   thing that I had been stuck on for, like,  two months, it was like, "Okay," right? And… I would love to have been in the room   at the time, it was like, oh my God. But I bring this up as an analogy of   even though I had that, I already knew the mental  model of what mutable means "is uniquely owned,   and therefore has permission to do certain  things," it hadn't occurred to me that,   I didn't, like, put two and two together  with the implications of that, that, like,   oh, if I want to put these things on threads, I  need to IterMut, even though I'm not gonna mutate   them. So, it's just a lot of stuff like that. It's almost like you're being more restrictive   than you need to, in some senses, right?  But because the mental model is, okay,   this is a restrictive memory model,  so I wanna be overly restrictive   And I think, in this case, it was more of a  language terminology thing, in the sense that,   I think if instead of calling it IterMut, they  called it, you know, IterUnique... I'm not saying   that they should rename it. It's more just, like,  if they called it that, I think I would have more   quickly realized, like, "Oh, yeah. To hand these  things out to the threads, they have to be unique  

because the whole point is I don't want them  to be shared across the threads." That's, like,   another aspect of Rust that makes it tricky,  is that part of what the borrower checker   does has to do with when things are, like,  available in memory, like, when they're in,   the lifetimes of, like, when they're alive  and when they're, you know, can be reclaimed.  Also about mutation access, like, this thing can  or cannot mutate it. Also, multi-threading and,   like, which things have permission to  mutate things, which has to do with, like,   preventing data races, in addition to memory  safety. So, there's just a lot of different  

things that all kind of come together, and put  it all together, you get a big learning curve. You've spent a lot of time building this  compiler. But the aim of it, presumably,   is to compile this new language. So, maybe you can  talk a bit about Roc, and what makes it unique...  Yes. ...and why?   Why do you decide to write a new language? For Roc, the tagline is "Fast, friendly,   functional." And I was just talking  to Dave Thomas, and he mentioned  

that he knows someone who made another  language. I think it was, was it K maybe?  Yes. Richard Feldman:, the   tagline was "Fast, fun, functional," which  I did not know existed, but it's very,   very close to what I independently came up with.  But the basic idea is, I really wanted a language   that felt like Elm in terms of the ergonomics and  the overall user experience, but which is for,   instead of being focused on browser-based UIs,  which is sort of Elm's bread and butter, I wanted   it for sort of, not just, like, one other domain,  but sort of, like, the long tail of domains. So,   I'm not just thinking about, like, servers  and commandline apps, although those are the   two things that people are most interested in it  for. Or desktop GUI applications, which I'm also   interested in. But also things like... If you can replace Electron,  

the world will be a happier place.. Well, that's a very big challenge, right? It's   not an easy thing. There's a reason Electron's  so popular. But definitely, I've always run into   these little cases, where it would be, and Vim  script is gonna be the one that comes first to   mind. I wanna write a Vim plugin. I don't wanna  learn Vim script. I don't wanna use Vim script.  

I've heard, you know, it doesn't have a good  reputation as a language. But what I wanna use is   I wanna have, like, an Elm-like experience, this  really pleasant experience I've had with Elm. But   Elm, being a focused language, is not ever gonna  get into that. There's never gonna be an Elm for  

Vim script. So I wanted to make a language that  was capable of being used in lots of different   domains, while still feeling like it was, to  some extent, domain-focused, like how Elm is. Richard Feldman:So, without getting too much  into how we achieve that, there's this basic,   high-level concept of platforms and  applications. So, what we mean by that,   an application is basically just, like, you know,  my project. I'm building a thing. A platform is   something like a framework, in the sense that  it's sort of the foundation that you build on.  

You never have more than one platform. You always  have one. But unlike most languages, in Roc,   you have to pick a platform. There's no such thing  as, like, a platformless Roc application, or,   like a, you know, framework list, if you will. And the reason for that is that platforms,   although they kind of feel like frameworks,  they're scoped differently. So, a framework,   typically, like, let's use Rails for example,  Rails will be in charge of things like database   access, and how do you do routing, and, like,  request handling and stuff like that. In Roc,  

sure, that would be true too, but also, it's  gonna be in charge of all of your low-level   IO primitives. So, it's gonna say, here is all the  things you can do, in terms of HTTP and, you know,   database access and this and that. And for a web  server, maybe you have, like, the full range,   but you probably don't have, like, reading from  standard in on a web server. Does that make sense?  Yes It does.  Maybe you leave that one out. Now, a better  

example, though, is let's say that you wanna make  a platform for, like, a database extension. When   you're writing a Postgres extension, do you even  wanna, like, have network access? Do you wanna   have arbitrary file system access? Does that make  sense? So, the way most languages do this is the   standard library has all these really low-level  IO primitives, and then there's certain use cases   where it's like, eh, don't do that. Don't write to  that. But a problem this creates in the ecosystem,   for these, sort of, long tail of use cases, is  that you use a library, and that library is like,   "Oh, I can just, like, create a temp dir, and put  stuff in there, right? And it's like, I don't know   if I want you doing that on my database server,  you know? And so, the idea is that, by basically   making it so that you have to pick a platform,  the platform says which primitives are available,   the ecosystem will sort of naturally design itself  to be accommodating to that, and to be aware of   that, and to be like, "Oh, if I choose to, you  know, use a temp dir or whatever, that's gonna   restrict which platforms I can potentially  run on." If I read from standard in, that's   gonna restrict which platforms I can run on. Another thing is that the platforms, because   they're in charge of the IO primitives, they can  implement certain, like, sandboxing features. So,   one example of something that I'd be...I hope  someone builds in Roc, because they now can,  

which I would love to use, as a sort of, a sandbox  script runner. So, for example, like, this is   something that Deno has at the language level,  but in Roc you can just, anyone can implement   it in user space, which is basically, like, you  know, if I download a script from the internet,   and I run it, I know it might mess up my machine.  Like, it might give me a virus, it might write to   places on my disk that I didn't want it to  write to. But because in Roc you have this   platform-application split, if I have a platform  that's like, "I'm a commandline runner, but I'm   a sandbox commandline runner," and because I'm in  charge of every single one of the IO primitives,   I can say, "Yeah, look. I give you access to all  the IO primitives, but guess what? If you try to   write to this part of the file system, or you try  to read from there, I'm gonna prompt the user, and   there's nothing you can do about it." So it's now  as safe as a web browser, in terms of, you know... 

That's very interesting.  But at the command line. And   I would love to have that, because I run stuff  that I download from the internet all the time,   and I'm either doing it in a VM, right? You heard it here first, folks. You   shouldn't run stuff you download from... Yeah, well? And we all do, right? And I   would love to have something where I just had this  confidence that I don't need to audit the whole   thing. I just need to look what platform are you  using? Okay, it's the sandbox one. Great. Done.  I think this is a really interesting idea,  because, I mean, I've only sort of come across   this maybe a couple of times before, but it  seems to have...people aren't talking about  

it. It seems now, but five years ago, there  were lots of people talking about unikernels,   for a different reason. This was about  security, and about the kind of, you know,   the different, yeah, the surface, the attack  surface area, essentially. Can we limit the   amount of stuff we're gonna compile into our OS  so that it's not available? You can't even use   any of it. It's just not there. And I think I  had a line at one point that Docker, 30% of the   way to unikernels. It's like, you know what I  mean? That was five years ago. I talk about it,   but it seems like, in some ways, a similar idea,  but coming at it from a different perspective. 

It's definitely about, I mean, I would say  the thing that you have in common there is the   idea of security through just, like, absolutely  not making things available in the first place,   rather than having them be available and trying  to make sure you played Whac-a-Mole and locked   everything down, right? Just saying, like, it's  not even there by default, and we are only gonna   opt into giving you access to the minimal set of  things necessary to do whatever you need to do.  Cool. And what sort of language  is it? Is it a purely functional   language? You said it's functional?  It's functional, and I would say, like Elm,   there's a very heavy focus on usability  and user friendliness and stuff like that.   There's different sort of schools of thought  of, like, functional programming languages. So,  

I would say that, like, Haskell is very focused  on, like, mathematics, or at least, like, it   culturally feels that way. Maybe different people  would disagree with that, but... And I would say,   like, Clojure is a very, like, you know, it's  all about Lisp, and, like, macros and, like,   these particular set of primitives that are not  necessarily required for functional programming,   but, like, fit together in interesting  ways with functional programming. And,   like, Elm and Roc are very much, like, typed,  purely functional, very focused on, like,   having a small set of simple language primitives  that work well together, and then nice compiler   error messages and ergonomics and stuff like that. I would say we're, on the tooling side,  

we're drawing a lot of inspiration from Go, where  we're like, we have the test runner built in,   we have the formatter built in. We wanna make  it so, you know, you download the Roc binary,   and then you can just go. You don't need to,  you know, pick a bunch of things off the shelf,   you know, to get things that... Everybody  agrees you should have a testing system,   but you don't need to go pick one off the shelf.  It's like, it's there. It's right there, built in.  And have you taken the same tooling of Go? I mean,  have you taken the same decisions around things   like testing with...or is it Rust? With Rust you,  test inline. You have the test in the same file   You can do that, yeah. Yeah. So, we do have  inline tests. You just, it's, the keyword is  

called "expect," so you can just, like, write  your function, right below it, next line,   expect whatever, and then you're done. Ah, super cool.  Actually, I guess a nice example of ergonomics.  This is always something I'd liked. Power Assert   is the one that comes to mind, that I use, and  I also, back in the day, I did a little bit   of development with Groovy, and they had that  built into their tester, and I always thought   it was cool, is that when you run your tests in  Roc, you can just write normal booleans. Like,   you don't need to do, like, assert this or that.  You just say, like, you know, expect X equals, you   know, for X equals five, and that's it. And what  it'll do is, if that test fails, is it'll show,  

first of all, it'll print out the source code of  the actual test that you wrote, and then also,   any named variables that you had, it'll just tell  you what their values were. So you don't have to   go back and be like, "Oh, wait. What was this  and that?" Just trying to give you...and we've   also talked about maybe expanding that a little  bit to tell you, like, what's on either side of   the equals, or if you had, like, a less-than, you  know, show you those things, because you might   wanna know. Just try to give you the info that  you want anyway, and don't make you go back, and,   like, debug log the test. Yes, cool. 

That's the first thing you usually do anyway,  right? So might as well save you the trouble.  I was always of the opinion... I don't write as  much code as I used to anymore, has to be said,   but I was always of the opinion if you  use the debugger, you're failing somehow,   but it was a...I come from a very, sort of,  purist TDD kind of background, if you like, so…

Well, but regressions still happen. Yes, of course.  Yes. So, is it out? Is Roc out now?  I would say it's pre-release. So, we don't have a  numbered version yet. You can download a nightly   release. We're in the process of making a real  website right now. Depending on when you watch   this, maybe it'll be out. But right now, it,  like, as of this exact moment, there's kind  

of a placeholder website, that sort of describes  the language, but it's very bare-bones. But now   we've gotten to the point where it's useful for  things. So, before this point, like, last year,   I would say, like, "Well, you can try it out  and play around with it, but it's not, you know,   really that useful," but now it is useful. I would  say it's useful, but very immature and early,  

and there are bugs and stuff like that, but you  can, like, build stuff with it for real now. And   now that we're at that point, we're like, "Okay,  now we need a real website, and," you know, so   it's ready to be used by early adopters who aren't  afraid to sort of roll up their sleeves with a new   technology. But, like, I have a lot of fondness  for my time at the beginning of Elm, because,   on the one hand, when you have a small set of  people using the technology, yes, there's sharp   edges and bugs and stuff, and the ecosystem's not  there yet. But on the other hand, you know, I used   to work with Bill Venners, who made ScalaTest,  and I remember thinking, "How could you have   made something that's used by so many people?"  and I asked him about that, and he's like,   "Oh, that's very easy. Back then, there was no  testing thing, so I made one." And that's how it   is in the early stages of a language. Somebody's  gotta be the first person to write whatever X is  

for that particular, you know, use case. My career goes back through, you know,   before, my programming career goes back through  before Java, essentially, and that sort of   completely changed my life, right? So, when Java  came out, and the internet, essentially, well, the   World Wide Web, and Java really sort of changed  pretty much the way, I think, many programmers   went about their job. But the interesting thing  with that, and especially in Thoughtworks,   is everything was a first. You know, everything  you were doing was a first, in a lot of ways. The   kind of, the testing frameworks were a first. The  continuous integration servers were a first. The,   you know, acceptance testing frameworks, like  Selenium and these, they were a first. All  

these sort of things were, the innovations that  were happening were because people were facing,   were hitting these issues, and then kind of trying  to come up with a way of solving a problem that   they were experiencing on a day-to-day basis. I  do sort of wonder now, are we still seeing that,   or are all these sort of solved problems  now? It's just when we have a new thing,   new language, say, like Roc, we need to  create the test runner for them, you know,   and there's someone who's gonna be the first  person to do that. There's someone who's gonna be   the first person to do X, rather than it being... Or, another example would be things like machine   learning, you know, applying engineering  discipline to machine learning. So, you know,   there was a period, not so long ago, where the  idea that you might version control your model   was, like, a crazy idea. Why would you think about  doing... But that's now a kind of normal thing,   so things are repeatable and so on. Is this  a case of, sort of, we're applying, I guess,  

a set of tested and known patterns  to the new things? Is that a kind of… I'd say it's a mix. So, an example that comes to  mind is, so, in Roc, we have a, as far as I know,   unique, I don't know of any other language that  does it this way, approach to serialization and   deserialization. So, two different ways that  this is, like, commonly done today... So,   there's, like, the JavaScript way, the Ruby way,  where you get some JSON in, and you just say,   like, JSON.parse(), and it's like, cool. Now  you have a JavaScript object. And of course,   the downside of this is, you know, you  get partway through your program...  Cool, now you have a JavaScript way What if the JSON doesn't match what you thought it   was gonna match? You're gonna find out about that  eventually, but it might be pretty distant from   where that original problem happened. So, that's  one way of doing things. Another way of doing   things is, I'm thinking of Rust, but, I mean, I  know in Java, you can do it the same way, where   you have a schema up front, and you say... So,  this would be, like, Jackson in Java. So, you say,  

"Here is exactly what I expect it to look like,  and, you know, come and parse the JSON, and if it   doesn't match that, fail right away, right there." So, that, in terms of, you know, how easy it is to   debug later, I would say that's easier to debug  later. But a downside of that is that you do need   to actually write out the whole schema, and, you  know, sort of keep it in sync with your program,   and so forth. So, something we've introduced  in Roc, that as far as I know is novel,   is that we kind of have both. So, you can write at  the same time... So, you write the equivalent of,   like, JSON.parse(), and it does just, you don't  have to write a schema, but what it does is it   uses type inference to infer the type that you're  parsing into, and based on how it's using the rest   of the program. And so it actually will decode it  right there at the call site, and if it doesn't  

match how you're going to be using it throughout  the rest of the program, it fails right away.  That's super interesting. Yeah. Now, what's interesting about that is that   that's not specific to JSON. It's something that's  just, like, we call it, you know, "decoding" is   the general term for it. So, in order to make it  work for, let's say, JSON, somebody needs to write  

a particular, like, JSON-aware parser, that works  with this framework, so that it can, you know,   translate between JSON and Roc values. So, on the  one hand, you could look at that and say, "Well,   this is just somebody needs to write a JSON parser  for Roc." But on the other hand, structurally,   it's different from how it's done in  other languages. It's not like you're just   translating it into a normal JavaScript object. Is there a TypeScript library called io-ts   or something like that? I've heard of this, yeah. I believe that   that works like it works in Java, and in Elm and  Rust, where you do make a schema, and, you know,   somehow you define, in code, like, you write  some code that, you know, does this. I assume,  

I don't know for sure, but I assume that you  either write it by hand or you run some code that   generates it or something like that. But as far  as I know, in TypeScript, it's either you do that,   or else you'd just say JSON.parse(), and,  you know, that part's just not type-checked.  Yes, right. But, yeah. But the point being, like, you know,   if you're writing this, it's like, you're doing  it in a different way than has been done before.  

But on the other hand, it is still just, you  know, for JSON, for XML, for CSV, whatever.  It's good. We're talking about functional  programming languages, and we've finally got   to the point where something's a bit monad-like.  Which is good, right? Because that is interesting,  

right? That's why I found it interesting about  TypeScript, and you're parsing stuff over the   wire, and you've got this lovely type safety  within the environment you're working in,   which is the front end. But, as you say, like, you  could be sent garbage that is essentially, you've   got no way of knowing until you try and parse it,  decode it, whatever. So, I kind of like the idea   that actually there's maybe an attempt to solve  some of those problems, where you're actually   being type safe across the entire, I guess,  back, front end, etc. And across the wire. And   one thing I do... I did a lot of integration,  a lot of XML parsing in my day. And, you know,  

we used to use XML. We used to... What  was it called? XPath, that was the thing.  Oh, yes. I remember that. Where, rather than do the,   kind of, like, take the schema, basically have  a client that's generated from the schema,   and you kind of, you know, when you receive a  message, you turn that into the object, and if it   doesn't match the schema, you blow up. You'd say,  instead of that, you'd use XPath to just pick out,   and Schematron, actually, was the thing, you  pick out just the bits from the message that you   wanted, and therefore, you would know  were insulated from changes to the schema,   if you like. So, you know, if someone changed the  schema, you weren't just suddenly gonna blow up.  

Because this is the main problem, right? I mean,  how do you avoid that issue, of, essentially just   falling over in a heap if the thing that turns up  isn't what you were expecting? So, if it doesn't   conform to...If you can't decode it, right? Do you  just blow up, and just, like, sorry, we're done?  Well, the default is, I mean, it's not, like,  throwing an exception, it's just, like, you get   back a value that says either it succeeded, and  here's your answer, or it failed, and then here's,   you know, the error that it failed with, such as,  like, you know, this field is missing or something   like that. So, recovery is sort of up to you as  the application author. It's not, you know... I   don't think there's a one-size-fits-all  way to recover from data being missing.  Which, it's the compile time versus runtime  checking of these things, right? So, that's what   we used to do. We used to do it at build time. So,  we'll generate a library based off a schema, and   then that library's gonna be quite fragile in the  face of changes elsewhere, if you like, and you'd   have to recompile your application if someone's  schema changed somewhere, which is, like, suck. 

Now, having said that, if you want to write  something that is more flexible at runtime,   like, you can say, well, it's okay, if this field  is missing, I wanna default to this or that. You   can do that, but then at that point, you need to,  at least in Roc's case, you would need to sort of,   I'm gonna use the term eject, you know, like,  translate the automatic thing that's happening   into, like, an actual, like, written-out schema,  like a decoder that you can then customize. So,   this is how we do it in Elm, is, like, it's  always done that way, which makes it very easy   to customize. Another nice thing about that  is, if you have it all written out, that it   means that if you wanna change your variable  names or something like that, you can do that   without worrying that you're accidentally causing  a regression in the decoding, which, you know,   hopefully, a test catches, but it might not. But then again, there's another trade-off there,  

which is that when you have it all written  out, it becomes a little bit more brittle to   internal changes. Like, so if I need to, like,  you know, add a field somewhere that happens to   be in a data structure that's used quite often  throughout this thing, I have to go through and   change it in a bunch of different places. And  so, certain things, like being synchronized,   either can be a source of bugs or can be a source  of convenience, and it's just an innate trade-off.   But yeah, if you do sort of eject the decoder,  and have it all written out, then you can be a lot   more flexible in terms of, if the runtime value is  this or that, or this is missing but that's not,   or, you know, I can say, well, I'll accept  any of these three names here, and I'll just   internally convert them to the same thing. So,  a lot more flexibility if you go that route. I feel like we've gone quite deep into some random  part of the language, which is, like, parsing   responses. But let's maybe chunk it up a bit. So,  what are you excited about in terms of features? 

For Roc? For Roc, yeah.  Great question. I mean, that is, to be fair, one  of the things I'm excited about. So, in general,   like, it's 100% type inference, so you can, you  know, you don't need to write any type annotations   if you don't want to. I mentioned that, like,  you know, it's fast, friendly, functional. So, in   terms of fast, the thing that I'm excited about,  there are two parts to that, one is really fast   compile time. So, we've spent a lot of time doing  that. We still have a number of projects to go,   but one of the things that, I mean, you mentioned,  like, TDD earlier, one of my hypotheses for why   there's a really strong testing culture in Ruby,  like, for example, and I think in Python also,   more so than I've seen in, like, type-checked  languages, I think part of the reason for that   is that you get a really fast feedback loop when  you have a dynamic language for two reasons. One  

is that there's no compile step. So, we wanna just  make our compiler so fast that you don't notice   it. But the other part of that is that, from a  workflows perspective, if I am writing a test in   Ruby, or let's say I've got a bunch of tests, and  I'm refactoring something, all my tests go red,   because, you know, I've changed this thing. Okay,  fine. Well, I can go and fix them one at a time.   I can go, like, change my implementation, fix  whatever, and then they go green one at a time.   Now, in a type-checked language, the norm, today,  is that I make my changes and I get a bunch of   type errors, and all of my tests are not runnable  anymore, until I fix every single one of the type   errors. So, the whole, like, make the tests green  one at a time by fixing implementation details,  

that workflow is inaccessible until you've  fixed every single one of the type errors.  But quite often, I don't wanna do that.  I wanna go through and, like, you know,   change the behavior one at a time, and make sure  that the new behavior actually passes all the   tests. And then maybe there's still some leftover  type errors because I changed the interface,   but those are just going through and updating, you  know, callers to do the new thing. In isolation,   I still just wanna just do this thing. Or, let's  say I'm trying something out because I think the   new implementation will have better performance,  or I'm trying something out and I just wanna see   how it feels to use it. Again, I don't wanna have  to go and fix every single implication of that.  

So, this gets me to another thing that I'm excited  about, which is that we've designed the compiler,   it doesn't 100% work this way yet, but we've at  least designed it, and, you know, will get to a   point where this does work this way, where, the  compiler always type checks your code, and always   tells you about problems, but they don't block  you. So you can still run even, if it's got type   errors or naming errors or whatever. So, the  idea is that, much like a dynamic language,   you still have those workflows available. So, I wanna get that same experience. This is   always something I missed when going from dynamic  to statically-typed languages, is that workflow   of, like, I can always run my tests, no matter  what's going on. And I can see which ones fail,   and, you know, if they have a type mismatch,  fine, that's a failure. Failed test. But only   if that affects that test. If the type mismatch is  some distant part of the codebase, I don't wanna  

see that. Don't block me. Just let me run these  tests, and I'll come back to that later. So, that   requires sort of building the whole compiler with  that in mind. And when I say it'

2024-02-24 23:53

Show Video

Other news