Why GIS Needs the Cloud (And What’s Next)
And I'm like, okay, so we're, the way we're doing this is we're asking everybody to download a gigabyte of data and then throw away 60, 70% of it. - Did you know that for years, geospatial professionals have been forced to download massive data sets only to throw out most of the data. It's an outdated, inefficient process, and it's holding back the future of geospatial analysis. But a major shift is happening. Cloud-native geospatial is revolutionizing how we store, access, and compute geospatial data, making it faster, more scalable, and accessible to everyone. Today, I'm joined by Jed Sundenwald, executive director of Radiant Earth, to talk about how this movement started, where it's headed, and why it matters for anyone working with spatial data.
If you wanna stay ahead of the curb, stay tuned and be sure to follow all the way through for some important information about the CNG Forum. Hey, everyone, welcome again to another episode. I'm happy to have Jed Sundenwald here.
He is the executive director of Radiant Earth, and we're gonna talk about a topic that's near and dear to my heart, which is cloud-native geospatial, and some of the things that are happening within that community. But Jed, I just wanted to say thank you, first of all. And yeah, tell me a little bit about kind of what you're up to, and what things have been going on. - Sure, yeah, so yeah, no, thank you. I mean, thanks for having me on here. This is great.
Awesome to see how you do this. So yeah, so I'm executive director of Radiant Earth, we're a nonprofit. Our mission, our stated mission is to increase shared understanding of our world through community-led efforts that make data easier to access and use.
And as you mentioned, one of those things is what we call the Cloud Native Geospatial Forum, where we sort of see our job as creating an environment, like a forum, where geospatial data users, you know, the people who like, it is their job to fight with data all day and figure out how to work with increasingly large volumes of data to come together in a vendor-neutral space and just talk about what works and doesn't work. So yeah, that's, and I wanna emphasize also, you know, we're very grateful to you for being on the editorial board of the forum. Like I said, you know, Radiant Earth is all about community-led efforts to make data easier to access and use. And so if you go to the Cloud Native Geo website, cloudnativegeo.org, the only faces you really see on there are our editorial board, and it's a group of amazing, generous people who are leaders in the community that are willing to, you know, share their knowledge with everyone else.
So thank you for that. - Yeah, well, yeah. I mean, I thank you, I mean, obviously, for the opportunity. It's been cool to see that group come together. I mean, there's people that I knew, people I didn't know, but all kind of leading a lot of different things. So it's pretty interesting once you see some of the emails get going about some different topics and the ideas rolling.
Like it's, that's been fun to get going. But let's kind of rewind a little bit to talk about, obviously, you know, CNG has kind of evolved a bit. And obviously I want to get to the event that's coming up this year, which I think is gonna be important for a lot of people.
But how did CNG evolve to the point kind of where it's at today? - Yeah, so I mean, if we go back, like the whole story, it's actually a 10 year story now. I just realized this recently. So it was about 10 years ago.
I mean, 10 years ago today, we were in the process of bringing Landsat data onto AWS. I used to run the open data program at And one of the first big things I did there was bring Landsat into S3. And at the time, you know, sort of the normal way of sharing raster data or like Landsat data was like, you just download a gigabyte TIFF file and then you do what you want with it and whatever. But like the first step is downloading a gigabyte of data, which is still a pain, you know, like that takes time. And in the process of bringing Landsat onto AWS, I asked everybody I could find who used Landsat data, said, "How much of this data do you actually use?" You know, you get 12 bands, like maybe 13 bands, there's, you know, there's a QA band in there.
And I'm like, "How much of that do you use?" And they're like, like three or four bands maybe," you know. And I'm like, "Okay, so we're, the way we're doing this is we're asking everybody to download a gigabyte of data and then throw away like 60, 70% of it every time." I'm like, "I'm not gonna do that." And so I was really emphatic that we would have to create a pipeline where we unpack the tarballs and just hosted the TIFFs individually on S3. And we worked really closely with Planet on this, with Frank Warmerdam, Amit Kapadia at Planet, and then Peter Becker at Esri was also very involved. And it was Peter Becker who was like, "You should do internal tiling on these TIFFs."
And full disclosure, like I had no idea what he was talking about. Like I was not, I was not also like a geospatial guy at the time. And long story short is we put internal tiling and overviews in the bucket with the TIFF files that we hosted. And that was the, what turned out to be like sort of the prototype of the Cloud Optimized GeoTIFF. Chris Holmes then sort of took the concept, gave it the name Cloud Optimized GeoTIFF and evangelized it. And now it's become, now it's like the standard way that you host Landsat data.
But that was, I think, really where it all started, where people, where we were in an interesting position where, you know, AWS people sort of pay attention to what AWS does. And we put a ton of data on S3 and it was immediately apparent to people, it's like, "Wow, this is a much better way to work with the data." And we can build applications in real time on the web. And that led to, you know, eventually Stack, the creation of Stack, the metadata specification for sharing lots of data in object stores.
And an entire community emerged around this sort of thing. And so now we find ourselves in this position where we're like, "Okay, how do we just keep that going?" And really the answer we've come to is like, just getting people together who like to work on stuff and having them talk to each other and see what they come up with. So the Cloud Native Geospatial Forum is just kind of an attempt at formalizing that with a very light touch, which is just like, can we repeat this process of getting people together to tack on things together? - Yeah, yeah. - That's the whole story. - Yeah, I'm excited to see kind of what comes out of some of the, where the conference goes.
I think, you know, two things that I've noticed about when I first started really talking about and using some of the Cloud Native formats is, you know, number one is, you know, it kind it felt like, oh, it's only cloud and oh, it's only big data, right? And that was like, I'm trying to like, kind of reposition that a little bit because, while yes, it can scale up significantly to large volume, it doesn't necessarily have to. And there's still a significant benefit in terms of simplicity and even speed, just if I'm doing this on my computer alone, you know, and there's a lot that you can do there. The second bit is also kind of, okay, what's, now that we have, I think, a pretty robust ecosystem in place for storage, what does the compute layer start to look like? Which is kind of the other component when we talk about Cloud Native, the broader Cloud Native computing, what does that look like as well? So, I mean, I think those two pieces are interesting. And I think, ones we'll start to kind of see a little bit, you know, come out of the conference, but I mean, sorry, the forum, I should say. - Yeah, yeah, yeah.
Well, it's the forum conference. - Yes, yeah. Well, let's talk a little bit about the forum because I think it's the first time that this has happened, right? - That's right, this is our first in-person event that we've ever done like this.
We did a virtual event November. There are YouTube videos of that somewhere. I should like figure out how to like send that around. But yeah, we have YouTube videos from the virtual event we did November last year. And now, yeah, we're doing our first in-person conference at Snowbird in Utah, April 30th to May 2nd. - Nice, nice.
- And cloudnativegeo.org/utah and you'll go to the website. - Awesome. What's gonna be there? What can people kind of expect? What's the goals, outcomes? Yeah. - Yeah, so the way that we're kind of like aiming to do this, I mean, we can't really control the mix of people who are buying tickets, but the conference is very much aimed at practitioners. So in like sort of my fantasy is that we would have about like 65% of the audience be geospatial data practitioners, builders, developers.
Like I said before, like people who like, it is their job to touch data, fight with it and build tools to work with it. Maybe about 20% are newcomers, students, young people, and then also people that have been working in I would say more traditional GIS, but they have sort of reached the limits of desktop-based software and they're looking for what's next. So about 20% that, and then I, why did I do this to myself? I have to do this math. I mean, I think the remaining 15% are more like sort of executive types, people with purchasing power and decision-making at sort of like higher levels to have conversations around data licensing, business models, policy, things like that, that need to happen. - Yeah, yeah. So, sorry, go ahead.
- You know, I was just gonna say, and I was gonna say, like in terms of like how things are shaping up though, like it's been really remarkable to see the response. I mean, two of the first people to submit ideas for presentations were from the Flemish Marine Institute or like the Flanders Marine Institute in Brussels. And then like the government of South Australia, which like, I don't know what the, I didn't know South Australia was a thing. And so we got people from all over the world wanting to fly in and talk about what they're doing, but we've got the World Bank, the New York Times, NOAA, a lot of the usual suspects, Development Seed, Element 84 that are gonna come out, but it's a good conference. - Yeah, I mean, that's interesting because I think there's been, you know, I talk to and hear a lot from, you know, folks working, well, folks both working in and with the data.
And there is kind of this like, I get a lot of questions, where should I be learning about this cloud native stuff? Right, you know, and I know there's been like, you know, a lot of the folks on the editorial board have tutorials spread across the ether of the internet, but I think it's gonna be cool to bring that together in one place. And actually you can pretty much advance and see how not only learn it, but how it's kind of coming together in real time. But then I really liked the added part because I get a lot of questions about, well, how can I integrate this into what I'm doing and start to advance my geospatial data stack to work in this way? And there's a lot of folks that are building both the teams and the skills and working on that. And I'm pretty hopeful that this will be a good place to start to learn that from a really cool mix of folks doing this type of stuff, so.
- Well, yeah, I mean, we hope so too. Like I said, like we've never done this before. And, you know, one of the big tracks of the conference is the on-ramp to cloud native geospatial. And we have some good workshops being lined up for that.
But yeah, this is something I'm like really eager to see if we can pull off and like do some surveys after to be like, were we able to educate people and grow our community? I mean, I think you've probably perceived this as well. Our community is like, we all know each other. Like, you know, we all see each other at the same conferences.
We're all friends. It's great. We love each other.
I'm not mad about it, but I'm like, we need to grow this community on purpose. And so that's gonna be a big part of the goal of the event. - Yeah, yeah. And I think in-person events is starting to, I mean, really become, I mean, it's not that it was never, it didn't go away. It just obviously hit pause for a few years, but I think, you know, people have that kind of energy. And I think there's a ton of energy around using them to really learn, right? You know, it's not gonna be sitting and listening, but there's like a ton of, you know, response to that.
Like we just had the, we were at the geospatial risk summit and there was, you know, a day's worth of workshops, you know, people writing code all day, you know? And I think like that energy is gonna be really cool. Plus it's in a pretty cool place. Utah is one of four states I haven't been to, I'll be honest. And so like, you know, tell me about Snowbird 'cause I have no idea.
- Oh, I mean, it's spectacular. Look, it's very funny that we're doing it in Utah 'cause I was born in Salt Lake City and my family lives there, you know, my mom and my siblings. But it was not my idea to do it in Utah. It was Michelle on our team who was like, "I think we should do it in Utah." And then one of our board members also recommended it.
And I was like, "Well, that actually sounds like a great idea." And so we found out that, you know, Snowbird does, they've got like a phenomenal event space. I've gone and visited it. It's beautiful up in the canyon, not too far from Salt Lake.
And it kind of like, it just, it really seems to be kind of like a sweet spot where you've got a really good airport that's easy for most people to get to and not a huge drive from that airport to get up into the mountains and be somewhere beautiful. And working with Michelle as we were sort of designing the conference, that was one of the first things we really agreed on is that like, we got to figure out how to bring people somewhere that's beautiful, like where they really want to be. And yeah, I don't know, maybe it's a little bit too hippie-dippie, but I'm like, we're geospatial people, like let's like love the earth a little bit and be somewhere that's just like great. So I'm really excited about it. - I think I've had this conversation with a few folks at whereabouts and we're like, there's gotta be a strong correlation to like mountains, hiking, skiing, and folks in geospatial.
Like it just seems like everyone's like, oh, we're going skiing, we're going skiing, we're going skiing. And like, I, you know, so I'm like, okay, I think it makes sense. - Yeah. You know, I mean, actually I've got pretty strong feelings about some of this stuff. Like, I don't know if you're familiar with like SatCamp.
So SatCamp is another event that's been happening over the past few years, which is like, hey, let's go like gravel biking and, you know, trail running and all this sort of stuff. And which, you know, I'm down. I love trail running.
But this is actually a thing that concerns me a little bit about our communities being maybe a little too like, like bro-y in that way, or, you know, it's not just bros. There are a lot of, fortunately we have a lot of gender diversity as well in the community, but like it's still, I think a lot of people are intimidated. They're like, look, I don't know what a gravel bike even is, you know, or like, I've never run a mile in my life.
Like I'm not, I'm not welcome there. And so I'm kind of hoping we can do a little bit of both. We can bring people to the mountains. They're going to be in a really nice hotel and they'd be very comfortable. And if they want to, maybe they can go on a hike. - Right.
- And so like just have, yeah, have both. I mean, I'm sure there's some people who are going to be like back country skiing or something like that. That's not for everybody.
But I mean, but I'm also serious about this. When I talk about bringing people in the community, new people in the community, this is maybe just me fantasizing a little bit, but like I do hope that someone comes and they also, they come to the conference, they have a time and they go on a hike for the first time in their life. And that's a good impact, you know.
- Yeah. - Well, I'm daydreaming here, but. - I think, no, it's fair. I mean, I, yeah, I'm very much, you know, I'm trying to blink. I totally lost my train of thought, but good news, we can edit this out.
(both laughing) No, mean, I think that having that space, I mean, regardless, you're out in nature, it's beautiful, but you can also have the option, you know, to learn different things. I mean, you know, not just from the core of what you're doing day to day, but also expand those things, but also kind of expand some different things and learn from the space you're in as well, which is, I mean, honestly. - Yeah.
- It's pretty much what we're doing every day is learning about the spaces we live in. So that makes sense. Yeah, awesome. Cool, so people who are interested, and I hope you are interested after we had this discussion, what should they do? Where should they go? What's kind of the plan from here? - Sure, so I mean, the easy thing is, you know, cloudnativegeo.org,
there's a big yellow button to like learn about the conference, if you're interested in that. You can also become a member of Cloud Native Geo. So the way the forum works, and what we're trying to do here is figure out, can we get the community to fund us, to keep it going operationally, so that we can, the mission of the organization is to make geospatial data easier to access and use. And you'll notice it's not like, the mission isn't to like get it into the cloud or get people to adopt the cloud. We even, we refer to the forum as CNG.
We just call it CNG, kind of to get cloud out of people's mouths for the same reasons you said before, where people are like, oh, do I have to use the cloud for this? We're like, no, what this is really about is just making geospatial data easier to access and use. And so you can join the forum, you can have your company join the forum, fund us so that we can make these things happen. And the goal is to make as much free content available as possible.
The conference isn't free, 'cause it takes money to put on a conference, but it's gonna produce a lot of potentially videos and other blog posts and follow up tutorial content that we wanna make available for free. And I feel really strongly about this as the internet gets kind of just craftier and more covered with ads and spam and stuff like that. But I want people all over the world to be like, I'm interested in Geo Parquet, or I'm interested in whatever the new thing is that they're interested in.
They search for it, they end up on cloudnativegeo.org, and they're like, wow, this is a site that loads quickly, has no ads popping up all over the place, and I'm learning from. And that's the kind of brand we're trying to create.
So go to cloudnativegeo.org, join if you can, and come to the conference if you can. The only other thing I'll say is that the hotel is awesome, and we have an amazing rate. So it's a great rate. So that's all. Normally, these rooms are quite expensive.
I don't know how we pulled this off. I think Luis and our team did some magic. - All right, well, that should be incentive for everyone.
Is there like an, sorry, we'll cut this out. Was there an early bird window or something like that? - Oh yeah. Yes, I think we had super early bird ticket prices. Those are gone.
I think we're still in the normal early bird phase. So get your tickets now. And then also, if you wanna do a group ticket buy, let me know. So you can write us too. You could write me directly at jed@radiant .earth. And if you wanna talk about group ticket sales.
- Awesome, very cool. Well, yeah, I'm looking forward to it. I, you know, hope crossing off one more state that, you know, I'll be down to three then. So I have, you know, I'm close, but yeah. - I'm impressed.
I'm not anywhere near that, I don't think. - My first job out of college was, it required a lot of traveling. So I quickly covered a lot of territory and it's down to Alaska, Hawaii, Utah, and Wyoming. And-- - Those are like the four of the best states. - I know. - They're amazing.
I've never been to Alaska, but like, - Yeah. - There's no doubt in my mind, it's amazing. - I know, it has to be. So I'm gonna, you know, check off one. It's gonna be a nice place.
And I get to do it with people talking about cloud native geospatial, which is gonna be exciting. So yeah, cool. Yeah, I'll put a bunch of links down in the show notes. So you can check that out, get registered, sign up for the forum and get some more information there. But Jed, yeah, thank you again for taking some time out of your busy schedule.
I know you're working on a lot of very important stuff. I'm sure you'll talk more about that soon, but yeah, I'm excited and looking forward to it. - Thank you.
2025-02-22 15:37
