James: From Agerpoint I'm James Kotecki and this is The Point Cloud. My guest is the Director of NASA Harvest, the National Aeronautics and Space Administration's global program to promote food security and agriculture with satellite observations. Dr. Inal Becker Reshef welcome to the show. Inbal: Thank you.
James: How'd I do with that introduction? Is that how uh, you introduce NASA Harvest? When you meet someone at a cocktail party and tell them your cool. Inbal: Um, no, I think you did a much more dramatic and better job than I would make sure. I'll take around with you to the cocktail party. James: Well, NASA's all about, uh, dramatic, uh, sweeping vistas of space and the planet, and that's what we're here to talk about.
But before we get into all of that, uh, just wanna know what, what drew you to this work? Where does this spark for the passion for satellite observation, NASA harvest? Where does that come from for you? Inbal: It's probably a combination of my. Experiences and on a personal level and, and then kind of as I developed professionally, um, I think I always knew that I wanted to make some kind of a positive impact. I grew up moving around every year or two. I lived on three different continents, saw a lot of different ways of living in, and also a lot of different challenges. And I started studying, um, soil science and agriculture and. Kind of by chance got into remote sensing and started to recognize how important and how impactful that kind of technology can be. And I think it was a natural evolution,
um, working with satellite data and agriculture and, and recognizing the impact that we can have in so much still untapped potential that we have using satellite data to make a difference. I know it sounds kind of cheesy. James: No, but it's, it's, it's so cool what you're doing. So na, NASA Harvest. Um, tell me the origin story then of NASA Harvest. Are you the first director of this? Is this a program that you helped to start? Where did this actually come from? Inbal: That's a really good question. So we launched NASA Harvest officially in
2017, but the roots for that program go a lot further back. Um, I joined a team at University of Maryland that was working with U S D A in in NASA back in mid two thousands, um, who was already working on how do we better integrate satellite data into the U S D A foreign, an agricultural. Agricultural monitoring system, how do we make satellite data more accessible and kinda not needing to be a remote sensing specialist to be able to use that kind of data. Um, and in doing that, we recognized that there were a lot of different countries around the world trying to do very similar things. And,
and this was led by somebody called, uh, professor Chris Justice, who I worked, uh, with very closely and eventually brought together the international community. Under what's now called glam, it's a G 20 initiative. At the same time, NASA recognized that within its applied sciences program, and that's where NASA Harvest sits, uh, didn't have a specific application area focused on agriculture.
And so over the years, they made the case to. Put together a program that would be specifically focused on agriculture. And they wanted to look at a different framework or a different model for doing that than they traditionally had. And so initially, NASA Harvest was an experiment. They competed out, uh, the program. Um, and the objective to doing, setting it up in, in this way was so that it could have a, a perhaps, A bigger impact so we could partner with governments more easily. We could par partner with private sector companies, humanitarian organizations,
because I actually don't sit inside of NASA headquarters, so I report into NASA headquarters. NASA Harvest is a, is a headquarters program, um, but it's actually run out of University of Maryland and that enabled us to be quite dynamic in terms of how we partner and, and, um, run the, the program. When I think about agriculture and space, if you say those two words to someone, they [00:04:00] might be thinking about growing a new strand of wheat in zero gravity or terraforming mars or something. Is that the reaction that you sometimes get when you start talking about this? Do people need to kind of reorient on the fact that No, actually what we're doing is we're using satellites to look at agriculture on earth, the most important planet that we. Yeah, that's right. Yes. But I would say less and less, right? So more and more people are aware that we're using satellite data in, in a lot of different ways for monitoring agriculture on, um, on our planet, on on earth. I do of course, often get a question about, you know, can we grow food on the moon and,
um, or other planets? Uh, but, but yeah, I think there is some element of surprise for people initially when they secure NASA and agriculture. Um, but I think as. Area and, and field really are growing very rapidly. There's more and more familiarity with, um, with this and it makes more sense to people. [00:04:57] James: So how does this actually work [00:05:00] in practice? Can you give us just like some basic numbers here, just the basic framework for understanding this. I know that there are a lot of satellites orbiting the planet right now. How many of them can we use for agriculture? How many does NASA harvest actually use to, to look at agriculture? Do you share these satellites with other people? Are you just kind of tapping into satellites that are there doing other things a lot of the time? Like how? Like logistically, what is actually happening with NASA harvest? [00:05:26] Inbal: So we use a lot of different kinds of satellites, both in the public sector and in the private sector. Um, I think today they're around a thousand, maybe a little bit over a thousand Earth observing satellites that are orbiting the earth all the time, taking a lot of imagery.
We've seen a really rapid increase in satellites in the last few years. I think if you look just like five or six years back, there would've been around 400 satellites, um, that are earth observing specif. And one of the trends and why we see so many more of those is that there's been much increasing, kind of sending up these [00:06:00] fleets of satellites. Like you might have heard of a company called Planet for example, or others
that are working together and synchronous to ultimately enable us to have more frequent observations of the same spot all the time. Um, our primary. Data that we use have, have been the public open and, and free and open satellite data. So a lot of people might not recognize that a lot of the satellite data that's out there, um, by nasa, by the European Space Agency and other, uh, space agencies are actually free and open, recognizing that to have an impact. The fact that satellite data are free and open means that we can use them and, and utilize their, the insights from them a lot more than if we have to pay for those. But there
has become a really important role also for the commercial satellites, especially these small, um, satellites that are, uh, Lower cost than they would've been just even a few years ago. Uh, and, and also providing a lot of really valuable information. So on the Har NASA Harbor side, we are using a lot of different kinds of satellites that are, if [00:07:00] you think about, you know, they're monitoring the environment around us all the time. They're, they're, they're able to monitor, Almost every field or every field across the globe on an almost daily basis. And so what satellites can do is they can see kind of in the same wavelengths that you and I see in the visible range of the electromagnetic spectrum, but they can also see in a lot of other wavelengths. Um, and that means that we can see a lot of things that we can't see with the naked eye. And of course we have the vantage point
of space, and so we can see also a lot more at one time or monitor it essentially the entire earth. [00:07:32] James: So is it fair to say that you at NASA Harvest can basically see everything that humans are growing outdoors to eat every day? [00:07:46] Inbal: Close to that. Okay. So we can see a lot of those fields. I think a lot of what it is is being able to convert, you know, this tremendous amount of data into actual information, right? Into what is [00:08:00] each field growing, um, how is it developing? When was it planted, when was it harvested? Um, what kinds of, uh, management practices were applied, right? Like satellite data can give us a lot of insights about. But in order to be able to do that, that means we've gotta develop models, we've gotta have reference data from the ground that are training and and developing these models to turn that data into actual information. [00:08:23] James: Can you speak a bit more about the way that this kind of fits into the overall puzzle here? Because someone listening to this for the first time, I think, oh, we have total so satellite coverage of the earth. So as long as there's a plant growing outdoors game over, we can. We can see everything and all we have to do is run some models on it,
like, but you're talking about ground validation and other kinds of technologies. So where does, how do satellites fit in your mind in the overall picture of how we collectively, whether that's you at NASA harvest or just we as a species kind of figure out and understand what we're growing. [00:08:54] Inbal: Sure. So satellite data, um, you know, for one of the ways that we can, for example, identify a [00:09:00] specific crop type and, and start to map it, is it has different kinds of signatures, right? So if you want it, it has different signatures and different wavelengths in the visible versus the near infrared versus the shortwave infrared. But it also has a temporal signature that is very, very important, right? So it's planted, it starts to emerge, it grows, it will mature, it will sse, it will be harvested. And so also looking at that signature through time and through different wavelengths becomes very important for then developing models that we can apply to then classify, for example, entire country or the globe. But, I think, but what we need to do to be able to do that is that I need to be able to say, okay, this is what corn looks like, or This is what corn looks like in all possible different vari variations of it.
Train a model to then be able to pick that up, to understand that variability and to then be able to map it. And then crucially, we need to be able to validate that. Right. So it's not good enough that I can run a model and produce a map, um, that, you know, a model will [00:10:00] produce a result, but how good is that result? How accurate is that result? And that's something that we spend a lot of time, not just we, the whole remote sensing community. Spends a lot of time on, on being able to make sure that we understand the uncertainty about what we're producing, the types of information that we're producing. So having that connection
back reference to the ground to what we're actually monitoring is absolutely critical for being able to use satellite data. [00:10:23] James: Makes sense. Are, are there other limitations when it comes to satellite observation in terms of agriculture data that people might not be aware of. [00:10:33] Inbal: Sure there are lots of challenges and, and, and limitations, right? Going from having the, the necessary ground data to looking at very complex agricultural systems. You can imagine small holder systems that are very small fields that might have intercropping in the field. How do we monitor that, um, to being able to develop models that can forecast and predict yields accurately, uh, to being able to monitor d.
Agricultural practices. But when you start to be able to develop [00:11:00] those kinds of products and start to, um, understand what the uncertainty and those are, you can also start to answer really critical kinds of questions around, you know, what's driving yield, where are yield gaps? What, what, what kinds of practices might be driving those? And, and those sorts of things. Um, but there's also a challenge on the, the amounts of data that we have. How do we process it? How do we make that more equitable? How do we make sure that we are. Developing products that are actually gonna be used, right? So how do we make sure that the things that we do are user driven, um, and that are ultimately gonna be taken up? And that's a big focus for us on the NASA Harbor side. We try to always start with an end user, because otherwise, and, and that's really key for being able to bridge this kind of science and research into actually having impact in, in, in operations. Because if I'm gonna develop. And not having ever
talked to you in your, in a ministry of agriculture, why would you ever use this map? Right? So it's gotta be co-developed, it's gotta be. Um, and I think the other side of that is really thinking about how do we ultimately produce that capacity [00:12:00] to, um, in the institutions to be able to uptake and use these kinds of information and, and satellite systems. So there are a lot of technical and, and a lot of other kinds of, uh, challenges I think, in ultimately being able to reach the, the potential that satellite data offer. [00:12:15] James: So let's talk a bit more about how that plays out. So there's all
this data. NASA Harvest is hopefully putting some analysis and insight on top of that data. So tell me about who you're sharing that data with and what they are actually using it to do. Let's get into some of the users and the use cases here. [00:12:32] Inbal: And I should just say we are one player in this field. So there's a lot of different entities and, and organizations that are developing different satellite products and, and, and information. And maybe one thing I didn't say upfront was that NASA Harvest is a consortium. So we're over 50
different partners from across the world, from public and private sector organizations working together. Um, but the types of users that we have ranged from humanitarian organizations. Uh, un types of organizations to ministries of [00:13:00] agriculture or statistical agencies, uh, down to different kinds of private sector organizations, whether it's, um, we have some partnerships with insurance companies, some with farmer advisory, um, some with those that are really looking at the transition to sustainable agriculture and how do you better monitor or be able to, um, show the different practices, for example, that, that farmers are implementing at a broad and large scale. How do you understand what the implications of those can be? So we have, we, we purposely have tried to target really across the agricultural sector, um, different types of end users for ultimately being able to uptake and, and utilize better satellite information. [00:13:40] James: Something that I know you've spoken about before, and that's certainly top of mind right now, is the work that NASA Harvest has done with its observations of Ukraine and agriculture production. Uh, during and subsequent to the Russian invasion. So I'd love to hear that story again for our audience. Uh, I think it's really fascinating and meaningful [00:14:00] what
you were able to do there. [00:14:01] Inbal: I should probably say that I actually started a lot of my career in going into all this is working in Ukraine and I did my PhD on yield forecasting of wheat in Ukraine. So spend a lot of time, um, going to Ukraine and, and working with U S D A and with partners in Ukraine. And so developed a lot of relationships doing that. [00:14:19] James: Just to say,
I think Ukraine is. A a bread basket, right? It's a major producer of wheats, and so the ability for them to be agriculturally productive is very important, not just for Ukraine itself, but for the broader region. Is that fair to say? [00:14:34] Inbal: Absolutely, absolutely. Ukraine is an absolute critical bread basket. Um, it produces, I think prior to the word was exporting around 10% of global wheat exports, about over 40% of sunflower oil. A very important producer for corn as well. Um, and a lot of, I think the world Food Program. Uh, in terms of its wheat food aid was over 40% was [00:15:00] sourced out of Ukraine.
Um, and you have different countries that 80 to 90% of their imports of wheat, for example, depended on Ukraine. So very important country. Um, and. When the, when Russia invaded Ukraine, um, we immediately were in contact with the Ministry of Agriculture through connections and, and work relationships that we were already had, and they asked us to initially, um, support their analysis and assessment of what was happening under the occupied. Russian territories, temporarily occupied territories where they didn't really have a lot of information coming in anymore. And where satellite data actually was providing the only means for really being able to monitor that entire territory. And,
um, so we, the, one of the first questions they asked us was, well, what proportion of. Uh, Ukraine's crop plans were under Russian occupation and specifically what proportion of winter wheat, which is one of the main production crops in Ukraine, was under Russian occupation. Um, [00:16:00] We set up several different partnerships in order to be able to rapidly answer those questions. Uh, one of 'em was with the Institute for the Study of War that, as you probably know, provide every day an update on where that front line is and how that's moving.
Um, and one of the reasons this question was important too, is that most of the statistics mean going, going backwards in time are reporting on administrative level units. Right. So you, you, but the, the line of contact, or the o the occupation area, of course doesn't follow any kind of administrative boundary. And so there were no statistics to be able to look at really what proportion is, is that occupying [00:16:34] James: So the equivalence of a county or a state level production data. Right so- [00:16:38] Inbal: That's right. Exactly. [00:16:39] James: But, but of course if, if half of that, I'm just gonna say county or state or province is occupied and half isn't, then you don't have good data about how that could actually work. [00:16:47] Inbal: That's right. Right. And so
again, that's where satellite data can do that because we can monitor it at the pixel level and start to then come up with statistics on that. And so we, we partnered also with Planet, one of those [00:17:00] commercial companies that I, that I talked about earlier, that has a fleet of satellites, um, that gives daily observations. And that was very helpful for us because Ukraine can be very cloudy. And so it meant if a certain area didn't have a cloud. Over it. Then we can start to, in, in a sense of thinking about as a puzzle, every time there was a small area that didn't have clouds, you could puzzle that area together and mosaic that, um, every two weeks to get a, a pretty close to cloud free image. Um, I should say that also radar data can, can help and do the same. And we also use those kinds of data sets. But, so that was the first thing we, we did.
And we found that around 22 to 23% of Ukraine's cropland overall cropland was occupied by Russia. Um, and we found that around 29% of its wheat, uh, planted area was, was occupied by Russia. And then there were two really big uncertainties. Um, and there were a lot of assumptions. Being
talked about, which was how much of their spring planted area, right? So that's the corn, the sunflower, really [00:18:00] important crops would actually be able to be planted. Um, recognizing again that the winter wheat was planted prior to the war, so that area was gonna be planted. But again, a big uncertainty how much of that would actually be able to be harvested. And a lot of the numbers that were running out there was anywhere between 30 to 50%. Wouldn't be planted, wouldn't be harvested on, on the wheat side. Um, And so we started to map. So the first thing we did was, of course we mapped where the winter crops were, and then we started to follow all that area that could potentially be planted into spring crops.
And what we found was is that most everything looked like it was getting planted, including in the occupied territories. What we saw was that there was a concentration of. Area of, uh, fields that were not getting planted right along that line of contact. The, the, the, uh, occupation, um, front line, but for the rest, a lot of the fields or the majority of the fields were being planted. And ultimately it was around 88% of fields or of area in the occupied territories was [00:19:00] left, was planted, sorry, 88% was planted 12%. Approximately 12% was left unplanted in the occupied territories. Um, And so that was very different than all the other numbers that were being put out there. Their expectation was, was that a lot less would be planted. And then as the season progressed,
we could, uh, progressively map other crop types and, and look at what was going on. Um, we found that around 21% of the summer crops were under occupation. 14% of the winter rape seed. This is an important oil seed was under occupation. Um, And we ultimately then as the harvest season started to, to progress, we were monitoring then again the progression of, of the wheat harvest in particular. And again, what we saw was contrary to mo, what most of the assumptions were is that the fields were. Also all getting harvested. Um, and this really surprised us. We did a lot of due diligence to make sure that we had a lot
of validation. What the way to validate what we were doing was to zoom [00:20:00] in and look at the daily imagery at a three meter resolution and look at that, the actual date that we see the, the harvest happening. And um, so based on our maps, and I should say this before too, it's not enough to make a map out of remote sensing data that map we still need to. Uh, random stratified samples in order to be able to come up with actual statistics, with an uncertainty around them. And so we did that for the harvested area. Um, and again, what we found was, uh, the large majority of we had been harvested, including in the occupied territories. The area where we see most of the areas not being, um, not harvested, was again, around that front line, the, the, the line of contact.
And then what we could finally do is ultimately the, the number everybody's after is product. And so we took the, the, the yield numbers that were coming out of the Ministry of Agriculture for the Ukrainian controlled territories. We ran our own yield model on the occupied territories and where they didn't report. And so together [00:21:00] with our, um, area planted estimates,
the harvested proportion of those and our yield. Our combined yield numbers from our side and the ministry came up with, uh, with around 26.6 million tons of wheat that were harvested, and that was much higher than expected around close to 6 million tons above most leading estimates. Um, and 22% of that production was coming out of the occupied territories.
So what satellite data can do is it can show us that all that's getting harvest. What they can't do is tell us who's harvesting it. Where is that wheat right ending up? Is it getting exported? Where is it being stored? Right? Like that. Those are things we can't comment on or we can't see. Um, what we can see and what we do is really very much try to stick to the facts and what we can monitor and observe, provide the validation, provide the uncertainty around those numbers.
And so of course that brings out a very important point. What happens to that 5.8 million tons of wheat that we saw that was getting harvested off of the occupied territories? And one of the things we saw is that a lot of the different statistics also that are coming out there is, is not always very clear. Is it Talking [00:22:00] about all of Ukraine is talking about Ukraine minus Donbass and Crimea is it talking about? And so there's some variation in those statistics, but I do think it's important to really be transparent around what's being produced. It obviously translates into a tremendous. To Ukraine, that 22% or those,
um, you know, 5.8 million tons. I think we estimated around 1.2, 1.3 billion in losses just from the harvested wheat. Um, To Ukraine and we are working very closely with the Ministry of Agriculture. We are interacting with them very closely, providing these information and, and estimates. Um, and I should also say we've collaborated and coordinated with other entities that are working on remote sensing, including a project called the ISA World Cereals.
And so just to mention that, um, there has been some. Exchanges and, and collaborations, but it did really enable us to be able to provide information in areas that otherwise there is no information or very limited information coming out. Um, and I think that highlights really the value and importance of satellite data in, in these kinds of situations. Now in [00:23:00] Ukraine, this is due to war and conflict, but you can imagine. Um, an extreme event, right? A flood event or a drought or other. Um, and as we see more climate change, we know there's an increasing frequency and severity of extreme events. And again, I
think the value that satellite data can provide in terms of being very rapid, being able to look at a large spots of, of land becomes very important and, and can provide really valuable information. [00:23:24] James: You know, you mentioned that satellite data can take you, uh, much further than people might expect, but maybe not all the way towards kind of the actual truth of, you know, what's happening to that wheat. You can know how much is getting harvested. You can't exactly know who's harvesting or, or what they're doing with it. Do you sense that policymakers, the people who use this data, Understand that. Do you get people who come to you and wanna use satellite data as kind of the be all, end all, and, uh, are maybe disappointed that it can't tell them these, you know, quote deeper truths that we want to get to? Or are people kind of understanding the role that satellites play and that they're a [00:24:00] piece of a, of a bigger puzzle? [00:24:03] Inbal: I think for the most part, people understand that they are a piece of a, of a, you know, a broader, you know, they have a role to play in providing in information. I think on the remote sensing side,
we all have to be very careful about not over promising what the capabilities and what the accuracies are of, of the, of the models or the maps and, and the statistics. And we've always gotta be very careful to communicate uncertainties and, and what it can and cannot do. I mean, there were some interesting studies that we saw. That we're using satellite data actually to track ships and to track ships that go dark and then, you know, come back and that, you know, that's certainly not our expertise, but there was actually utilization of satellite data for, for looking at that and, and still is as, as well by other groups. [00:24:44] James: Interesting. You mentioned getting down to a three meter resolution with the work you were doing, uh, observing Ukraine. Is that the state of the art right now? And is there an expectation
about, you know, in the near term, just how good this [00:25:00] technology can get? Like what's the cusp of what you're almost able to do? What are you excited about being able to do with the next generation of whatever the technology? [00:25:08] Inbal: Yeah, and I think the resolution has to be suited to the, to the task and to the target. So fields in Ukraine are actually really large. We didn't need three meter resolution to do everything we, we did. It was helpful. For example, we delineated the field boundaries, right, of every field across Ukraine. And for that, that resolution was actually very helpful. Um, I think what's exciting about what's coming are also new sensors. More, there's hyperspectral, there's looking at more sensors that can provide more information, for example, on soil moisture, deeper into the soils and more, um, radar kinds of systems. But I think it's the combination
of technologies that's. That's very exciting and being driven then, and making sure that we're also investing in the systems, in the institutions that can ultimately use these and, and take those up. And so it's a combination of the cloud computing capabilities, the supercomputing [00:26:00] capabilities, the machine learning and AI methodologies and, and models, but also driven by and working closely with domain experts in, in, in different areas in agriculture and, and policy makers, and making sure that we're able to convert really. These large amounts of information into actually actionable data, actionable information. And I should say one other thing that, that I'm particularly excited about or. That I think is
really important. And one of the things, a new initiatives that we're setting up under Harvest is being able to set up a center for rapid response, agricultural rapid response, right? And so being able to rapidly look at these different kinds of situations, it might be uncertainty of a big producer. Exporting country, or it might be due to a flood or drought or other kind of a, of a climatic hazard or, or disaster, or it might be due to conflict, but being able to very rapidly be able to respond into policy needs or into decision makers. [00:27:00] And being able to harness this technology through the large network of experts that, that we've developed, I think can be very important. And also thinking about anticipatory, right? Like where might we see and, and building out different scenarios as well. Um, these
kinds of data can be very important. [00:27:15] James: There's a famous picture of the earth from space. I think it was one of the first pictures that was taken, and I think it's called the blue marble, where people looking at that picture had a new understanding and perspective. Their home planet maybe wanting to take a little bit better care of it. Does the work that you do have an emotional impact like that? Does it, does it change your relationship to, to sound cheesy how you see the world? [00:27:42] Inbal: Yeah, absolutely. I mean,
I think if you think about that blue marble, right? It was one recognizing that most of our earth is. Right. And like seeing this little dot of blue in, in the vast emptiness of nothing. This is our planet. This is the only place we have to live. And there there's a lot of [00:28:00] exciting space exploration, but this is our planet and um, And I think, you know, food security is one of the biggest challenges we face today. Climate
change, right? There are a lot of, and, and those are obviously very closely intertwined. There are a lot of massive challenges that we see today and I think, um, this work, I would say for many people is very personal. It feels like there is a lot we can do. We need to be able to measure, monitor, and be able to inform and understand. What are we doing? How do we make progress actually to our target goals? And, and, um, looking at how do we ultimately live in a more sustainable way on, on this planet? We don't have more land we can cultivate. We have a lot more people coming. How do we do that? And,
and I think data has to, decisions have to be driven by data and information. [00:28:45] James: Dr. Inbal Becker Rashef, director of NASA Harvest, thank you so much for joining us today on The Point Cloud. [00:28:51] Inbal: Thank you. It's been a pleasure. [00:28:54] James: Well, that is our show. Thanks so much for watching. Please subscribe. Rate us
five stars. Tell your friends and [00:29:00] join us on social media at Agerpoint, where the conversation continues from Agerpoint. I am James Kotecki and this is The Point Cloud.
2023-04-26