Álvaro Santos Andrés - From Farm to Cloud: Using DevOps to Optimise Agriculture Operations
[applause] Thank you. Oh, so many people here I think is. I know a lot of people, so thank you very much. Thank you for my talk "From Farm to Cloud."
I going to explain a little bit what can we do for the farmers. First of all, who I am. I'm from Madrid, Spain. Many people who is there, but, well, I've been working in IT for 15 years. I'm not a DevOps architect, but I know.
I've been playing with their tools like Terraform, Ansible, many pretty cool tools. I'm going to explain a little bit what I learned and why agriculture is happy to have us. My family have farms also in our city in Spain, but, well, their disclaimer is basically I know very little about agriculture but I know how we can help them. That's what I'm going to do. First of all is why agriculture needs us, why the farmers, why the companies who work in agriculture need us.
Second is how we implement DevOps. This is very general but maybe could help you. Finally, what we learn about DevOps in agriculture because well, it's not as an industry that in banking and so on. This is great. I think many people don't know about this, but right now in the world we are a lot of people, more than 2 billion people, I don't know how much, but basically right now we need food.
A lot of food, and every year is more. Countries like China or India, it's increasing out the population and basically, we need more fields, but the problems with the climate change basically we need, let's say, space to plant. It's having a lot of pressure on agriculture, and it's very tricky.
That's why we are here, the IT people to help these guys. First of all, which is my company? My company is Syngenta. This is one of the biggest agriculture companies.
Is the first one in crop protection. The third one in seeds. We provide seeds to the farmers and also we have a lot of presence in China. Another cool thing about that is basically we are moving to digital. Basically, we are providing many digital services for the farmers. That is in the run of Cropwise, but there, if you want more information there you go to cropwise.com.
The need for cloud technology. Why agriculture needs us a lot. When I mean a lot is many.
Many times they need our help. The first thing is just to point is that basically, they don't care about us. They know what to do. Basically, for them, it is that, okay, I don't know. Who knows something about agriculture here? Something. Well, more than me. More than me that's super, that's for sure.
Basically for me, if I go to my family town, they ask me to fix their, I don't know, their computer, their phone and this kind of thing. They didn't know what we do here basically and why. Because basically, they're too busy for this staff. They have to plant crops, they have to manage cows, and many things. The second thing is that basically, they prefer to spend time in their things and not in a computer.
I know that basically is very typical, okay, outside our industry. The third thing is basically for them, when we say about cloud, they only think about weather forecast. [laughter] That's pretty bad. These things happens. The last thing is about, well, they prefer to speak with plants than with ChatGPT. That's for sure.
When I go to my home town and that guy like the [?]. Apart from this what, can we do to help farmers? In agriculture, I think in the last 5 years or 10 years. In all the industry basically, they have emerged a lot of data-driven services.
Basically, we are doing that. Basically, we are gathering a lot of data from the farmers, from the fields, for the weather, for many things, and basically, we are creating services around that data. Basically with that, we can help the farmers just to increase their profitability. Basically, that means more food for everyone.
Some examples that we can say about this is basically, as I said before, we can increase the production using fewer resources and less land because we can, I don't know, help them just to say when to plant or what to plant. We also give advanced recommendations like as I said, for I know specific seeds that basically are better for this field, for this specific part of UK, and so on. Also, we have a lot of data that came from separate satellites. We can give in real time monitoring of their fields so they don't have to go to the field and see what is happening.
We can show them in the device meanwhile they are in their office. The final thing is this very, I'm working on it right now, is basically we have a machine learning models that basically can identify diseases in the crops. These are various samples as I said.
Well, now we said about why agriculture needs the IT industry and why the cloud is the right place to do it. Because basically for getting the services, we need the latest technologies that basically give us the best performance. As always, working with data is sometimes overwhelming with maybe about big data.
This kind of thing. Basically, we need more sources, we need more space, we need more power in general. Basically, we need the cloud. Basically, the cloud providers is the place for that. Basically, before we cannot do that is basically when they store unlimited data.
Before this maybe we can store, I don't know, one year. Right now we can store 20, 30 years of data. We can go to the past and see what happens 10 years ago. That thing was impossible.
Another thing is basically we can experiment and use the latest artificial intelligence algorithms like ChatGPT. For example, some months ago, we have a hackathon, using ChatGPT. Basically, we figured out ways in order to use this algorithm to help farmers.
This is something that basically is in the cloud. It's very difficult for me and in my laptop to create something similar where there are no questions there, open source project, but it's not easy for a small company to use these latest technologies. Amazon basically is, or Azure is very easy for us to work with these services basically is plug and play. The final thing is about, for example, another example is the automation of a machinery. This wastes a lot of power because basically you are receiving a database of data for a specific drug, a specific machinery, and basically with the old computers basically in your company, it's very difficult, but with the cloud computing, you can scale. Sure. It's a challenge before, but right now it's easy for doing that.
Basically, why Syngenta was an early adopter because basically you want to be the leader and you want to be in the forefront of R&D. Basically you have to spend money in technology. Basically, we were here in Amazon, I think since 2017. I think we have had to be in the top basically of our companies. I don't know our competitors, but in our case, basically, we were in Amazon for many things. All the things we store basically go to the cloud or to Amazon, and basically, this provides us, as you may know basically scalability, flexibility.
We can, I don't know, more or less maintain the cost and this kind of things. A big thing that I think I had to mention is that go to the market faster. This is very important as I mentioned before working with ChatGPT is good.
You can do it in your home, but basically, a company is not the same. If you work with Amazon that provides us the services that service very easy for you. You can innovate much easier and faster and that's always good. That's mostly what I know about agriculture What we can do now, I think for this audience, is much easier. Some things we know about DevOps. Okay, so this is about how we implement the DevOps.
Maybe the fair about your companies or projects, but I think it's good to know. Why need DevOps for the cloud? That's the first thing. Basically, I'm a developer, as I said a data solution architect.
We don't have enough skills to know what has to do with infrastructure, how we deploy to the Kubernetes, as the previous presenter says. It's very difficult for us to do these kinds of things, that basically we're saying today. Okay, so this is the first thing. The second thing is that it's not-- so DevOps is not only about putting this in production and running it.
It's about designing and building a thing that basically works in the time, so you have to manage the services, you have to monitor them, you have to give the proper support and well, for me, this is not easy. These tools like Github, Terraform, Ansible, Kubernetes, these kind of things. Basically, these things are tools that basically help us, but basically are not easy to maintain, and you need someone who know well about that, and it's hard. DevOps is going to help that part. The final thing is about DevSecOps, about security, so we need specific people who know about security and basically, I don't know about those things. I don't have to care about them, but someone will help me.
Basically, that's why we need DevOps. I split it into two parts. Basically, the bad way and the good way.
The first thing we do is in the bad way. Basically, what we did is using the old-fashioned way of doing things, so we have a specific department for DevOps. Basically, we resist to this change about DevOps because DevOps is not only about tools. It's not only about technology, it's about doing things another way, so basically, you have a department who is DevOps. We ask them, "Okay, you have to do it. Raise a ticket." I put what I need, they deliver it to me, but it's like outsourcing.
Sometimes it works, but in the end, you are going to have friction, you are going to have problems, and basically you work in silos and it's not good. Another thing is about experience, about tools, infrastructure as code automation, these kinds of things. My team doesn't know about this you have to go to the DevOps and ask for them.
Maybe they don't know what are the specific things about your project and sometimes they don't atomize everything, and the last one is about feedback. If you work in different teams, you have to speak, I don't know, by mail, by chat and this feedback is not good enough. Sometimes it's not fast and well, it's not easy to improve over time, and maybe you know about the Spotify model. It's a new way of doing things. We started this some years ago. It's about how we can do this in an agile way.
Instead of having different teams, what we do is, we have a DevOps guy we are going to put in our team. Includes in all of our meetings, I don't know what the rest of the team, like a frontend developer, backend developer, DevOps guy will help me with everything. Basically, what is help, what is going to happen? Everything is going to run much faster.
I'm going to have many time, I'm going to focus in the specific part that my problem is facing. For example, I don't know my service needs more automation because I have to deploy many times if my DevOps guy is in my team, we can do it much faster. Also, we have this kind of thing that I know really know a lot about guilds, chapters, these kind of things, but what maybe someone from agile will know more about that. Perfect. This is something I didn't want to have a lot of time. Maybe I can spend more time in lessons.
I'm going to say something about something I understood about the farm working with agriculture, I think, about DevOps. Okay, maybe I'm going to have some problems saying this, but well disclaimer, basically, this is my experience. This is what I did. Maybe you don't understand what I say other times not, but I think it's important to know, so the first thing is to be open, so what means to be open? This is very important for agriculture if you go for other industries, maybe it's different, but agriculture basically, if you go to a farmer and say, "Okay, I want to use this service, I want to pay for an app," basically he's going to tell you, "Okay, this season is going to be very difficult because we'll have some problems.
Come next year, and basically we can start together and see if it works." What that means basically is a long-term thing, and what that means for us, basically that we need something, but basically it's going to be there for a long term for years. We're not saying about months or one year, two years.
We have to maintain the same service for 10 years and basically be open is going to help us for that. How come we be open? Being open is not only about open source, it is basically using open standards. This means, for example, use HTML with we saw before about Kubernetes for gestation. Using this kind of open standards will help us to be in the industry for 10 years. We don't have to change our product. The second thing is about how we build things.
For me at least my case using open source libraries, like for example, React, for example, Spring Boot, or other things was that has a lot of contributors will help me to be on the field for many time. I don't remember how much, but I think the Spring framework has been for 15 years. Postgres for example, 50 years. I don't know.
Kubernetes, I think they have, okay, sorry, 10 years, more than 10 years, so this is not easy. This is not easy. If you go for a service, we can go to Google and see how is the timeline of Google with the products. These are kind of things that I haven't tested a lot about Google Cloud.
It's maybe tomorrow I want to have that service in Google. Maybe yes, but maybe not, I'm not sure, but with this kind of open source, with this open mind, it's much difficult to have, sorry. Another thing that I learned is about if you go to the cloud, try to use the open services that is based on open technologies. I working with data, I used to work, for example, with Kinesis data streams, just to have data.
It's very good technology, it's cheap, but what happens if I have to leave Amazon, I don't know, I need China, for example, Amazon closed, what can I do? Maybe it happens that I can make a small change and it's easy for me, but I don't know, maybe it's not so, I don't know. If you go, for example, to have a database, go to Postgres, you have the Postgres services in all the cloud providers like RDS, you have the same in Azure, SQL, so try to use that. The final thing is if you go to open source project, please go and use someone that is for a reputable open foundation, like Apache or Linux. This is very important.
The next slide is going to give you an example. Okay, sorry, please [?] don't see this part. Sorry, I didn't know you come here as a sponsor, but this happens very readily about HashiCorp. Many of you know about Terraform.
I know something about Terraform, but when I see this, I was very disappointed because for me, HashiCorp was the kind of company that is open source, we thought was an example, but I don't know, one month ago we saw that basically, they close the license. In my case, basically, I have to figure out, "Okay, my company gives great produce for the farmers, but I'm right with the license. Maybe yes, maybe not."
In the case of this BSL license, it's very big, so could be. In my case, what I'm going to do, I'm not really sure if I'm going to work. We'll have to see about this.
OpenTofu? Holy shit, tofu. I don't know. Do you know half of the name? Holy shit. [laughter] OpenTofu is like, I don't know, I like meat.
Why I have to eat now tofu? I don't know, whatever. The same thing happens with Elastic, the same thing with Mongo. In this case, we'll just say Mongo is like this maybe. They closed many times ago but no one cares about Mongo.
You have better solutions, like Postgres. The same with Elastic. I don't know. You have to be very careful. Be very careful, be open, but at the same time, you have to take care of these kind of things. The second thing is about FinOps.
This is very important. If you see the margins of the agriculture of the farmers, you ask them how much money they have in benefits. It's very small. When you go and create a product for them, you have to be very careful with the pricing. Sometimes, I startup basically with this wonderful product. Basically, they're losing a lot of money, a lot of money, a lot of money.
Sometimes it's because, I don't know, they have a lot of people, other times, it's because of the infrastructure. When you create services, you have to have a finance mindset. What does it mean? Basically, when you create an architecture, you also have to think about money, about how can we create a product that basically is going to be good, but also is not going to cost a lot with time. Basically, we have several ways of implementing FinOps. Basically, I've been using it many times in Amazon, cost allocation tags, and this kind of thing. Basically, when you create, using Terreform is very easy, example, you create a bucket, you create a database, and you put a tag.
You do the same for all the projects. You go to the Cost Explorer application. It's going to be easy. "Okay, this is my bill. I'm going to see. Okay, which is my cost? Holy shit." This time, I spend a lot in DocumentDB,
a lot of S3, a lot of whatever. These kind of things has to be done every month, every day. We have a lot of tools to help us in order to maintain the cost under some kind of budget.
The other thing is about alerts and dashboard. I didn't do it maybe after every six months, but three things have happened. Maybe a lot of the business people is very happy with your time but two or three months later, there is an email saying, "How much money is costing me this?" Okay, it's $10,000, $100,000 per month. "You didn't tell me this, that it's going to be very costly."
Basically, alerts, dashboards, these kind of things is very easy to use. The last part about FinOps mindset is about being productive. I think it's very important. I remember when I was part of the [?] framework in Amazon, basically, this is one of the pillars.
Basically, the good thing about knowing these pillars in Amazon is about being productivity. A good way to do this is if I go to the bill every month and say, "Okay, I am spending this part, this money, this part, that money, and see how can I improve." We have plenty of ways to reduce cost. For example, reserves instance, spot instance, use these tools that has Kubernetes for scaling your resources. If you are in that QA, you can switch off. That's okay. We don't care about QA, so we can do it.
These are some tools we use in Syngenta. Cloud9, Kubecost, Infracost, also CloudWatch, I think. I don't know. That is just to say because I have read, some weeks ago, there is a web page that basically is a quiz about the Amazon icons.
You can find it in Google. You will have a quiz. Basically, they will tell you. They will put you an icon of Amazon service, and you have to figure it out with this.
I think this is CloudWatch. The third one, automation. Here, we have Pulumi. We have the other one, but many tools that basically can help us to automate.
This is very important. I have been in many, many projects that I like a lot to go to the Amazon console and do things manually. When I see that, they put me in my nerves because I have a lot of problems with this kind of way of working. We have to do that.
We have to automate. This is our duty. This is how we should do things. Basically, how can we do that? Because maybe you are in a big project that has been for 10 years. Maybe you are starting. We have some tips to do. The first one is start small.
This is, I think, the best approach is that don't go to the Kubernetes thing and automate it. Maybe I not go to Kubernetes to deploy things, is not everything. Maybe what you should do, for example, is to automate test. Go to GitLab, go to [?] create a small workflow, and basically create a small bash script that basically executes pytest or executes whatever framework you have.
Start small then with the time, you can go. Like I'm doing, for example, to have everything as a code, I use Terraform, I use LiquiBase. I use the same for testing.
We have also SonarQube. Everything is automated. If you are starting, just start small. As I said, everything is code.
There are many things we can do. We have Tattoo, we have Pulumi, we have Terraform. Configuration, as I said, Ansible, Chef for testing, we have the same database I mentioned about LiquiBase. We have also Flyway, many things. It's very easy to automate. The plan is that we have to spend time on it.
Another thing is about avoid over-engineering. Sometimes basically, I'm a developer. I have been coding in Python, in Java, in Rust, in Go. I have experience. I was in a company that basically, okay, I have time. Basically, I go.
You make an escape in Go, just for copying a file. Don't do it. You can do it with a simple basket. The final thing is transparency.
Just share everything with your team. Sorry, there are some things. One important thing is to be developer-centric.
Just have it in mind, you are DevOps. You are there for helping your developers. Try to do that. You have to help developer, not the opposite.
You have some tools that you can use. This IDP, this internal development framework, like Backstage.io, that you can have in order to have a central repository for the tools so you don't have to spend time having meetings. They go directly to Backstage.io and see what you have. You have created this pipeline.
You have created this testing workflow, whatever. You put all the DevOps tools you have. You put there. They will know about them. The final thing is development environments. I've been using DevContainer.
I encourage you to use it, so please try to use them. Finally, observability. I think it's very important just to have everything. If something happens, you have the metrics, you have the logs, now try to spend time on that.
That's something more but I think it's enough. Sorry, thank you. Maybe you can help. If you have more questions, just ask. I'll be there for some hours.
Thank you, everyone. [applause]
2023-10-03 22:11
