Computer Vision and Deep Learning for Agriculture

---

Table of Contents

---

Computer Vision and Deep Learning for Agriculture

The agriculture sector is the foundation of any economy. However, with an increase in population, the agriculture sector will feel pressure and need to scale its supplies several times to cope with the increasing consumption. In addition, uncertain factors like climate change, diseases, and infertile land have propelled the sector to adopt innovative approaches like artificial intelligence to protect and increase crop yield.

AI has the potential to change the agriculture sector by helping farmers minimize the risk of diseases, proactively adapt to changing climate conditions, monitor the security of crops using drones, etc., while keeping labor costs down (Figure 1). As a result, the overall AI in the agriculture market is projected to grow from an estimated $1B in 2020 to $4B by 2026, at a compound annual growth rate (CAGR) of 25.5% between 2020 and 2026.

This series is about CV and DL for Industrial and Big Business Applications. This blog will cover the benefits, applications, challenges, and tradeoffs of using deep learning for agriculture.

This lesson is the 4th of the 5-lesson course: CV and DL for Industrial and Big Business Applications 101.

1. Computer Vision and Deep Learning for Government
2. Computer Vision and Deep Learning for Customer Service
3. Computer Vision and Deep Learning for Banking and Finance
4. Computer Vision and Deep Learning for Agriculture (this tutorial)
5. Computer Vision and Deep Learning for Electricity

To learn about Computer Vision and Deep Learning for Agriculture, just keep reading.

---

Computer Vision and Deep Learning for Agriculture

---

Benefits

Major benefits of using AI in the agriculture sector (Figure 2).

---

Minimizing Risk and Cost and Maximizing Yield

AI applications can analyze weather and soil conditions, water usage, and risk of diseases to help farmers reduce the risk of crop failures by providing valuable insights like the right time to sow seeds, right crop/seed choices. Detecting plant diseases, weeds, and pests beforehand can reduce the use of chemicals like herbicides and pesticides and bring cost savings. Many companies have started using robots that can eliminate 80% of the volume of the substances generally sprayed on the crops and bring down the expenditure on herbicides by 90%

Further, the use of AI in harvesting, picking, and vacuum apparatus can quickly identify the location of the harvestable produce and help determine the proper fruits. The Strawberry Harvest is a classic example. A robot can pick strawberries as fast as 8 acres/day. The exact amount of harvest takes 30 humans per day. This can save time and significantly reduce labor costs.

---

Right Decision Making

With satellite imagery and weather data, AI applications can analyze the market trends, like which crops are in demand and which are more profitable. This helps the farmers to increase their revenue by guiding them about future price patterns, demand level, type of crop to sow for maximum benefit, pesticide usage, etc.

---

Applications

---

Farming: Livestock, Poultry, and Fish

Livestock, dairy products (milk, cheese, and butter), Poultry (eggs and meat), and Fish are the source of daily protein intake for a substantial proportion of the population. With the increase in food consumption demand, farmers feel the pressure to maintain a quality environment for an increasing number of animals per management unit. In addition, labor shortages make the task more challenging.

Computer vision models can analyze the closed-circuit television (CCTV) feed of animals like cows, pigs, sheep, and hens to ensure proper environmental conditions and welfare. In addition, they can provide valuable insights via house management automation, behavior analysis, disease estimation, weight measurement, egg examination, fish detection, identification, counting, etc. (Figures 3 and 4).

Livestock: AI systems can transmit the movement of animals (cow, pig, sheep) and detect whether the animal is walking, drinking, or eating, making animal management in huge barns easier. Such systems can also recognize if the animal is in pain by identifying different parts of the animal’s face and comparing them with standard facial patterns provided by vets.

Poultry: AI-based robots can feed animals, collect eggs, and remove manure repetitively. They can also automate the counting and packaging of eggs after inspecting their quality and eliminating ones that are near hatching, have cracks, or are dirty (contain feathers and blood). In addition, AI systems can determine the accuracy of fertility in the early stage of incubation.

The Viscon Group introduced VINOVO Live Embryo Detection to detect live embryos inside eggs using a patented heartbeat sensor. However, the presence of non-viable eggs inside the hatcher can be a biological hazard and occupy extra space unnecessarily.

Fish: Drones can monitor offshore fish farms and perform an inspection on underwater nets for any damages and holes. Drones can also provide fish stock information, track environmental changes, and collect data like oxygen levels, pH, salinity, and pollution level of water. In addition, detection of the hunger level of the Fish by sensors can help farmers or even robots to feed them accordingly.

The AquaCloud platform launched by Seafood Innovation Cluster and IBM led to practical breeding, area management, and control of pathogens by monitoring sea lice which are said to be a threat to Norway’s wild salmon population.

---

Yield Estimation and Maximization

Yield Estimation: A crop yield overestimation can impact profitability. However, underestimation can lead to crop wastage and additional costs. Alternatively, the proper crop yield estimate can help farmers know when they should start harvesting to maximize their profits by selling it reasonably. The challenge with yield estimation is that it depends on several dynamic factors like crop genotype, environmental factors, management practices, and their interactions.

Modern-day deep neural networks can accurately predict the yield by counting the number of fruits, vegetables, and flowers via images/video feed. They are even robust to occlusion caused by leaves, branches, illumination, etc.

Various machine learning algorithms using linear regression, projection pursuit regression (Drummond et al., 2003), weighted regression (Marko et al., 2016), decision trees (Romero et al., 2013), and rule mining have been proposed for crop yield estimation. Rahnemoonfar and Sheppard (2017) propose a yield estimation network for counting tomatoes under occlusion. They generate synthetic images for training by filling an entire picture with green and brown color circles to simulate the background and the tomato plant, which are later blurred by a Gaussian filter. Several circles of random size are drawn to create variable-size tomatoes (Figure 5).

In their architecture, they use a modified Inception-ResNet-A layer (Figure 6). It can capture features at multiple scales by concatenating the results from different convolutional layers. The result of this concatenation is then passed through a Rectified Linear function for estimation.

Mekhalfi et al. (2020) propose the first vision-based kiwifruit counting and yield estimation by emphasizing the kiwifruit tip rather than the whole fruit as it can manifest various shapes, orientations, and occlusions.

Yield Maximization: Microsoft is currently working with farmers of Andhra Pradesh to build a machine learning system that can help recommend the most profitable crops, adequate sowing date, market demand and trend, land preparation, farm yard manure application, seed treatment, optimum sowing depth, best irrigation system, the right amount of fertilizers and pesticides, etc., which can increase their crop yield by 30%. The model uses Microsoft’s Cortana Intelligence Suite.

An ML model can determine if the price elasticity curve for a given crop is inelastic, unitary, or highly elastic by considering the total demand for a given product. Knowing this data can help farmers forecast the prices and save agricultural businesses millions of dollars a year in lost revenue.

Algorithms such as support vector machine (SVM), random forest, logistic regression, deep neural networks, etc., can easily predict price elasticity. Features such models can be related to include weather parameters (rainfall, temperature, etc.), fertilizers used, land type, soil-related information, etc.

---

Monitoring

Intrusion Detection: Surveillance feeds from CCTVs can perform face recognition, identify intrusion and anomalies in large farms, and send alerts to farmers to take appropriate action. This can help avoid the risk of domestic and wild animals destroying crops and livestock.

Growth and Stress Monitoring: Computer vision systems can use the feed from drones to monitor the growth patterns of crops, wet/dry spots, specific water patterns, and the effect of fertilizers and pesticides. Further, sensors can provide data about the soil’s nutrition, water content, pH, and salinity, which can be analyzed to choose proper fertilizers and irrigation patterns (Figure 7).

High-resolution images and multiple sensor data from plants can be used as inputs for a Machine Learning algorithm to enable data fusion and feature identification for stress recognition. The entire approach can be classified into four stages (identification, classification, quantification, and prediction) to make better decisions.

One example of such stress is water stress. It occurs due to the limited water availability to the roots/soil or increased transpiration. Machine learning techniques can estimate leaf water content (LWC) by monitoring and identifying such scenarios. LWC is an essential indicator of plant productivity and yield in early crop growth stages. Regression and ensembling-based methods can easily predict LWC value which can be combined with other parameters to classify water stress levels via a classification model.

Weed Management: Weed causes significant loss in crop yield as they consume a lot of soil nutrients. Research shows that the competition between crops and weeds accounts for over an $11 billion loss. Hence their detection and management are of very importance to farmers. Computer vision systems can analyze the drone feeds to segment weeds from crops and remove them in an online fashion through lasers and sprays.

BoniRob, an agricultural robot, uses camera feeds and a computer vision algorithm to identify weeds and remove them by driving a bolt into the soil. The algorithm has learned to distinguish between crops and weeds by analyzing leaf size, shape, and color. This enables BoniRob to travel through a field and eliminate undesirable plants online (Figure 8).

Disease Identification and Prevention: Health monitoring livestock and crops is another essential application where the feed from CCTVs and drones can be used to monitor livestock health through behavior analysis. CattleEye is an excellent example of a company that uses overhead cameras and computer vision algorithms to monitor the health and behavior of cattle. Scanning crops in RGB and near-infrared light makes it possible to generate multispectral images using drone devices. With this, it is possible to specify which plants have been infected, including their location in a vast field, to apply remedies instantly. The multispectral images combine hyperspectral images with 3D scanning.

Leaves and fruits can be monitored for any infectious disease that can destroy the crop yield. A computer vision algorithm can analyze the color of leaves and the presence of spots to differentiate them from healthy ones. However, a challenge here is to get labeled plant/crop images which can be addressed to some extent by generating synthetic data points using generative adversarial networks (GANs). This is where DC-GAN (Deep convolutional GAN) significantly alleviates the class imbalance issue by generating synthetic images. A deep learning model such as EfficientNet can be trained to classify and detect crop/plant diseases.

Plantix is a great app that can convert your smartphone into a crop doctor, which can be used to detect pests and diseases in crops within seconds. Besides detection, it provides solutions to treat your crops (Figure 9).

---

Challenges

---

Uncertain Factors

The most challenging part of deploying an AI model in the agriculture sector is to ensure that the model is resilient to changing environmental conditions. Most agricultural activities have inherent uncertainty because of factors like rainfall, humidity, sunlight, temperature, and water availability which are not in our control. So what might look like an excellent solution to a deep learning model may not be optimal because of changes in external parameters.

---

Data Cost and Scarcity

AI systems require a lot of supervised data to make precise predictions. In the case of agriculture, even though spatial data can be collected quickly, temporal data is hard to manage because of its seasonal nature. For example, crop-specific data can be obtained only once a year when crops grow. This limitation makes it challenging to produce robust deep learning models in a reasonable time.

Further, the agricultural data is scattered across millions of farms that are not connected via a centralized system. This makes it difficult to collect a diverse set of samples. One way around this problem is to use satellite imagery data. However, that will have to deal with privacy issues as farmers must provide consent first.

---

Cost of Mistakes

The cost of a false recommendation or prediction in farming can have high repercussions. It is very different from other sectors like customer service and information technology, where an incorrect prediction doesn’t have a high impact. However, following a wrong recommendation could mean losing crop yield for an entire year, directly impacting farmers’ lives and global food security. One way to address this issue is to experiment with AI systems on a small portion of farmers’ land before deploying them to the whole farm.

Another way to handle it is through probabilistic approaches. In a probabilistic model, the predictions are represented by a probability distribution, and a statistical confidence interval is next to a prediction. While issuing a prediction, the model takes into account its confidence. If the confidence is not high, the prediction is discarded, thus marginalizing the risk associated with changes prescribed by AI solutions.

---

Lengthy Adoption Process

Farmers are not tech-savvy, meaning they have to entirely depend on a third party to understand and analyze the recommendations given by an AI system. This makes them reluctant to adopt the technology at first. A simple solution is to provide them with simpler technologies, such as an agriculture trading platform. Once they understand and are used to these simple solutions, offering them AI features through a workshop will be reasonable.

---

---

Summary

The agriculture sector forms the foundation of any economy. With an increase in population and food demand, farmers are propelled to adopt innovative approaches like artificial intelligence (AI) to protect and increase crop yield. As a result, AI has the potential to change the way we see agriculture as follows:

• Farming: Computer vision models can use the CCTV feed of animals like cows, pigs, sheep, and hens to provide valuable insights via house management automation, behavior analysis, disease estimation, weight measurement, egg examination, fish detection, identification, counting, etc.
• Yield Estimation: Deep neural networks can provide robust (occlusion caused by leaves, branches, illumination, etc.) prediction of the yield by counting the number of fruits, vegetables, and flowers via images/video feed.
• Yield Maximization: An ML model can predict the price elasticity curve for a given crop which can help farmers forecast the prices and save agricultural businesses millions of dollars a year in lost revenue.
• Security Monitoring: Face recognition systems can identify intrusion and anomalies in large farms and send alerts to farmers to take appropriate action. This can help avoid the risk of domestic and wild animals destroying crops and livestock.
• Health Monitoring: Leaves and fruits can be monitored for any infectious disease that can destroy the crop yield. A computer vision algorithm can analyze the color of leaves and the presence of spots to differentiate them from healthy ones.
• Weed Management: Computer vision systems can analyze the drone feeds to segment weeds from crops and remove them online through lasers and sprays.

However, using AI in the agriculture sector comes with its challenges:

• The most challenging part is to ensure that the model is resilient to changing environmental conditions like rainfall, humidity, sunlight, temperature, and water availability which are not in our control.
• Temporal data is hard to manage because of the seasonal nature of the agriculture industry.
• A wrong recommendation could mean losing crop yield for an entire year, directly impacting farmers’ lives and global food security.
• Adoption is difficult as farmers have to rely on a third party to understand and analyze the recommendations given by an AI system.

I hope this post helped you understand the benefits, applications, challenges, and tradeoffs of using deep learning in the agriculture sector. Stay tuned for another lesson where we will discuss deep learning and computer vision applications for electricity.

---

Citation Information

Mangla, P. “Computer Vision and Deep Learning for Agriculture,” PyImageSearch, P. Chugh, R. Raha, K. Kudriavtseva, and S. Huot, eds., 2022, https://pyimg.co/tqlmn

@incollection{Mangla_2022_CVDL4Agri,
  author = {Puneet Mangla},
  title = {Computer Vision and Deep Learning for Agriculture},
  booktitle = {PyImageSearch},
  editor = {Puneet Chugh and Ritwik Raha and Kseniia Kudriavtseva and Susan Huot},
  year = {2022},
  note = {https://pyimg.co/tqlmn},
}

---

---