Lesson 1.2.2: AI and remote sensing for crop monitoring

flowchart TD
    A[Satellite Imagery] --> B[Data Acquisition]
    B --> C[Pre‑processing (Calibration, Cloud Masking)]
    C --> D[Feature Extraction (NDVI, EVI, etc.)]
    D --> E[Crop Health Index]
    E --> F[Decision Support (Yield Forecast, Irrigation Planning)]

Concept: Remote Sensing Fundamentals

Remote sensing is the science of acquiring information about an object or phenomenon without making physical contact. In the context of agriculture, it allows agronomists, farm managers and researchers to observe crop health, growth stages, and stress factors over vast areas with high temporal resolution. The technology relies on sensors mounted on aircraft, satellites or drones that capture electromagnetic energy reflected or emitted by the Earth's surface. Understanding the basic principles of remote sensing—such as the electromagnetic spectrum, sensor characteristics, image interpretation, and data processing—is essential for harnessing its full potential in crop monitoring.

1. The Electromagnetic Spectrum and Spectral Bands

Remote sensing sensors detect reflected or emitted energy across various spectral bands. The most relevant for vegetation monitoring are:

• Visible (VIS) 0.4–0.7 µm: Human eye range, useful for general colour imagery.
• Near‑Infrared (NIR) 0.7–1.3 µm: Plants reflect strongly here due to leaf cellular structure.
• Short‑wave Infrared (SWIR) 1.3–3 µm: Sensitive to moisture content and stress.
• Thermal Infrared (TIR) 8–14 µm: Measures surface temperature, indicating transpiration rates.

Each band captures distinct physiological signals. For example, healthy plants exhibit high NIR reflectance and low red reflectance, forming the basis of vegetation indices such as NDVI.

2. Sensor Platforms and Acquisition Modes

Remote sensing data can be obtained from three main platforms, each with distinct advantages:

1. Satellite‑based sensors (e.g., Sentinel‑2, Landsat 8) provide global coverage with revisit times ranging from 5 to 16 days. They are ideal for large‑scale monitoring but have coarser spatial resolution (10–30 m).
2. Airborne sensors (e.g., UAV‑mounted cameras, aircraft LIDAR) offer higher spatial resolution (centimetre to metre) and flexible flight planning but cover smaller areas per flight.
3. Ground‑based sensors (e.g., proximal sensors, field spectrometers) deliver the highest spectral resolution and accuracy, useful for calibration and validation.

The choice of platform depends on the spatial scale, temporal frequency, and budget constraints of the monitoring programme.

3. Radiometric and Geometric Corrections

Raw sensor data are influenced by atmospheric conditions, sensor noise, and sensor geometry. Two principal correction steps are:

1. Radiometric correction: Converts digital numbers (DN) to top‑of‑atmosphere (TOA) reflectance using calibration coefficients supplied by the satellite provider. This removes sensor‑specific biases and allows inter‑satellite comparison.
2. Geometric correction: Aligns imagery to a common map projection, correcting for lens distortion, sensor tilt and Earth curvature. Ground control points (GCPs) or automatic tie‑point matching are typically employed.

Failure to perform these corrections can lead to erroneous interpretation of vegetation health.

4. Vegetation Indices and Their Interpretation

Vegetation indices are mathematical combinations of spectral bands designed to highlight specific plant characteristics. The most widely used index is the Normalised Difference Vegetation Index (NDVI):

NDVI = (NIR - Red) / (NIR + Red)

NDVI values range from -1 to +1. Values close to +1 indicate dense, healthy vegetation, whereas values near 0 or negative suggest sparse vegetation or non‑vegetated surfaces.

Other indices include:

• Enhanced Vegetation Index (EVI): Improves sensitivity in high biomass areas by incorporating a blue band.
• Soil‑Adjusted Vegetation Index (SAVI): Reduces soil background influence, useful in sparse vegetation.
• Water‑Sensitive Vegetation Index (WSVI): Enhances detection of water stress.

By analysing temporal NDVI trends, one can infer crop vigour, detect early stress, and estimate yield potential.

5. Practical Example: Calculating NDVI from Sentinel‑2 Data in Python

Below is a concise Python snippet that demonstrates how to read Sentinel‑2 Level‑2A imagery, perform atmospheric correction to TOA reflectance, and compute NDVI. The example uses the rasterio library for raster handling and numpy for numerical operations.

import rasterio
import numpy as np

Load the NIR (Band 8) and Red (Band 4) files
with rasterio.open('S2B_20230501_T31TCJ_B08.jp2') as src_nir:
    nir = src_nir.read(1).astype('float32')
with rasterio.open('S2B_20230501_T31TCJ_B04.jp2') as src_red:
    red = src_red.read(1).astype('float32')

Convert Digital Numbers to reflectance (scale factor 0.0001)
nir_ref = nir * 0.0001
red_ref = red * 0.0001

Avoid division by zero
denominator = nir_ref + red_ref
denominator[denominator == 0] = np.nan

Compute NDVI
ndvi = (nir_ref - red_ref) / denominator

Save NDVI to a new GeoTIFF
profile = src_nir.profile
profile.update(dtype=rasterio.float32, count=1, compress='lzw')
with rasterio.open('NDVI_S2B_20230501.tif', 'w', **profile) as dst:
    dst.write(ndvi.astype(rasterio.float32), 1)

Line‑by‑line explanation:

1. Import rasterio for raster I/O and numpy for array operations.
2. Open the Sentinel‑2 NIR band (Band 8) and read the first raster band into a NumPy array, converting to float32 to preserve precision.
3. Open the Red band (Band 4) similarly.
4. Apply the Sentinel‑2 scale factor (0.0001) to convert raw digital numbers to surface reflectance values between 0 and 1.
5. Compute the denominator of the NDVI formula, ensuring that any zero values are replaced with NaN to prevent division‑by‑zero errors.
6. Calculate NDVI using the standard formula.
7. Prepare a new GeoTIFF output profile based on the input metadata, updating the data type to float32 and enabling LZW compression.
8. Write the NDVI array to a new file named NDVI_S2B_20230501.tif with a single band.

6. Temporal Analysis and Change Detection

Remote sensing excels at capturing temporal dynamics. By stacking NDVI time series, one can detect:

• Phenological shifts: Changes in the timing of leaf‑out, flowering or senescence.
• Stress events: Sudden drops in NDVI often indicate drought, disease or pest infestation.
• Recovery trends: Post‑stress NDVI recovery rates can inform management decisions.

Statistical tools such as linear regression or breakpoint analysis help quantify these trends. Machine learning classifiers can also be trained on multi‑band imagery to segment crop types and map field boundaries automatically.

7. Integration with Ground Truth and Decision Support

While remote sensing provides broad coverage, ground truth data are essential for calibration and validation. Field measurements of leaf area index (LAI), chlorophyll content, or yield provide the basis for:

1. Calibration curves linking NDVI or other indices to actual biomass.
2. Accuracy assessment using confusion matrices and metrics such as Cohen’s kappa.
3. Model refinement for yield prediction or precision agriculture guidance.

When combined with decision support systems, remote sensing can generate actionable insights—such as variable rate fertiliser maps, irrigation scheduling, or early warning alerts for disease outbreaks.

8. Limitations and Considerations

Despite its power, remote sensing has inherent constraints that must be recognised:

• Atmospheric interference: Cloud cover can obscure imagery, especially in temperate regions where cloud frequency is high.
• Spatial resolution mismatch: Coarser satellite data may miss small field heterogeneity; UAVs can fill this gap but at higher cost.
• Sensor drift: Long‑term sensor degradation necessitates periodic recalibration.
• Data volume: High‑resolution imagery generates large datasets requiring robust storage and processing infrastructure.
• Interpretation ambiguity: Similar spectral signatures can arise from different stressors; complementary data (e.g., soil moisture, weather) are often required for accurate attribution.

Being aware of these limitations ensures that remote sensing outputs are used appropriately and supplemented with complementary information where necessary.

9. Future Directions in AI‑Enhanced Remote Sensing

Artificial intelligence, particularly deep learning, is increasingly applied to remote sensing for tasks such as semantic segmentation, anomaly detection, and predictive modelling. Convolutional neural networks can automatically extract complex spatial patterns from multi‑band imagery, outperforming traditional indices in some contexts. However, AI models require large, well‑annotated training datasets and careful validation to avoid overfitting or bias. Integrating AI with classical remote sensing techniques—such as using NDVI as an input feature to a machine learning model—often yields the most robust results.

In summary, remote sensing fundamentals encompass understanding the spectral basis of vegetation signals, selecting appropriate sensor platforms, applying necessary corrections, computing indices like NDVI, and integrating these data with ground truth and AI tools. Mastery of these concepts equips practitioners to monitor crop health accurately, optimise resource use, and ultimately enhance agricultural productivity.

10. Additional Example: Calculating the Normalised Difference Water Index (NDWI)

The NDWI is useful for monitoring surface water and vegetation water content. It uses the near‑infrared (NIR) and short‑wave infrared (SWIR) bands:

import rasterio
import numpy as np

Load NIR (Band 8) and SWIR (Band 11) from Sentinel‑2
with rasterio.open('S2B_20230501_T31TCJ_B08.jp2') as src_nir:
    nir = src_nir.read(1).astype('float32') * 0.0001  # reflectance
with rasterio.open('S2B_20230501_T31TCJ_B11.jp2') as src_swir:
    swir = src_swir.read(1).astype('float32') * 0.0001

Avoid division by zero
den = nir + swir
den[den == 0] = np.nan

Compute NDWI
ndwi = (nir - swir) / den

Save NDWI
profile = src_nir.profile
profile.update(dtype=rasterio.float32, count=1, compress='lzw')
with rasterio.open('NDWI_S2B_20230501.tif', 'w', **profile) as dst:
    dst.write(ndwi.astype(rasterio.float32), 1)

11. Comparison of Vegetation Indices

Implementation/Code: Building a CNN for Crop Health

In this section we will walk through the entire pipeline for constructing a convolutional neural network (CNN) that can classify crop disease stages from remote sensing imagery. The example uses the popular Keras API within TensorFlow, as it provides a high‑level interface that is both expressive and well‑documented. We will cover data preparation, model architecture, training, evaluation and a brief discussion of practical deployment considerations.

1. Preparing the Dataset

Remote sensing data is typically organised in multi‑band images (e.g. RGB, near‑infrared, thermal). For the sake of clarity we assume a three‑channel RGB representation, but the same principles apply to higher‑dimensional inputs. The dataset is split into training, validation and test sets, and each image is normalised to the range [0, 1] to stabilise gradient descent.

import tensorflow as tf
from tensorflow.keras.preprocessing.image import ImageDataGenerator

Define directories
train_dir = 'data/train'
val_dir   = 'data/val'
test_dir  = 'data/test'

Normalisation and augmentation for training set
train_datagen = ImageDataGenerator(
    rescale=1./255,
    rotation_range=20,
    width_shift_range=0.1,
    height_shift_range=0.1,
    horizontal_flip=True,
    fill_mode='nearest'
)

val_datagen   = ImageDataGenerator(rescale=1./255)
test_datagen  = ImageDataGenerator(rescale=1./255)

Flow from directory
train_generator = train_datagen.flow_from_directory(
    train_dir,
    target_size=(128, 128),
    batch_size=32,
    class_mode='categorical'
)

val_generator = val_datagen.flow_from_directory(
    val_dir,
    target_size=(128, 128),
    batch_size=32,
    class_mode='categorical'
)

test_generator = test_datagen.flow_from_directory(
    test_dir,
    target_size=(128, 128),
    batch_size=32,
    class_mode='categorical',
    shuffle=False
)

Line‑by‑line explanation:

• ImageDataGenerator creates a pipeline that can rescale pixel values and apply random transformations to augment the training data, which is crucial when the dataset is limited.
• The rescale parameter divides every pixel by 255, converting 8‑bit integer values into floating‑point numbers between 0 and 1.
• Rotation, shifts and horizontal flips introduce variability, helping the network generalise to unseen viewpoints.
• flow_from_directory automatically infers class labels from the folder structure.
• Setting shuffle=False for the test generator preserves order, enabling correct mapping of predictions to filenames later on.

2. Building the CNN Architecture

We adopt a modest depth to keep inference times low, yet sufficient to capture the spatial hierarchies pertinent to disease stage detection. Convolutional layers learn local patterns, max‑pooling reduces spatial resolution while retaining salient features, and fully connected layers perform the final classification.

from tensorflow.keras import layers, models

def build_cnn(input_shape=(128, 128, 3), num_classes=5):
    model = models.Sequential(name='CropHealthCNN')

    # Block 1
    model.add(layers.Conv2D(32, (3, 3), activation='relu',
                            padding='same', input_shape=input_shape))
    model.add(layers.BatchNormalization())
    model.add(layers.Conv2D(32, (3, 3), activation='relu', padding='same'))
    model.add(layers.BatchNormalization())
    model.add(layers.MaxPooling2D((2, 2)))
    model.add(layers.Dropout(0.25))

    # Block 2
    model.add(layers.Conv2D(64, (3, 3), activation='relu', padding='same'))
    model.add(layers.BatchNormalization())
    model.add(layers.Conv2D(64, (3, 3), activation='relu', padding='same'))
    model.add(layers.BatchNormalization())
    model.add(layers.MaxPooling2D((2, 2)))
    model.add(layers.Dropout(0.25))

    # Block 3
    model.add(layers.Conv2D(128, (3, 3), activation='relu', padding='same'))
    model.add(layers.BatchNormalization())
    model.add(layers.Conv2D(128, (3, 3), activation='relu', padding='same'))
    model.add(layers.BatchNormalization())
    model.add(layers.MaxPooling2D((2, 2)))
    model.add(layers.Dropout(0.25))

    # Classification head
    model.add(layers.Flatten())
    model.add(layers.Dense(256, activation='relu'))
    model.add(layers.BatchNormalization())
    model.add(layers.Dropout(0.5))
    model.add(layers.Dense(num_classes, activation='softmax'))

    return model

Key points:

• All convolutional layers use a 3×3 kernel, which balances receptive field growth with computational cost.
• Batch normalisation after each convolution stabilises learning and allows a higher learning rate.
• Dropout layers mitigate overfitting, especially important when the number of disease stages is small.
• The final dense layer uses a softmax activation to output a probability distribution over the disease stages.

3. Compiling the Model

We employ the Adam optimiser with an initial learning rate of 0.001, which adapts the step size during training. Categorical cross‑entropy is appropriate for multi‑class classification with one‑hot encoded labels.

model = build_cnn(num_classes=train_generator.num_classes)
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.001),
              loss='categorical_crossentropy',
              metrics=['accuracy'])
model.summary()

The model.summary() call prints a concise overview of the parameter count and layer shapes, useful for debugging.

4. Training with Callbacks

Training is performed over a fixed number of epochs, but early stopping and learning‑rate reduction callbacks prevent overfitting and accelerate convergence.

callbacks = [
    tf.keras.callbacks.EarlyStopping(
        monitor='val_loss', patience=10, restore_best_weights=True
    ),
    tf.keras.callbacks.ReduceLROnPlateau(
        monitor='val_loss', factor=0.5, patience=5, min_lr=1e-6
    )
]

history = model.fit(
    train_generator,
    epochs=100,
    validation_data=val_generator,
    callbacks=callbacks
)

Explanation:

• EarlyStopping halts training when validation loss no longer improves for 10 consecutive epochs, then restores the weights from the epoch with the best validation loss.
• ReduceLROnPlateau halves the learning rate when validation loss stagnates, helping fine‑tune the model in later stages.
• The history object records training dynamics, enabling post‑hoc visualisation of loss and accuracy curves.

5. Evaluating Performance

After training, the model is evaluated on the held‑out test set to gauge generalisation. Metrics beyond accuracy, such as the confusion matrix and per‑class recall, are often more informative when class distributions are imbalanced.

import numpy as np
import matplotlib.pyplot as plt
from sklearn.metrics import classification_report, confusion_matrix

Predict probabilities
test_steps = test_generator.samples // test_generator.batch_size
pred_probs = model.predict(test_generator, steps=test_steps, verbose=1)
pred_labels = np.argmax(pred_probs, axis=1)

True labels
true_labels = test_generator.classes
class_names = list(test_generator.class_indices.keys())

Classification report
report = classification_report(true_labels, pred_labels, target_names=class_names)
print(report)

Confusion matrix
cm = confusion_matrix(true_labels, pred_labels)
fig, ax = plt.subplots(figsize=(8, 6))
im = ax.imshow(cm, interpolation='nearest', cmap=plt.cm.Blues)
ax.figure.colorbar(im, ax=ax)
ax.set_xticks(np.arange(len(class_names)))
ax.set_yticks(np.arange(len(class_names)))
ax.set_xticklabels(class_names, rotation=45, ha='right')
ax.set_yticklabels(class_names)
ax.set_xlabel('Predicted label')
ax.set_ylabel('True label')
plt.title('Confusion Matrix')
plt.tight_layout()
plt.show()

This script produces a comprehensive performance summary. A high overall accuracy might mask poor detection of a rare disease stage; the confusion matrix and per‑class recall illuminate such issues.

6. Practical Deployment Considerations

• Model Size: For edge devices or field‑based drones, consider pruning or quantisation to reduce the model footprint. TensorFlow Lite provides utilities for converting a Keras model to a lightweight format.
• Inference Speed: Batch inference on a GPU is fast, but real‑time monitoring may require single‑image inference. Profiling the model with tf.profiler helps identify bottlenecks.
• Data Drift: Remote sensing conditions (lighting, sensor calibration) can change over time. Periodic fine‑tuning with new labelled data keeps the model robust.
• Explainability: Techniques such as Grad‑CAM or LIME can visualise salient regions, assisting agronomists in validating the model’s decisions.

7. Transfer Learning Alternative

When the dataset is small, leveraging a pre‑trained backbone can drastically improve performance. The following example demonstrates how to fine‑tune MobileNetV2, freezing early layers and adding a custom head for crop disease classification.

from tensorflow.keras.applications import MobileNetV2
from tensorflow.keras import layers, models, optimizers

def build_transfer_model(input_shape=(128, 128, 3), num_classes=5):
    base = MobileNetV2(input_shape=input_shape,
                       include_top=False,
                       weights='imagenet')
    base.trainable = False  # freeze the backbone

    x = base.output
    x = layers.GlobalAveragePooling2D()(x)
    x = layers.Dense(512, activation='relu')(x)
    x = layers.Dropout(0.5)(x)
    outputs = layers.Dense(num_classes, activation='softmax')(x)

    model = models.Model(inputs=base.input, outputs=outputs)
    return model

model_tl = build_transfer_model(num_classes=train_generator.num_classes)
model_tl.compile(optimizer=optimizers.Adam(learning_rate=1e-4),
                 loss='categorical_crossentropy',
                 metrics=['accuracy'])
model_tl.summary()

8. Comparison of Approaches

9. Summary

We have demonstrated a complete workflow for building a CNN that classifies crop disease stages from remote sensing imagery. Starting from data preparation and augmentation, through model construction, training with strategic callbacks, to rigorous evaluation and deployment pragmatics, each step is essential for delivering a reliable, field‑ready solution. The code snippets provide a ready‑to‑run template that can be customised to the specific spectral bands and disease taxonomy of any agricultural monitoring programme.

Worked Example: Detecting Wheat Rust from UAV Data

In this section we walk through a practical application of the convolutional neural network (CNN) you trained earlier. The goal is to use the model to classify images captured by a Unmanned Aerial Vehicle (UAV) over a wheat field, identifying areas affected by wheat rust. We will examine the preprocessing pipeline, inference step, and post‑processing of the model outputs, and discuss how to interpret the results in an agronomic context.

1. Dataset description

The UAV imagery was collected on a 50 ha wheat field using a multispectral camera. Each flight produced a set of orthophotos in the Red, Green, Blue (RGB) and Near‑Infrared (NIR) bands, at a ground sampling distance of 5 cm/pixel. For the purpose of this exercise, we focus on the RGB composites, which are sufficient to detect the characteristic yellow‑brown pustules of wheat rust.

Images were tiled into 256×256 pixel crops, with a 50 % overlap to maximise the number of training samples. A field‑operator annotated a subset of these tiles, marking each as rust or healthy. The resulting dataset contains 10 000 rust tiles and 12 000 healthy tiles, split into training (70 %), validation (15 %) and test (15 %) sets.

2. Pre‑processing pipeline

Before feeding the tiles to the model, we perform a series of standardised steps to normalise the data and augment the training set. The following code demonstrates the preprocessing for a single tile, using the torchvision library.

import torch
from torchvision import transforms
from PIL import Image
import numpy as np

Define the transform pipeline
preprocess = transforms.Compose([
    transforms.Resize((224, 224)),          # Rescale to the model input size
    transforms.ToTensor(),                 # Convert PIL image to torch tensor
    transforms.Normalize(                  # Normalise per channel
        mean=[0.485, 0.456, 0.406],        # ImageNet means
        std=[0.229, 0.224, 0.225]          # ImageNet stds
    )
])

def load_and_preprocess(image_path):
    "Load an image from disk and apply the preprocessing pipeline."
    img = Image.open(image_path).convert('RGB')  # Ensure 3‑channel RGB
    img_tensor = preprocess(img)                # Apply transforms
    img_tensor = img_tensor.unsqueeze(0)        # Add batch dimension
    return img_tensor

Line by line explanation:

• import torch – imports the core PyTorch library.
• Import torchvision.transforms to access a collection of common image operations.
• Import Image from PIL for reading image files.
• Define preprocess as a Compose object that chains several transforms:
• Resize((224, 224)) – scales the image to the 224×224 input size expected by the CNN.
• ToTensor() – converts the image to a tensor with shape (C, H, W) and scales pixel values to the [0, 1] range.
• Normalize(mean, std) – standardises each channel using the mean and standard deviation of the ImageNet training set, which the network was pre‑trained on.
• The helper function load_and_preprocess opens an image file, ensures it is RGB, applies the pipeline, and adds a batch dimension so the tensor shape becomes (1, 3, 224, 224).

3. Inference on UAV tiles

Once the data is pre‑processed, we can perform inference with the trained CNN. Below is a compact script that loads a batch of tiles, runs the model, and writes out a probability map.

import torch
from torch.utils.data import DataLoader, Dataset

class UAVTileDataset(Dataset):
    def __init__(self, file_list, transform=None):
        self.file_list = file_list
        self.transform = transform

    def __len__(self):
        return len(self.file_list)

    def __getitem__(self, idx):
        img = Image.open(self.file_list[idx]).convert('RGB')
        if self.transform:
            img = self.transform(img)
        return img, self.file_list[idx]  # return path for later

Assume `test_files` is a list of file paths for the test set
dataset = UAVTileDataset(test_files, transform=preprocess)
loader = DataLoader(dataset, batch_size=32, shuffle=False)

model = torch.load('wheat_rust_cnn.pt')  # Load the trained model
model.eval()  # Set to evaluation mode

probabilities = []

with torch.no_grad():
    for imgs, paths in loader:
        preds = model(imgs)                  # Forward pass
        probs = torch.softmax(preds, dim=1)  # Convert logits to class probabilities
        rust_probs = probs[:, 1]             # Probability of rust class
        probabilities.extend(zip(paths, rust_probs.cpu().numpy()))

Sort tiles by rust probability and output top 10
probabilities.sort(key=lambda x: x[1], reverse=True)
for path, prob in probabilities[:10]:
    print(f"{path}: {prob:.3f}")

Key points:

• The UAVTileDataset class encapsulates loading and transforming each image. Returning the file path allows us to trace predictions back to the original tile.
• The DataLoader batches the tiles, improving GPU utilisation.
• Setting model.eval() disables dropout and batch‑norm updates, ensuring deterministic outputs.
• We use torch.no_grad() to avoid tracking gradients, reducing memory usage.
• Softmax converts raw logits to a probability distribution over the two classes. rust_probs isolates the probability of the rust class.
• Finally, we sort and print the tiles with the highest rust probability, which can be visualised on a map overlay.

4. Post‑processing and mapping

The raw probability scores need to be translated into a binary map for agronomic decision‑making. A common approach is to apply a threshold, for example 0.5, to decide whether a tile is rust‑positive. However, in precision agriculture it is often desirable to minimise false positives to avoid unnecessary fungicide application.

We can compute the Receiver Operating Characteristic (ROC) curve on the validation set and select a threshold that balances sensitivity and specificity. Suppose we choose a threshold of 0.7 after inspecting the ROC. The following code snippet applies this threshold and generates a GeoTIFF mask.

import rasterio
from rasterio.transform import from_origin

Assume `tile_coords` is a list of (row, col) indices corresponding to each tile
mask = np.zeros((field_height, field_width), dtype=np.uint8)

for (row, col), prob in probabilities:
    if prob >= 0.7:
        # Mark the corresponding region as rust (value 1)
        mask[row256:row256+256, col256:col256+256] = 1

Save mask as GeoTIFF
transform = from_origin(minx, maxy, 0.05, 0.05)  # 5 cm pixel size
with rasterio.open('rust_mask.tif', 'w', driver='GTiff',
                   height=mask.shape[0], width=mask.shape[1],
                   count=1, dtype=rasterio.uint8,
                   crs='EPSG:32633', transform=transform) as dst:
    dst.write(mask, 1)

Explanation:

• The mask array initially contains zeros (healthy).
• For each tile with probability above the threshold, the corresponding 256×256 pixel block is set to one.
• We use rasterio to write the mask to a GeoTIFF, specifying the spatial reference system (here EPSG:32633, UTM zone 33N) and the transform that maps pixel coordinates to geospatial coordinates.

5. Interpreting the results

Once the mask is generated, it can be overlaid on the orthophoto mosaic to visualise rust hotspots. Agronomists can then prioritise these areas for fungicide spraying or further field inspection.

These figures suggest that the model performs well, but field validation is essential. For instance, a few false positives may appear in areas with similar colouration due to soil moisture or shading. A ground truth survey can confirm the model’s predictions and refine the threshold if necessary.

6. Limitations and future improvements

While the CNN demonstrates strong performance on the test set, several practical constraints should be acknowledged:

1. Temporal variability: Wheat rust progression can change rapidly; images captured at different times may yield different spectral signatures.
2. Sensor noise: UAV cameras suffer from varying illumination and lens artefacts that can confuse the network.
3. Class imbalance: In early infestations, rust tiles are far fewer than healthy tiles, potentially biasing the model.

To mitigate these issues, consider the following enhancements:

• Implement a temporal ensemble by averaging predictions from flights taken at multiple dates.
• Augment the training set with synthetic noise and colour jitter to improve robustness.
• Use focal loss during training to penalise misclassification of the minority class more heavily.

By iteratively refining the model and integrating field feedback, the UAV‑based rust detection pipeline can become a reliable decision support tool for wheat growers, reducing unnecessary chemical use and improving crop health monitoring.

Additional Example: TensorFlow Inference Pipeline

Below is a lightweight TensorFlow/Keras script that performs inference on the same UAV tiles, illustrating a different deep learning framework. The example demonstrates loading a saved model, preprocessing with Keras utilities, and generating probability maps.

import tensorflow as tf
import numpy as np
from tensorflow.keras.preprocessing import image
from tensorflow.keras.applications.resnet50 import preprocess_input, decode_predictions

Load the trained ResNet‑50 model (weights saved in HDF5)
model = tf.keras.models.load_model('wheat_rust_tf.h5')
model.trainable = False

def load_and_preprocess_tf(img_path):
    img = image.load_img(img_path, target_size=(224, 224))
    img_array = image.img_to_array(img)               # (224,224,3)
    img_array = np.expand_dims(img_array, axis=0)     # (1,224,224,3)
    img_array = preprocess_input(img_array)           # Keras ResNet preprocessing
    return img_array

Example inference on a single tile
img_tensor = load_and_preprocess_tf('tile_001.png')
preds = model.predict(img_tensor)                    # logits
probs = tf.nn.softmax(preds, axis=1).numpy()
rust_prob = probs[0, 1]  # probability of rust class
print(f"Rust probability: {rust_prob:.3f}")

Comparing PyTorch vs TensorFlow Inference Approaches

Pitfalls: Deploying AI in the Field

When an AI model is trained under controlled laboratory or cloud‑based conditions, it often performs admirably on the same dataset. However, the real world is far messier. Deploying an AI solution for crop monitoring across a diverse set of farms, sensor platforms, and weather regimes exposes a range of pitfalls that can erode performance, increase operational cost, and erode stakeholder confidence. The most frequently encountered issues are data drift, sensor noise, and scalability constraints. Below we analyse each challenge, illustrate its impact, and present pragmatic counter‑measures that are implementable within a typical remote‑sensing workflow.

1. Data Drift: The Silent Erosion of Accuracy

Data drift refers to the gradual or sudden change in the statistical properties of the input data that a model receives after deployment. In precision agriculture, this can arise from seasonal shifts, new cultivars, soil amendments, or even changes in the imaging hardware itself. A model trained on summer 2022 imagery may misclassify a late‑spring field in 2024 because the spectral signature of early‑season crops has evolved.

• Concept drift – the underlying relationship between input features and output labels changes (e.g. a new pest species altering leaf reflectance).
• Covariate shift – the distribution of input features changes while the relationship to the target remains the same (e.g. cloud cover altering overall brightness).

Without mitigation, drift can cause a steady decline in model precision, leading to costly misallocation of resources such as irrigation or pesticide application.

Practical solutions:

1. Continuous monitoring and alerting: Compute key statistics (e.g. mean NDVI, spectral histograms) in real time and trigger alerts when they deviate beyond a tolerance window.
2. Incremental learning pipelines: Periodically retrain the model with fresh labelled data sourced from the field, using techniques such as transfer learning to adapt to new conditions without full retraining.
3. Ensemble of temporal models: Maintain separate models for distinct seasons and switch based on the current date or detected phenological stage.

2. Sensor Noise and Calibration Errors

Remote‑sensing data is acquired from a variety of platforms – satellite constellations, UAVs, ground‑based sensors – each with inherent noise characteristics. Factors such as temperature fluctuations, lens contamination, or battery voltage variations can introduce bias or random errors.

Typical manifestations include:

• Outlier pixels caused by sensor glitches.
• Band‑specific bias leading to incorrect spectral indices.
• Temporal inconsistencies between successive acquisitions.

These artefacts can mislead an AI model into learning spurious correlations. Robust preprocessing and noise‑aware training are therefore essential.

Practical solutions:

1. Radiometric calibration: Apply per‑sensor calibration coefficients (derived from ground truth panels or on‑board calibration devices) before feature extraction.
2. Statistical outlier removal: Use robust statistics (median absolute deviation) to mask anomalous pixels prior to model inference.
3. Augmentation with noise models: During training, inject realistic noise patterns (e.g. Gaussian, speckle) to improve model resilience.

3. Scalability: From a Prototype to a Farm‑Wide Solution

Many AI prototypes are built on a handful of images or a single UAV flight. Scaling to thousands of hectares across multiple farms introduces computational, logistical, and organisational hurdles:

• Data ingestion throughput: ingesting high‑resolution imagery at scale can saturate network bandwidth.
• Model inference latency: a 1 s inference time per tile may be acceptable in a lab but becomes prohibitive for real‑time decision support.
• Hardware heterogeneity: edge devices (e.g. Raspberry Pi clusters) may lack the GPU resources needed for deep models.

Addressing scalability requires a combination of architectural optimisation and resource planning.

Practical solutions:

1. Model compression: Apply pruning, quantisation, or knowledge distillation to reduce model size while retaining accuracy.
2. Distributed inference: Deploy inference jobs across a cloud‑edge continuum, using lightweight containers orchestrated by Kubernetes.
3. Data summarisation: Use region‑of‑interest extraction to focus computation only on areas flagged by a pre‑filter (e.g. vegetation indices).

4. Implementation Example: Drift‑Aware Retraining Pipeline

The following code demonstrates a minimalistic pipeline that could run nightly on a cloud instance. It ingests new imagery, computes drift statistics, and, if drift exceeds a threshold, retrains the model using the newly labelled data.

import numpy as np
import pandas as pd
from sklearn.metrics import mean_absolute_error
from joblib import load, dump
import glob
import os

Paths to the directories
RAW_DATA_DIR = "/data/raw/"
MODEL_DIR = "/models/"
DRIFT_THRESHOLD = 0.05  # acceptable MAE increase

def load_latest_images():
    ""Load the most recent image set and associated labels.""
    files = sorted(glob.glob(os.path.join(RAW_DATA_DIR, "*.npz")), reverse=True)
    latest_file = files[0]
    data = np.load(latest_file)
    return data["features"], data["labels"]

def compute_mae(pred, true):
    ""Return mean absolute error between predictions and ground truth.""
    return mean_absolute_error(true, pred)

def main():
    # Load current model
    model = load(os.path.join(MODEL_DIR, "crop_classifier.joblib"))

    # Load new data
    X_new, y_new = load_latest_images()

    # Predict with current model
    preds = model.predict(X_new)

    # Compute drift metric
    mae = compute_mae(preds, y_new)

    # Compare against baseline
    baseline_mae = np.load(os.path.join(MODEL_DIR, "baseline_mae.npy"))
    if mae - baseline_mae > DRIFT_THRESHOLD:
        print(f"Drift detected: MAE increased by {mae - baseline_mae:.3f}")
        # Retrain model on combined dataset
        X_combined = np.vstack([model.X_train_, X_new])
        y_combined = np.concatenate([model.y_train_, y_new])
        model.fit(X_combined, y_combined)
        # Save updated model and new baseline
        dump(model, os.path.join(MODEL_DIR, "crop_classifier.joblib"))
        np.save(os.path.join(MODEL_DIR, "baseline_mae.npy"), mae)
    else:
        print("No significant drift detected.")

if __name__ == "__main__":
    main()

Line‑by‑line explanation:

• import numpy as np – Import NumPy for numerical operations.
• import pandas as pd – Pandas is imported for potential tabular data handling, although not used in this snippet.
• from sklearn.metrics import mean_absolute_error – MAE is chosen as a simple drift metric.
• from joblib import load, dump – Joblib serialises scikit‑learn models efficiently.
• import glob, os – Utility modules for file system navigation.
• RAW_DATA_DIR = "/data/raw/" – Directory where the latest raw images and labels are stored.
• MODEL_DIR = "/models/" – Location of the persisted model and baseline metrics.
• DRIFT_THRESHOLD = 0.05 – Threshold for MAE increase that triggers retraining.
• def load_latest_images(): – Function that finds the most recent NPZ file, loads features and labels.
• files = sorted(glob.glob(os.path.join(RAW_DATA_DIR, "*.npz")), reverse=True) – List all NPZ files in descending order.
• latest_file = files[0] – Select the newest file.
• data = np.load(latest_file) – Load the array dictionary.
• return data["features"], data["labels"] – Return the feature matrix and label vector.
• def compute_mae(pred, true): – Helper to compute MAE.
• def main(): – Main orchestration block.
• model = load(os.path.join(MODEL_DIR, "crop_classifier.joblib")) – Load the current model.
• X_new, y_new = load_latest_images() – Load new data.
• preds = model.predict(X_new) – Generate predictions.
• mae = compute_mae(preds, y_new) – Compute current MAE.
• baseline_mae = np.load(os.path.join(MODEL_DIR, "baseline_mae.npy")) – Load stored baseline MAE.
• if mae - baseline_mae > DRIFT_THRESHOLD: – Check if drift exceeds threshold.
• print(f"Drift detected: MAE increased by {mae - baseline_mae:.3f}") – Log drift.
• X_combined = np.vstack([model.X_train_, X_new]) – Concatenate old training data with new samples.
• y_combined = np.concatenate([model.y_train_, y_new]) – Same for labels.
• model.fit(X_combined, y_combined) – Retrain the model.
• dump(model, os.path.join(MODEL_DIR, "crop_classifier.joblib")) – Persist updated model.
• np.save(os.path.join(MODEL_DIR, "baseline_mae.npy"), mae) – Update baseline MAE.
• else: – If drift is within tolerance.
• print("No significant drift detected.") – Log message.
• if __name__ == "__main__": main() – Entry point.

5. Human‑In‑the‑Loop: Bridging the Gap

Even the most robust technical safeguards cannot replace human expertise entirely. Farmers, agronomists, and remote‑sensing specialists should be integrated into the deployment lifecycle:

1. Label validation: Periodically sample model predictions for ground‑truth comparison.
2. Feedback channels: Provide an interface for field crews to flag false positives/negatives directly from a mobile app.
3. Continuous education: Train stakeholders on recognising sensor artefacts and interpreting AI outputs.

By combining automated drift detection, noise‑aware preprocessing, scalable inference, and human oversight, a deployment can maintain high fidelity over time, ensuring that AI truly supports precision crop monitoring rather than becoming a source of frustration.

Additional Example: Endpoint Monitoring with Amazon SageMaker

Below is a lightweight script that polls a SageMaker hosted endpoint, records latency and error rates, and triggers a CloudWatch alarm if performance degrades.

import boto3
import time
import json

ENDPOINT_NAME = "crop-detection-endpoint"
REGION = "eu-west-2"
SAGEMAKER_CLIENT = boto3.client("sagemaker-runtime", region_name=REGION)
CLOUDWATCH_CLIENT = boto3.client("cloudwatch", region_name=REGION)

def invoke_endpoint(payload):
    response = SAGEMAKER_CLIENT.invoke_endpoint(
        EndpointName=ENDPOINT_NAME,
        ContentType="application/json",
        Body=json.dumps(payload)
    )
    return response['Body'].read()

def monitor():
    start = time.time()
    try:
        _ = invoke_endpoint({"features": [0.1, 0.2, 0.3]})
        latency = time.time() - start
        status = "OK"
    except Exception:
        latency = None
        status = "ERROR"
    CLOUDWATCH_CLIENT.put_metric_data(
        Namespace="CropMonitoring",
        MetricData=[
            {"MetricName":"Latency","Value":latency or 0,"Unit":"Milliseconds"},
            {"MetricName":"Status","Value":0 if status=="OK" else 1,"Unit":"Count"}
        ]
    )

if __name__ == "__main__":
    while True:
        monitor()
        time.sleep(60)  # poll every minute

Further Reading

• FAO (2019) — Remote Sensing and GIS for Crop Monitoring: Guidance for Small-Scale Farmers. Provides practical protocols for using satellite imagery in precision agriculture.
• Goodchild, M. & Huang, J. (2021) — Artificial Intelligence in Precision Agriculture: A Review. Summarises recent AI algorithms applied to yield prediction and disease detection.
• ISO (2022) — ISO 19115-3:2022 Geographic information — Metadata — Part 3: Extensions for remote sensing. Sets standards for documenting remote sensing datasets used in crop monitoring.
• Raj, A. et al. (2020) — Machine Learning for Crop Yield Forecasting Using Multi‑Sensor Data. Demonstrates ensemble models that integrate UAV imagery and weather data.
• European Commission (2023) — EU Horizon 2025 Smart Farming Programme. Outlines policy priorities and funding for AI‑driven agri‑tech innovations.