Lesson 7.2.1: Importance of explainability in AI-based crop diagnosis

flowchart TD
    A[AI-based Crop Diagnosis] --> B[Explainability]
    B --> C[Trust in Model]
    C --> D[Farmer Decision-Making]
    B --> E[Regulatory Compliance]
    D --> F[Improved Crop Outcomes]
    E --> F

# Example: Using SHAP to explain a leaf disease classifier
import shap
import torch
import torchvision.transforms as T
from PIL import Image
import numpy as np

Load trained PyTorch model (same as earlier)
model = torch.load('blight_resnet18.pth')
model.eval()

Define preprocessing (same as before)
preprocess = T.Compose([
    T.Resize((224, 224)),
    T.ToTensor(),
    T.Normalize(mean=[0.485, 0.456, 0.406],
                std=[0.229, 0.224, 0.225]),
])

Load image
img = Image.open('example_leaf.jpg')
input_tensor = preprocess(img).unsqueeze(0)

Convert to NumPy for SHAP
input_np = input_tensor.numpy()

Create a SHAP KernelExplainer for the model
def model_predict(data):
    data_tensor = torch.tensor(data, dtype=torch.float32)
    with torch.no_grad():
        outputs = model(data_tensor)
    return outputs.softmax(dim=1).numpy()

explainer = shap.KernelExplainer(model_predict, input_np)

Compute SHAP values for the predicted class
pred_class = np.argmax(model_predict(input_np), axis=1)[0]
shap_values = explainer.shap_values(input_np, nsamples=50)[pred_class]

Visualise SHAP heatmap
shap.image_plot([shap_values], [img])

Script skeleton
import torch
from torchvision import models, transforms
from PIL import Image
import matplotlib.pyplot as plt
import numpy as np

model = models.resnet18(pretrained=True)
model.fc = torch.nn.Linear(model.fc.in_features, 10)
model.load_state_dict(torch.load('leaf_disease_resnet18.pth'))
model.eval()

transform = transforms.Compose([...])  # as defined earlier

img = Image.open('example_leaf.jpg').convert('RGB')
tensor = transform(img).unsqueeze(0)

Compute saliency map (refer to compute_saliency function)
saliency = compute_saliency(tensor, model)

Overlay and display
overlay_saliency('example_leaf.jpg', saliency)

Implementation/Code: Building a Saliency‑Map Pipeline

To visualise what a convolutional neural network (CNN) has actually learned when diagnosing leaf pathology, we generate saliency maps. These maps highlight image regions that exert the greatest influence on the network’s decision, thereby offering a transparent window into the model’s reasoning. Below is a detailed, step‑by‑step exposition of a minimal yet complete pipeline that takes a pre‑trained CNN, processes leaf images, and produces saliency visualisations. The code is written in Python using the PyTorch ecosystem, which remains the most widely adopted framework for research‑grade computer vision.

1. Environment Setup

Before diving into the algorithm, initialise a virtual environment and install the required packages. We emphasise reproducibility, so pin package versions where possible.

# Create virtual environment
python3 -m venv saliency-env
source saliency-env/bin/activate

Install dependencies
pip install torch==2.4.0 torchvision==0.19.0 pillow==10.4.0 matplotlib==3.9.0

PyTorch 2.4.0 offers the torch.autograd.functional.jacobian utilities, which simplify gradient extraction. The torchvision package supplies standard data transforms and a convenient models module.

2. Load a Pre‑trained CNN

In many agricultural settings, a ResNet‑50 architecture pre‑trained on ImageNet provides a solid feature extractor. We fine‑tune it on our leaf dataset, but for saliency generation the exact weights are immaterial – the key is that the network has learned discriminative filters.

import torch
import torchvision.models as models
import torch.nn as nn

Load a ResNet-50 pre‑trained on ImageNet
model = models.resnet50(pretrained=True)

Replace the final fully‑connected layer to match our 10‑class leaf disease taxonomy
num_ftrs = model.fc.in_features
model.fc = nn.Linear(num_ftrs, 10)

Assume model weights have been fine‑tuned and saved previously
model.load_state_dict(torch.load('leaf_disease_resnet50.pth'))
model.eval()  # set to evaluation mode

Line by line:

• import torch and import torchvision.models as models bring in the core deep‑learning library and the model zoo.
• The ResNet‑50 constructor loads a network with ImageNet weights. Setting pretrained=True ensures we start from a robust feature set.
• We inspect the fully‑connected layer (model.fc) and replace it with a new nn.Linear that outputs ten logits, one per disease class.
• Weights previously saved after fine‑tuning are loaded with torch.load.
• model.eval() switches the network into inference mode, disabling dropout and batch‑norm updates.

3. Prepare the Input Data

Leaf images come in a variety of sizes and colour profiles. The standardisation pipeline involves resizing, centre‑cropping, normalisation with ImageNet statistics, and conversion to a Torch tensor.

from torchvision import transforms
from PIL import Image

Define a transform pipeline
transform = transforms.Compose([
    transforms.Resize(256),               # Scale shorter side to 256
    transforms.CenterCrop(224),           # Crop to the model's expected input size
    transforms.ToTensor(),                # Convert to a [0,1] tensor
    transforms.Normalize(                 # Normalise with ImageNet mean & std
        mean=[0.485, 0.456, 0.406],
        std=[0.229, 0.224, 0.225]
    )
])

def load_image(path):
    "Load and transform a single image."
    img = Image.open(path).convert('RGB')
    return transform(img).unsqueeze(0)  # Add batch dimension

Explanation:

• transforms.Resize(256) ensures the image is large enough for a centre crop.
• transforms.CenterCrop(224) matches ResNet‑50's expected input.
• transforms.ToTensor() converts the PIL image into a 3‑channel tensor with values in [0,1].
• Normalisation uses the ImageNet dataset's mean and standard deviation, which aligns the data distribution with the network's training regime.
• load_image function returns a batched tensor ready for model inference.

4. Computing the Saliency Map

We focus on the most straightforward saliency method: gradient of the predicted class score with respect to the input image. This yields a 3‑channel map that we later visualise as a heatmap overlay.

import torch.nn.functional as F
import numpy as np
import matplotlib.pyplot as plt

def compute_saliency(img_tensor, model, target_class=None):
    "Generate a saliency map for a given image tensor.

    Parameters
    ----------
    img_tensor : torch.Tensor
        Input image with shape (1, 3, H, W) and requires_grad=True.
    model : nn.Module
        Pre‑trained CNN in evaluation mode.
    target_class : int, optional
        Index of the class to compute gradients for. If None, use the predicted class.

    Returns
    -------
    saliency : np.ndarray
        2‑D array of saliency magnitudes, shape (H, W).
    "
    # Enable gradient tracking on the input
    img_tensor.requires_grad_(True)

    # Forward pass
    outputs = model(img_tensor)  # shape: (1, num_classes)

    # Determine target class
    if target_class is None:
        target_class = outputs.argmax(dim=1).item()

    # Extract the scalar score for the target class
    score = outputs[0, target_class]

    # Backward pass to compute gradients w.r.t input
    model.zero_grad()
    score.backward()

    # The gradient of the score w.r.t the input image
    gradient = img_tensor.grad.data[0]  # shape: (3, H, W)

    # Compute the absolute value and collapse across channels
    saliency = gradient.abs().max(dim=0)[0]  # shape: (H, W)

    # Convert to CPU numpy array
    saliency_np = saliency.cpu().numpy()

    # Normalise to [0, 1] for visualisation
    saliency_np -= saliency_np.min()
    saliency_np /= saliency_np.max()

    return saliency_np

Line‑by‑line walk‑through:

• We set requires_grad_(True) on the input tensor to allow PyTorch to record operations for gradient computation.
• The forward pass yields logits for all ten classes. If no target class is supplied, we default to the model’s top‑predicted class.
• Calling score.backward() triggers autograd to compute the partial derivatives of the chosen class score with respect to every input pixel.
• We zero out existing gradients on the model to prevent accumulation.
• The gradient tensor has the same shape as the input: three channels (RGB) times height and width. Taking the maximum absolute gradient across the colour channels gives us a single saliency value per pixel.
• Finally, normalisation to the [0,1] range prepares the map for overlay with the original image.

5. Visualising the Saliency Map

To interpret the saliency map, we overlay it as a translucent heatmap onto the original leaf image. The following helper visualises the result.

def overlay_saliency(original_img_path, saliency, alpha=0.4, cmap='hot'):
    "Overlay saliency heatmap onto the original image.

    Parameters
    ----------
    original_img_path : str
        File path to the source image.
    saliency : np.ndarray
        2‑D saliency array with values in [0,1].
    alpha : float
        Transparency of the heatmap.
    cmap : str
        Matplotlib colormap to use.
    "
    # Load original image for display
    orig = Image.open(original_img_path).convert('RGB')
    orig = np.array(orig)

    # Resize saliency to match original image dimensions
    saliency_resized = np.array(Image.fromarray(saliency).resize(orig.shape[:2][::-1]))

    # Create figure
    plt.figure(figsize=(6, 6))
    plt.imshow(orig)
    plt.imshow(saliency_resized, cmap=cmap, alpha=alpha)
    plt.axis('off')
    plt.title('Saliency Overlay')
    plt.show()

Key points:

• We open the original image with Pillow and convert it to a NumPy array.
• Because the model processes 224×224 images, saliency maps are resized back to the original resolution using Pillow’s resize method.
• Matplotlib’s imshow is called twice: first for the RGB image, then for the heatmap with a chosen colormap (e.g., 'hot') and transparency alpha.

6. End‑to‑End Example

Below is a concise script that ties all components together. It processes a single leaf image, computes its saliency map, and displays the overlay.

if __name__ == "__main__":
    # Path to an example leaf image
    img_path = 'data/leaf_samples/leaf_001.jpg'

    # Load and transform image
    img_tensor = load_image(img_path)

    # Compute saliency map
    saliency_map = compute_saliency(img_tensor, model)

    # Visualise
    overlay_saliency(img_path, saliency_map)

When executed, the output will display the leaf with a reddish‑orange heatmap highlighting the most salient regions. Typically, disease spots or necrotic lesions will register high saliency, confirming that the model is focusing on biologically relevant features rather than background artefacts.

7. Extending the Pipeline

While the gradient‑based saliency described above offers immediate insight, several refinements can enhance interpretability:

1. Integrated Gradients – Accumulate gradients along a path from a baseline image to the input, mitigating noise in raw gradients.
2. Guided Backpropagation – Modify the backward pass to suppress negative gradients, yielding sharper visualisations.
3. Grad‑CAM – Compute class‑specific localisation maps using the activations of the final convolutional layer, providing more global context.
4. Model‑agnostic methods – Techniques such as SHAP or LIME can be adapted to image data, offering alternative explanations that are not tied to a specific architecture.

Implementing these methods follows a similar structure: forward pass, gradient extraction, aggregation, normalisation, and overlay. The choice of technique should align with the domain’s regulatory requirements and the end‑user’s need for clarity.

Below is an example of an Integrated Gradients approach, demonstrating how to compute attributions by interpolating from a baseline image (e.g. black image) to the actual input.

def integrated_gradients(img_tensor, model, baseline=None, steps=50, target_class=None):
    "Compute Integrated Gradients for a single image tensor.

    Parameters
    ----------
    img_tensor : torch.Tensor
        Input image with shape (1, 3, H, W), requires_grad=False.
    model : nn.Module
        Pre‑trained CNN in evaluation mode.
    baseline : torch.Tensor, optional
        Baseline image for integration (defaults to a zero tensor).
    steps : int, optional
        Number of interpolation steps.
    target_class : int, optional
        Index of the class to explain; if None, use predicted class.

    Returns
    -------
    integrated_attributions : np.ndarray
        3‑channel attribution map of shape (3, H, W) normalised to [0, 1].
    "
    if baseline is None:
        baseline = torch.zeros_like(img_tensor)

    # Scale inputs and compute gradients
    scaled_inputs = [baseline + (float(i) / steps) * (img_tensor - baseline) for i in range(0, steps + 1)]
    grads = []

    for scaled in scaled_inputs:
        scaled.requires_grad_(True)
        outputs = model(scaled)
        if target_class is None:
            target = outputs.argmax(dim=1).item()
        else:
            target = target_class
        score = outputs[0, target]
        model.zero_grad()
        score.backward()
        grads.append(scaled.grad.detach().clone())

    # Average gradients and multiply by input difference
    avg_grads = torch.stack(grads).mean(dim=0)
    integrated = (img_tensor - baseline) * avg_grads
    integrated_np = integrated.squeeze().cpu().numpy()

    # Normalise to [0,1]
    integrated_np -= integrated_np.min()
    integrated_np /= integrated_np.max()
    return integrated_np

8. Performance Considerations

Generating saliency maps on large datasets can be computationally intensive. To optimise:

• Batch the saliency computation and leverage GPU acceleration. In PyTorch, placing both the model and input tensors on cuda dramatically reduces runtime.
• Cache intermediate activations if you plan to compute multiple explanation types (e.g. both Grad‑CAM and raw gradients) on the same image.
• Use mixed‑precision (FP16) inference where hardware permits; this halves memory usage with negligible impact on gradient fidelity.

9. Interpreting the Results

Saliency maps should not be taken at face value. A high saliency region indicates that small perturbations in that pixel will significantly alter the predicted score. However, this does not guarantee that the model genuinely recognises the underlying pathology; it merely reflects sensitivity. Cross‑validation with domain experts, visual inspection of salient regions, and comparison against known disease markers are essential steps before deploying the model in practice.

10. Summary

We have constructed a reproducible, end‑to‑end pipeline that transforms raw leaf images into interpretable saliency visualisations. By embedding gradient‑based explanations directly into the model inference workflow, agricultural practitioners can gain confidence that the CNN’s decisions are grounded in disease‑specific features. The modular design of the code allows easy substitution of alternative explanation techniques, ensuring that the pipeline remains adaptable to evolving research and industry needs.

Worked Example: Interpreting a Real‑World Diagnosis

In this worked example we take a concrete image of a tomato leaf that has been flagged by a deep‑learning model as infected with Septoria leaf spot. We will walk through the entire saliency‑map pipeline – from loading the image, generating a heatmap with a Grad‑CAM algorithm, overlaying it back onto the leaf and finally interpreting the visual clues that the model has highlighted. By the end of this section you will be able to reproduce the process on your own data and recognise the subtle differences between a trustworthy explanation and a misleading one.

1. Preparing the environment

Before we can create a saliency map we need a few Python packages. In a typical Moodle-based workshop the instructor would have prepared a virtual environment for you, but it is useful to see the explicit imports so that you can replicate the process locally if you wish.

Import standard scientific stack
import numpy as np
import matplotlib.pyplot as plt
from PIL import Image

Deep learning framework
import torch
import torchvision.transforms as T

Grad-CAM implementation
from torchcam.methods import GradCAM
from torchcam.utils import overlay_mask

Line by line: we import numpy for numerical manipulation, matplotlib for plotting, and PIL for image I/O. torch and torchvision provide the model and pre‑processing pipeline. Finally torchcam is a lightweight library that implements Grad‑CAM and a helper to overlay the heatmap.

2. Loading the model and image

We will use a pre‑trained ResNet‑50 fine‑tuned on the PlantVillage dataset. The model is expected to return a class probability distribution over the 38 disease categories. The image we analyse is tomato_leaf_23.jpg, located in the data/ directory.

Load the fine‑tuned model
model_path = 'models/resnet50_plants.pth'
model = torch.load(model_path)
model.eval()  # Set to evaluation mode

Load and pre‑process the leaf image
img_path = 'data/tomato_leaf_23.jpg'
original_img = Image.open(img_path).convert('RGB')

Normalisation values from ImageNet
normalize = T.Normalize(mean=[0.485, 0.456, 0.406],
                       std=[0.229, 0.224, 0.225])

preprocess = T.Compose([
    T.Resize((224, 224)),
    T.ToTensor(),
    normalize,
])

input_tensor = preprocess(original_img).unsqueeze(0)  # Add batch dim

The model.eval() call ensures that dropout layers are disabled and batch normalisation uses running statistics. The preprocessing pipeline resizes the image to the model’s expected size, converts it to a tensor and normalises it with ImageNet statistics – a standard practice that keeps the feature distribution stable.

3. Generating the Grad‑CAM heatmap

Grad‑CAM works by weighting the feature maps of a chosen convolutional layer with the gradients flowing back from the target class. For ResNet‑50 the last convolutional block, layer4, offers a reasonable trade‑off between localisation granularity and computational cost.

Identify the target class
with torch.no_grad():
    outputs = model(input_tensor)
    probs = torch.nn.functional.softmax(outputs, dim=1)
    pred_class = probs.argmax().item()
    pred_label = model.classes[pred_class]  # Assuming a list of class names

Initialise Grad-CAM on the chosen layer
cam_extractor = GradCAM(model, target_layer='layer4')

Compute the saliency map for the predicted class
cam = cam_extractor(pred_class, outputs)
cam is a (1, H, W) tensor – we squeeze the batch dimension
heatmap = cam.squeeze().cpu().numpy()

After obtaining the predicted class, we create a GradCAM instance pointing at layer4. Calling it with the predicted class index and the raw output gives us a heatmap that reflects the importance of each spatial location for that class. The resulting array is normalised automatically by torchcam to the range [0, 1], making it ready for overlay.

4. Visualising the heatmap

We now combine the heatmap with the original leaf image. To preserve the colour of the leaf we colour‑map the heatmap onto a transparent overlay.

Convert heatmap to RGB heatmap
heatmap_rgb = plt.cm.jet(heatmap)[..., :3]  # Drop alpha channel

Resize heatmap to original image size
heatmap_rgb = Image.fromarray((heatmap_rgb * 255).astype(np.uint8))
heatmap_rgb = heatmap_rgb.resize(original_img.size, Image.BILINEAR)

Overlay
overlay = overlay_mask(original_img, np.array(heatmap_rgb) / 255.0, alpha=0.5)

Plot
plt.figure(figsize=(8, 8))
plt.imshow(overlay)
plt.title(f'Grad‑CAM for {pred_label}')
plt.axis('off')
plt.show()

We use the jet colormap to map heat values to RGB, then resize the heatmap to match the original image. The overlay_mask helper blends the heatmap with the leaf image using a 50 % transparency factor. The final plot clearly shows a bright region over the circular blotches typical of Septoria lesions.

5. Interpreting the saliency map

With the visualisation in hand we can ask key questions:

1. Are the highlighted regions biologically plausible? In our case the heatmap concentrates on the dark specks and the surrounding necrotic tissue, exactly where a human agronomist would look for Septoria symptoms.
2. Is the model focusing on irrelevant artefacts? If the heatmap had highlighted the background or the labelling sticker, this would flag a potential bias in the training data.
3. How sharp is the localisation? A very diffuse heatmap suggests the model is uncertain or that the disease manifests in a spread‑out manner. A sharp focus indicates a confident prediction.

For this tomato leaf, the heatmap is both precise and biologically meaningful, giving us confidence that the model is not merely memorising colour patterns but is genuinely learning disease‑specific cues.

6. Potential pitfalls and how to recognise them

Saliency maps are powerful but can be deceiving. Below are common issues and how to detect them:

• Gradient saturation: If the gradients saturate, Grad‑CAM may produce a uniform heatmap. Check the variance of the gradient values; low variance hints at saturation.
• Layer choice: Using too early a convolutional layer yields coarse maps; too late yields overly specific features that may overfit. Visualise maps from multiple layers to confirm consistency.
• Class bias: If the model has been trained on an imbalanced dataset, the saliency may reflect class priors rather than true features. Compare heatmaps across classes to spot systematic biases.
• Post‑processing artefacts: Upscaling a low‑resolution heatmap can introduce blurring that obscures the true focus area. Always verify the heatmap at its native resolution.

By systematically checking these aspects, you can avoid over‑interpreting a saliency map that looks convincing at first glance.

7. Cross‑checking with a baseline explanation

To strengthen confidence in the Grad‑CAM output, we compare it against a simpler saliency technique: Integrated Gradients. This method attributes importance by accumulating gradients along a path from a baseline image (e.g. black image) to the input.

from torchcam.methods import IntegratedGradients

ig_extractor = IntegratedGradients(model, target_layer='layer4')
ig = ig_extractor(pred_class, outputs)
ig_heatmap = ig.squeeze().cpu().numpy()

Visualise side‑by‑side
fig, axes = plt.subplots(1, 2, figsize=(12, 6))
axes[0].imshow(overlay)
axes[0].set_title('Grad‑CAM')
axes[0].axis('off')
axes[1].imshow(ig_heatmap, cmap='jet')
axes[1].set_title('Integrated Gradients')
axes[1].axis('off')
plt.show()

If both methods highlight the same lesion area, the explanation is robust. Divergence between the two may indicate model uncertainty or a failure mode that warrants further investigation.

8. Drawing practical conclusions

From the saliency analysis we can summarise actionable insights:

• The model’s focus aligns with known Septoria symptomatology, suggesting that the training set captured the disease’s visual signature well.
• No significant background or artefact activation is observed, implying that the model is not over‑fitted to spurious correlations.
• Grad‑CAM and Integrated Gradients agree, reinforcing the reliability of the explanation.

These findings provide a solid foundation for deploying the model in a field‑based decision support system: users can trust that the alerts are grounded in the visual evidence they have seen.

9. Extending the pipeline

For larger datasets or real‑time applications, you might:

1. Batch‑process images and save heatmaps to a database for audit trails.
2. Use a lightweight Grad‑CAM variant (e.g. SmoothGrad‑CAM) to reduce noise.
3. Integrate the saliency overlay into a web interface where farmers can click on a highlighted region to receive a textual explanation.

Each of these extensions preserves the core interpretability principle: the model’s decision can be visualised and inspected, which is essential for regulatory compliance and farmer trust.

10. Bonus: Grad‑CAM++ for sharper localisation

Grad‑CAM++ modifies the weighting scheme to consider higher‑order gradients, often yielding more accurate and less diffuse heatmaps. Below is a concise example using torchcam's implementation.

from torchcam.methods import GradCAMPlusPlus

cam_pp_extractor = GradCAMPlusPlus(model, target_layer='layer4')
cam_pp = cam_pp_extractor(pred_class, outputs)
heatmap_pp = cam_pp.squeeze().cpu().numpy()

Visualise
plt.figure(figsize=(8, 8))
plt.imshow(heatmap_pp, cmap='jet')
plt.title('Grad‑CAM++ heatmap')
plt.axis('off')
plt.show()

Comparing Grad‑CAM++ with standard Grad‑CAM often reveals tighter focus on disease lesions, which can be particularly useful when lesions are small or subtle.

11. Comparative table of saliency methods