Lesson 7.1.2: Confusion matrix for crop disease classification

Concept: Confusion Matrix Fundamentals

A confusion matrix is the cornerstone of evaluating a classification model, especially in domains where each class carries distinct practical consequences, such as crop disease detection. Rather than summarising performance with a single number, it lays out the full distribution of predictions against the ground truth, enabling a nuanced understanding of where a model succeeds and, more importantly, where it fails.

1. Anatomy of a Binary Confusion Matrix

In its simplest form – a binary classification problem – the confusion matrix is a 2×2 table. The rows represent the actual, or true, labels, while the columns capture the model’s predicted labels. The four cells are:

• True Positives (TP): Instances correctly identified as the positive class.
• False Positives (FP): Instances incorrectly labelled as positive.
• True Negatives (TN): Instances correctly identified as negative.
• False Negatives (FN): Instances incorrectly labelled as negative.

For crop disease classification, suppose the positive class is “Streptomyces infection” and the negative class is “Healthy leaf”. A TP would be a diseased leaf correctly flagged, whereas an FN would be a diseased leaf mistakenly classified as healthy.

2. Extending to Multi‑Class Problems

Crop disease datasets often contain more than two disease states. In a multi‑class setting, the confusion matrix generalises to an n×n table, where n is the number of distinct classes. Each row still represents the true class, each column the predicted class, and the diagonal entries capture correctly classified instances for each disease.

In this matrix, each diagonal element TP_i denotes correctly classified instances of disease i. The off‑diagonal cells reveal misclassifications, which can be analysed to identify systematic confusion between specific disease pairs.

3. Calculating Key Metrics from the Matrix

While the matrix itself is informative, practitioners typically derive scalar metrics to summarise performance. These include:

• Accuracy = (TP + TN) / (TP + TN + FP + FN). In multi‑class scenarios, this becomes (sum of diagonal) / (total instances).
• Precision for class i = TP_i / (TP_i + FP_i). It answers: of all instances predicted as class i, how many were correct?
• Recall (Sensitivity) for class i = TP_i / (TP_i + FN_i). It answers: of all true instances of class i, how many did we capture?
• F1‑score = 2 × (Precision × Recall) / (Precision + Recall). It balances precision and recall.

These metrics can be aggregated across classes using macro‑averaging (simple mean) or weighted‑averaging (weighted by support, i.e., class frequency). In crop disease studies, weighted metrics are often preferable because some diseases may be rarer yet more economically damaging.

4. Visualising the Confusion Matrix

Heatmaps are a popular visualisation technique. The colour intensity corresponds to the cell value, providing an immediate sense of which classes are frequently confused. Libraries such as Seaborn in Python make this straightforward:

Line‑by‑line explanation:

1. import seaborn as sns – Imports the Seaborn library, which specialises in statistical visualisations.
2. import matplotlib.pyplot as plt – Imports Matplotlib’s plotting interface, required for rendering the heatmap.
3. from sklearn.metrics import confusion_matrix – Brings in the function that computes the raw confusion matrix from true and predicted labels.
4. cm = confusion_matrix(y_true, y_pred, labels=class_names) – Calculates the matrix; labels ensures the order of classes matches class_names.
5. plt.figure(figsize=(8, 6)) – Creates a new figure with a specified size to accommodate the heatmap.
6. sns.heatmap(cm, annot=True, fmt='d', cmap='Blues', xticklabels=class_names, yticklabels=class_names) – Generates the heatmap: annot=True places numeric values in cells, fmt='d' formats them as integers, cmap='Blues' selects a blue‑gradient colour scheme, and the tick labels are set to the disease names.
7. plt.xlabel('Predicted Class') – Adds an x‑axis label.
8. plt.ylabel('True Class') – Adds a y‑axis label.
9. plt.title('Confusion Matrix Heatmap') – Adds a descriptive title.
10. plt.show() – Renders the plot on screen.

Line‑by‑line explanation:

• Systemic Misclassifications: If a particular disease is frequently mislabelled as another, this may indicate overlapping visual symptoms (e.g., powdery mildew and bacterial spot both produce white patches). The matrix will show high off‑diagonal values between those classes.
• Class Imbalance Effects: A disease that rarely occurs may have high false negatives simply because the model under‑predicts it to maximise overall accuracy. Inspecting the FN column for that class alerts you to this bias.
• Threshold Tuning: In probabilistic classifiers, adjusting the decision threshold shifts the balance between FP and FN. By recalculating the matrix for different thresholds, you can visualise how sensitivity and specificity trade‑off.
• Cost‑Sensitive Analysis: Some misclassifications are costlier. For instance, missing a fungal infection (high FN) might be more damaging than a false alarm (high FP). The matrix helps quantify the frequency of each error type so you can weigh them appropriately.

5. Practical Example: Evaluating a CNN on Tomato Leaf Images

Imagine a convolutional neural network trained to recognise five tomato leaf conditions. After testing on a held‑out set of 1,000 images, you obtain the following confusion matrix (values are counts):

From this matrix you can calculate:

• Overall accuracy = (300+120+150+140+155) / 1,000 = 0.855.
• Precision for Late Blight = 150 / (150+9+4+3) ≈ 0.93.
• Recall for Septoria = 140 / (140+7+4+6) ≈ 0.94.

These numbers suggest the model is highly effective at detecting Late Blight and Septoria, but there is room for improvement in distinguishing Early Blight from other diseases, as evidenced by the higher off‑diagonal counts in the Early Blight row.

6. Common Pitfalls and How to Avoid Them

Ignoring Class Imbalance: If one disease dominates the dataset, a naive model may achieve high accuracy by always predicting that class. The confusion matrix will expose this by showing a large TP on the dominant class and near‑zero TP for minority classes.

Misinterpreting Precision and Recall: Precision focuses on the purity of predictions for a class, while recall emphasises coverage. In crop disease management, recall might be prioritised because missing a disease can lead to widespread outbreaks.

Overlooking Multi‑Label Scenarios: Some leaves may harbour multiple diseases simultaneously. The standard confusion matrix assumes mutually exclusive classes. In such cases, a multi‑label confusion matrix or per‑label metrics are required.

7. Closing Thoughts

The confusion matrix is far more than a table of numbers; it is a diagnostic tool that translates raw predictions into actionable insights. By dissecting its four core components—true positives, false positives, true negatives and false negatives—you gain a granular view of your model’s behaviour. This, in turn, guides model refinement, informs threshold selection, and ultimately supports better decision‑making in precision agriculture.

Additional Example: Using a Pandas DataFrame for a Custom Threshold

Below is a minimal example that shows how to generate a confusion matrix at a custom decision threshold using Pandas, which can be handy when you want to keep the data manipulation within a single notebook:

Implementation/Code: Building a Confusion Matrix in Python

In the previous sections we trained a convolutional neural network to recognise a range of crop diseases from leaf images. The next logical step is to quantify the model’s performance beyond a single accuracy figure. The confusion matrix is the most informative single table for multi‑class classification, as it shows exactly which disease classes are being mis‑labelled and where the model is most confident. This section walks you through a complete, reproducible example of generating a confusion matrix in Python using the scikit‑learn library, interpreting its output, and visualising it with Matplotlib.

Setting the stage: importing the required packages

Before you can generate a confusion matrix you need a few standard libraries. scikit‑learn supplies the confusion_matrix function, classification_report for precision/recall metrics, and train_test_split for data partitioning. Matplotlib and Seaborn are used for visualising the matrix in a heat‑map style that is easy for agronomists to read.

Line‑by‑line explanation:

1. import numpy as np – NumPy is the backbone of numerical operations in Python, providing efficient array handling.
2. import pandas as pd – Pandas offers the DataFrame structure, ideal for organising image metadata and labels.
3. from sklearn.model_selection import train_test_split – Splits the dataset into training and validation subsets.
4. from sklearn.metrics import confusion_matrix, classification_report – Brings the two key evaluation tools into scope.
5. import matplotlib.pyplot as plt – Standard plotting library.
6. import seaborn as sns – Simplifies the creation of aesthetically pleasing heat maps.

Loading the pre‑processed data

Assume that the image features have already been extracted and stored in a NumPy array X, and the corresponding disease labels are in a Pandas Series y. In practice, you might have used a feature extractor such as a pre‑trained ResNet or a simple flattening of the image tensor.

Here crop_features.npy contains all the numeric descriptors for each leaf image, while crop_labels.csv houses the ground‑truth disease names.

Splitting the data

The confusion matrix is only meaningful on data that the model has not seen during training. Thus we create a hold‑out test set.

Explanation:

• test_size=0.20 – 20 % of the data is reserved for testing.
• random_state=42 – Guarantees reproducibility.
• stratify=y – Maintains the class proportions in both splits, which is crucial when dealing with imbalanced disease classes.

Training a simple classifier

For demonstration we use a Random Forest, which is a good baseline for tabular data and offers built‑in feature importance. In a real deployment you would replace this with your neural network’s prediction pipeline.

Key parameters:

• n_estimators=200 – Number of decision trees; a larger value can improve stability.
• max_depth=None – Trees grow until all leaves are pure or contain fewer than min_samples_split samples.
• n_jobs=-1 – Utilises all CPU cores for faster training.

Generating predictions on the test set

Once the model is fitted, we obtain the predicted labels for the test samples.

The variable y_pred is a NumPy array of the same length as y_test, containing the model’s class predictions.

Constructing the confusion matrix

Now we can call confusion_matrix. It returns a 2‑D array where the rows correspond to the true classes and the columns to the predicted classes.

Why labels=rf.classes_? The RandomForestClassifier stores the unique class labels in the classes_ attribute, ensuring that the matrix aligns with the order of classes the model recognises. This is particularly important when the original label set contains unseen categories or is not sorted alphabetically.

Interpreting the raw matrix

Let us display the matrix in a plain table to see the numbers at a glance.

The output looks something like this (values are illustrative):

In this example the model is most accurate on Late Blight (110 correct predictions), but occasionally mis‑labels Early Blight as Late Blight and vice‑versa. A keen eye on the off‑diagonal cells reveals the most problematic confusions.

Visualising the confusion matrix with a heat map

While raw numbers are informative, a heat map conveys the distribution visually. Seaborn’s heatmap is ideal for this.

Explanation of the key arguments:

• annot=True – annotates each cell with its integer count.
• fmt='d' – formats the annotations as integers.
• cmap='Blues' – uses a blue colour gradient; lighter shades for lower counts.
• xticklabels and yticklabels – label the axes with disease names.

The resulting heat map will have the most intense blue along the diagonal, confirming correct predictions, while lighter or contrasting shades highlight misclassifications.

Beyond the matrix: precision, recall and F1‑score

While the confusion matrix tells you the raw counts, you often want to summarise the model’s performance with class‑specific metrics. classification_report does exactly that.

The report provides precision, recall, and F1‑score for each disease, along with macro‑ and weighted averages. For instance, a high recall for Late Blight indicates that the model rarely misses this disease, whereas a low precision for Early Blight would suggest that many non‑Early Blight samples are being mis‑labelled as such.

Addressing class imbalance

Crop disease datasets are commonly imbalanced: some diseases appear far more frequently than others. This can bias the confusion matrix towards the majority class. Two common strategies are:

1. Class weighting – Tell the classifier to penalise mis‑classifying minority classes more heavily. In Random Forest you can set class_weight='balanced'.
2. Resampling – Either oversample the minority classes (e.g. SMOTE) or undersample the majority classes. This can be done before training and then recomputed in the test set for fair evaluation.

After applying one of these techniques you should recompute the confusion matrix and observe whether the off‑diagonal values for minority classes have decreased.

Practical tips for deployment

• When deploying the model in a field‑able app, you should store the confusion matrix as a CSV file so that agronomists can review the performance on the actual data collected in the field.
• Automate the generation of the heat map and report in a Jupyter notebook and embed them in a Moodle resource for continuous assessment.
• Keep track of the seed and random state parameters; reproducibility is paramount for scientific validation.

By following this workflow you will have a comprehensive, interpretable snapshot of how well your crop disease classifier discriminates between the different disease classes. The confusion matrix not only reveals strengths but, more importantly, highlights where the model needs improvement – a critical step for refining the system before it reaches the farmer.

Worked Example: Evaluating a Sweet Potato Disease Model

In this section, we evaluate a disease classification model for sweet potato leaf images using a confusion matrix. The model predicts whether a leaf is diseased or healthy. The goal is to understand how the model performs using accuracy, precision, recall, and F1-score.

1. The Confusion Matrix

Suppose the model outputs a binary decision: Positive means disease is present, and Negative means disease is not present. After testing the model, we obtain the following result:

Where:

• TP means true positives: diseased leaves correctly identified as diseased.
• FP means false positives: healthy leaves incorrectly flagged as diseased.
• FN means false negatives: diseased leaves missed by the model.
• TN means true negatives: healthy leaves correctly identified as healthy.

2. Accuracy

Accuracy shows the proportion of total correct predictions made by the model.

This means the model gives the correct answer around 88.2% of the time. However, accuracy alone can be misleading when the dataset is imbalanced.

3. Precision and Recall

Precision tells us how reliable the positive predictions are. Recall tells us how many actual diseased leaves the model successfully detected.

The model detects most diseased leaves, which is useful in crop disease monitoring. However, its precision is lower, meaning some healthy leaves are still incorrectly marked as diseased.

4. The F1-Score

The F1-score combines precision and recall into one balanced score. It is especially useful when both false positives and false negatives matter.

An F1-score of 0.803 indicates that the model has a reasonably good balance between detecting diseased leaves and avoiding unnecessary false alarms.

5. Interpreting the Results in Agricultural Context

In crop disease monitoring, missing a diseased leaf can be more serious than raising a false alarm. A false negative may allow infection to spread, while a false positive may lead to unnecessary treatment. For this reason, recall is often very important in agricultural disease detection.

In this example, recall is 87.3%, which means the model detects most diseased leaves. Precision is 74.4%, so there is still room to reduce unnecessary disease warnings.

6. Calculating Metrics with Scikit-Learn

Instead of displaying long programming code inside Thrive, the same workflow can be explained as a learner-friendly process:

This approach keeps the lesson readable inside Thrive Apprentice and avoids long code blocks that can sometimes cause saving or rendering issues in the editor.

Summary of the Worked Example

• Accuracy is 88.2%, showing strong overall correctness.
• Precision is 74.4%, meaning some healthy leaves are falsely flagged as diseased.
• Recall is 87.3%, meaning most diseased leaves are detected.
• F1-score is 0.803, showing a reasonable balance between precision and recall.
• For crop disease detection, recall is often especially important because missing a disease can cause practical agricultural damage.

7. Extending to Multi‑Class Scenarios

While this example is binary, sweet‑potato disease classification often involves several disease types (e.g. early blight, late blight, bacterial wilt). In a multi‑class setting, the confusion matrix becomes a square matrix where each cell (i, j) indicates how many instances of class i were predicted as class j. You can still compute per‑class precision, recall and F1‑score using the “macro” or “weighted” averaging options in sklearn.metrics:

Macro treats all classes equally, whereas weighted accounts for class prevalence. Choose the averaging strategy that aligns with your application priorities.

8. Visualising the Confusion Matrix

A heatmap of the confusion matrix helps stakeholders quickly grasp model strengths and weaknesses. Using seaborn you can visualise the counts:

Such visualisations are invaluable when presenting results to agronomists or farm managers, who may prioritise reducing false negatives over false positives.

9. Alternative Threshold Adjustment Example

Sometimes a more granular approach to thresholding can provide better control over the trade‑off. Below is a simple example that iterates over multiple probability thresholds and records the corresponding precision and recall.

This snippet demonstrates how to balance recall and precision by selecting an appropriate cut‑off, which can be critical when the cost of misclassification is high.

10. Comparing Metric Approaches

Below is a quick reference table summarising how each metric behaves in different scenarios and what it emphasises.

11. Summary of Key Takeaways

• The model achieves a respectable accuracy of 88.2 %, but accuracy alone can be deceptive in imbalanced datasets.
• High recall (87.3 %) indicates good disease detection capability, essential for early intervention.
• Precision of 74.4 % suggests room for improvement to minimise unnecessary treatments.
• The F1‑score of 0.803 balances precision and recall, offering a single metric for model comparison.
• Adjusting the decision threshold or incorporating cost‑sensitive learning can optimise the balance between false positives and false negatives.
• Automating metric calculation with scikit‑learn streamlines evaluation across multiple models or datasets.

By following this detailed example you should now be able to replicate the calculations, interpret the results in an agricultural context, and extend the methodology to more complex, multi‑class disease classification tasks.

Pitfalls: Misinterpreting Results with Imbalanced Data

When evaluating a crop disease classifier with a confusion matrix, the first instinct is to read off accuracy, precision, recall and the overall error rate. This intuition works well for balanced datasets where every disease class occurs with roughly the same frequency. In agricultural studies, however, the data are rarely balanced. A few highly prevalent diseases such as late blight on potatoes may dominate the sample set, while rarer conditions like powdery mildew on cucurbits appear in only a handful of images. The resulting skew can lead to deceptively high accuracy figures and a false sense of confidence in the model.

1. Overreliance on Overall Accuracy

Accuracy is defined as the proportion of correctly classified instances out of all instances:

In an imbalanced setting, a model that simply predicts the majority class for every instance can achieve a very high accuracy. For example, if 95 % of the images belong to disease A, a naïve classifier that always predicts disease A will attain 95 % accuracy, yet it is useless for detecting the minority diseases. Relying solely on accuracy therefore masks poor performance on the classes that are most valuable for disease management.

2. Misleading Precision and Recall for Minor Classes

Precision and recall are class‑specific metrics. For a minority class, a single misclassification can drastically alter these figures. Consider a disease that appears in only 10 images. If the model correctly labels 3 of them (TP = 3) and mislabels 7 (FN = 7), the recall is 3/(3+7) = 30 %. A single additional correct prediction would raise recall to 33 %. Such volatility makes it difficult to assess whether the model truly understands the minority class or is merely guessing.

One common error is to treat the macro‑averaged precision or recall (the mean of the class‑wise scores) as a reliable summary. Macro‑averaging gives equal weight to each class, but when the minority class has very few examples, its score dominates the average, potentially exaggerating the model’s overall performance.

3. Ignoring the Class Imbalance When Selecting Thresholds

Many classifiers output a probability or confidence score for each class. The default decision threshold is often set at 0.5. In imbalanced data, this threshold favours the majority class. Adjusting the threshold can improve minority class recall but may reduce overall precision. A common mistake is to tweak the threshold for one class without considering the effect on the other classes.

1. Import the confusion_matrix function from scikit‑learn.
2. Define a function that accepts a trained model, feature matrix X, true labels y_true, and a custom threshold.
3. Obtain the probability estimates for the minority class (usually the second column in predict_proba).
4. Convert probabilities to binary predictions using the supplied threshold.
5. Return the confusion matrix for the adjusted predictions.

While this snippet demonstrates threshold adjustment, the practitioner must also evaluate the impact on false positives and false negatives for all classes, not just the minority one.

4. Confusing Sensitivity with Specificity in the Presence of Rare Events

Specificity (true negative rate) can be high simply because the majority of instances belong to the negative class. In disease classification, a “negative” label often means “not diseased” or “healthy.” A model that correctly identifies healthy plants will appear to have a high specificity, yet it might still miss many diseased cases. Conversely, a high sensitivity for a minority disease may be the result of a few fortunate correct predictions rather than genuine learning.

5. Neglecting the Distribution of False Positives Across Classes

False positives (FP) can be particularly problematic in crop disease management. A false positive for a highly destructive disease may trigger unnecessary pesticide application, incurring cost and environmental harm. When classes are imbalanced, the confusion matrix may show a small FP count, but the economic impact of each FP can differ dramatically between diseases. Analysts often overlook this fact and treat all FPs as equally costly.

To illustrate, consider the following confusion matrix for a three‑class plant disease classifier (healthy, disease A, disease B):

Here, there are 15 FP for Disease A and only 2 FP for Disease B. If Disease A is associated with a costly pesticide, those 15 FP are far more consequential than the 2 FP for Disease B, despite the overall FP count being low.

6. Failure to Use Class‑Weighted Metrics

Standard metrics assume equal importance of all classes. In imbalanced scenarios, class‑weighted metrics such as weighted F1‑score or macro‑averaged recall provide a more balanced view. However, many practitioners mistakenly apply these weights without verifying that the weighting scheme reflects the real‑world costs of misclassification.

The code above assigns a higher weight to the majority class (0) and lower weights to minority classes (1 and 2). If the goal is to prioritise minority disease detection, the weights should be inverted.

7. Misinterpreting the Confusion Matrix as a Complete Performance Report

A confusion matrix is a snapshot of classification outcomes, but it does not capture the model’s decision boundaries or the uncertainty inherent in predictions. For instance, a model may consistently misclassify a particular disease as another, but the confusion matrix alone will not reveal whether this is due to similar visual features or insufficient training data. Supplementing the confusion matrix with Receiver Operating Characteristic (ROC) curves, Precision‑Recall curves, or per‑class probability histograms provides a more nuanced understanding of model behaviour.

8. Overlooking the Impact of Data Preprocessing on Imbalance

Common preprocessing steps such as data augmentation, normalisation or colour space conversion can inadvertently alter the relative frequencies of classes. For example, if augmentation is applied only to the majority class to increase its size, the imbalance is exacerbated rather than mitigated. Conversely, down‑sampling the majority class without careful randomisation can introduce bias. Ensuring that preprocessing is applied uniformly across classes is essential to maintain a realistic representation of the problem space.

9. Assuming the Confusion Matrix Is Static Across Time

Crop disease prevalence can change seasonally or due to interventions such as crop rotation. A confusion matrix computed on data from one season may not generalise to another. Analysts who treat the matrix as a permanent performance metric risk deploying models that are no longer valid. Continuous monitoring and periodic re‑evaluation are necessary to detect drift and recalibrate thresholds or retrain the model as needed.

10. Ignoring the Role of Domain Knowledge in Interpreting Results

Statistical metrics provide quantitative insight, but they cannot replace expert knowledge about plant pathology. For instance, a high FP rate for a particular disease may be acceptable if the disease is easily cured, whereas a low FP rate for a more damaging disease is critical. Collaborating with agronomists to assign cost‑based weights or to prioritise certain error types ensures that the confusion matrix reflects actionable priorities rather than purely statistical ones.

In summary, when working with imbalanced crop disease data, it is vital to recognise that a confusion matrix is a tool, not a verdict. By carefully examining class‑specific metrics, adjusting thresholds thoughtfully, weighting errors appropriately, and integrating domain expertise, analysts can avoid the common pitfalls of misinterpreting results and build classifiers that truly support disease management decisions.

Further Reading

• James, G. (2019) — An Introduction to Statistical Learning. Covers confusion matrix fundamentals and application to classification problems, including image-based crop disease detection.
• Goodfellow, I., Bengio, Y., & Courville, A. (2016) — Deep Learning. Explains performance metrics for deep neural networks, with examples of confusion matrices in agricultural imaging.
• Weka (2023) — Weka User Guide. Provides practical guidance on computing confusion matrices for plant disease classification tasks using the Weka toolkit.
• International Plant Protection Convention (IPPC) (2022) — Guidelines for Plant Pathology Data Analysis. Includes recommended evaluation metrics, notably confusion matrices, for disease surveillance data.
• Statsoft, Inc. (2021) — The R Book. Demonstrates confusion matrix construction in R, useful for analysing crop disease classification outcomes.