Lesson 6.2.2: 2D CNN for spatial feature learning

From Definition to Mechanism: What is a 2‑D CNN?

A two‑dimensional convolutional neural network (2‑D CNN) is a specialised deep learning architecture designed to process data with a grid‑like topology, such as images or spectrograms. At its core, a 2‑D CNN replaces the fully connected layers of a traditional feed‑forward network with a sequence of convolutional, pooling, and normalisation layers that preserve spatial relationships while progressively abstracting features.

1. What Makes a CNN “Convolutional”?

The defining operation in a 2‑D CNN is the convolution of an input image with a set of learnable filters (kernels). Mathematically, for an input image \(I \in \mathbb{R}^{H imes W imes C}\) and a filter \(K \in \mathbb{R}^{k_h imes k_w imes C}\), the convolution output at position \((i,j)\) is:

O[i, j] = Σ_{c=1}^{C} Σ_{u=0}^{k_h-1} Σ_{v=0}^{k_w-1} I[i+u, j+v, c] * K[u, v, c]

Because the same filter is applied across the entire image, the network learns translation‑equivariant features: a pattern recognisable anywhere in the image will activate the same neuron. This weight sharing dramatically reduces the number of parameters compared to a fully connected layer, enabling efficient learning even with large inputs.

2. Core Components and Their Roles

• Convolutional Layers: Extract local patterns (edges, textures) by sliding filters over the input. Multiple filters per layer generate a set of feature maps.
• Activation Functions: Introduce non‑linearity. ReLU (Rectified Linear Unit) is the de‑facto standard because it mitigates vanishing gradients and is computationally cheap: \(f(x) = \max(0, x)\).
• Pooling Layers: Down‑sample feature maps to reduce spatial dimensions and computation, while providing a degree of spatial invariance. Common choices are max‑pooling and average‑pooling.
• Batch Normalisation: Normalises activations within a mini‑batch, stabilising training and allowing higher learning rates.
• Fully Connected (Dense) Layers: Typically appear at the network's head, integrating the high‑level features into a final classification or regression output.
• Dropout: Randomly zeroes a fraction of activations during training to prevent over‑fitting.

3. Architectural Variants and Trade‑Offs

While the basic building blocks are consistent, 2‑D CNNs differ in how they stack and connect these layers. Three canonical families illustrate the spectrum of design choices:

1. Shallow CNNs (e.g. LeNet‑5): Two convolutional layers followed by two fully connected layers. Advantages: fast to train, interpretable filters. Disadvantages: limited representational capacity for complex images.
2. Deep CNNs (e.g. VGG‑16/19): 13–16 convolutional layers with small \(3 imes 3\) kernels, interleaved with max‑pooling. Pros: excellent performance on large datasets, simple architecture. Cons: many parameters (~138 M for VGG‑16), high memory consumption.
3. Residual Networks (ResNet): Introduce skip connections that add the input of a block to its output, alleviating vanishing gradients in very deep stacks (e.g. ResNet‑50, ResNet‑101). Pros: state‑of‑the‑art accuracy, efficient training. Cons: more complex implementation, slightly higher inference cost.

Choosing between these variants depends on the application domain. For medical imaging where data is scarce, a shallow network with transfer learning may suffice; for autonomous driving, a deep residual network can capture fine‑grained spatial cues.

4. Spatial Feature Learning in Practice

Consider the task of identifying road signs in an image. A 2‑D CNN learns a hierarchy of spatial features:

1. Low‑level features: Early layers detect edges, corners, and colour blobs. These are general and transferable across tasks.
2. Mid‑level features: Subsequent layers recognise shapes and motifs, such as the circular outline of a stop sign or the triangular shape of a yield sign.
3. High‑level features: The deepest convolutional blocks capture the full geometric configuration of a sign, invariant to scale and minor perspective changes.

During training, back‑propagation adjusts filter weights so that activations corresponding to the target class (e.g. “stop sign”) increase, while irrelevant patterns are suppressed. Visualising feature maps after each convolutional layer often reveals this progressive abstraction.

5. Comparison with Alternative Spatial Modelling Techniques

While 2‑D CNNs dominate image analysis, other neural architectures also handle spatial data:

• Fully Convolutional Networks (FCNs) replace fully connected layers with convolutional layers to produce dense predictions (e.g. semantic segmentation). They preserve spatial resolution but are computationally heavier.
• Graph Neural Networks (GNNs) operate on irregular structures (e.g. point clouds). They can capture non‑grid spatial relationships but require explicit graph construction.
• Recurrent Neural Networks (RNNs) with convolutional gates (Conv‑LSTM) model spatio‑temporal sequences, useful for video analysis. However, they are slower to train and often outperformed by 3‑D CNNs for pure spatial–temporal blocks.

In most static image classification scenarios, a 2‑D CNN strikes the best balance between modelling power, computational efficiency, and ease of deployment.

6. Practical Considerations for Implementation

Implementing a 2‑D CNN in a deep learning framework (e.g. PyTorch, TensorFlow) typically follows the same pattern:

import torch.nn as nn

class SimpleCNN(nn.Module):
    def __init__(self):
        super(SimpleCNN, self).__init__()
        self.conv1 = nn.Conv2d(3, 32, kernel_size=3, padding=1)
        self.bn1   = nn.BatchNorm2d(32)
        self.pool  = nn.MaxPool2d(2, 2)
        self.fc1   = nn.Linear(32  16  16, 10)  # assuming 32x32 input

    def forward(self, x):
        x = self.pool(nn.functional.relu(self.bn1(self.conv1(x))))
        x = x.view(x.size(0), -1)
        x = self.fc1(x)
        return x

Key tips:

• Use padding to preserve spatial dimensions when required.
• Choose kernel size (commonly \(3 imes 3\) or \(5 imes 5\)) based on the scale of expected features.
• Balance depth and width: more layers increase capacity but also risk over‑fitting if data is limited.
• Employ data augmentation (rotations, flips, colour jitter) to expose the network to varied spatial configurations.

By understanding these building blocks and their interactions, one can design, analyse, and optimise 2‑D CNNs for a wide range of spatial learning tasks.

Mechanisms of Spatial Feature Extraction

In a 2‑D convolutional neural network (CNN) the extraction of spatial features is a tightly coupled process that relies on three core operations: convolution, pooling, and non‑linear activation. Each of these operations contributes a distinct capability to the network, and their interaction determines the quality and type of patterns that the model can recognise. This section will analyse how these components work together, compare alternative designs, and illustrate their effects with concrete examples.

Convolution: the feature detector

Convolution is the mathematical operation that slides a learnable kernel across the input image, producing a feature map that highlights local patterns. The kernel, typically of size 3×3 or 5×5, contains weights that are updated during training via back‑propagation.

• Locality – A kernel only looks at a small neighbourhood at a time, which preserves spatial relationships and keeps the number of parameters manageable.
• Weight sharing – The same kernel weights are applied at every spatial location, allowing the network to recognise a pattern regardless of its position.
• Translation invariance – Because the weights are shared, the network can detect the same feature across the whole image.

For example, a 3×3 kernel that learns to detect vertical edges will respond strongly wherever a vertical edge occurs, producing a high activation in the corresponding feature map. If the kernel is trained on a dataset of handwritten digits, it might learn a set of simple primitives (e.g. strokes) that combine later into complex characters.

Alternative convolutional strategies:

1. Depthwise separable convolution – Splits the operation into a depthwise pass (one filter per input channel) followed by a pointwise 1×1 convolution. This reduces computation and parameters, often used in mobile architectures.
2. Dilated (atrous) convolution – Introduces gaps between kernel elements, expanding the receptive field without increasing parameter count, useful for semantic segmentation.
3. Group convolution – Divides channels into groups, each processed independently, a compromise between full depthwise and standard convolution.

Choosing between these alternatives depends on the task: large‑scale image classification benefits from standard convolution for rich feature learning, whereas real‑time applications favour depthwise separable or group convolutions to reduce latency.

Activation: introducing non‑linearity

After convolution, the raw feature map is passed through an activation function. This step injects non‑linearity, enabling the network to model complex relationships.

• ReLU (Rectified Linear Unit) – The most common choice: f(x) = max(0, x). It is computationally cheap and mitigates the vanishing gradient problem.
• Leaky ReLU – Allows a small, non‑zero gradient for negative inputs: f(x) = max(αx, x) with α typically 0.01. It addresses the “dying ReLU” issue.
• ELU (Exponential Linear Unit) – Provides negative outputs that can drive mean activations closer to zero, speeding up learning.
• Swish – A self‑gated activation: f(x) = x * sigmoid(x). Empirically found to outperform ReLU on deeper models.

In practice, ReLU remains the default due to its simplicity. However, when training very deep CNNs or when encountering a high proportion of negative pre‑activations, switching to Leaky ReLU or Swish can stabilise convergence.

Pooling: down‑sampling and invariance

Pooling layers reduce the spatial resolution of feature maps, summarising local regions and providing translation invariance. The two most common types are:

• Max pooling – Selects the maximum value within a window (e.g. 2×2). It preserves the most salient activation, discarding weaker signals.
• Average pooling – Computes the mean value within the window, smoothing the feature map.

Pooling can be interleaved with convolutional layers (e.g. after each block) or applied only at the end. A typical design might use 2×2 max pooling with stride 2 after every two convolutional layers, halving the spatial dimensions while doubling the receptive field.

Alternative pooling strategies include:

1. Strided convolution – Replaces pooling with a convolution that has a stride greater than one. This allows the network to learn the down‑sampling operation.
2. Global average pooling – Reduces each feature map to a single value, often used before a fully connected classifier to reduce overfitting.
3. Spatial pyramid pooling – Pools at multiple scales and concatenates the results, providing multi‑scale context.

Choosing the pooling method affects both computational cost and the nature of the invariance. Max pooling favours strong local responses, while average pooling smooths and can be more robust to noise.

Interplay between convolution, activation, and pooling

While each operation has its own role, the emergent behaviour of a CNN arises from their sequence. Consider the following illustrative pipeline for a simple digit recogniser:

# Layer 1
conv1 = Conv2D(filters=32, kernel_size=3, padding='same')
act1  = ReLU()
pool1 = MaxPool2D(pool_size=2)

Layer 2
conv2 = Conv2D(filters=64, kernel_size=3, padding='same')
act2  = ReLU()
pool2 = MaxPool2D(pool_size=2)

After the first convolution and ReLU, the feature map contains responses to basic edges and corners. The subsequent max‑pooling halves the resolution, preserving the strongest of these responses while discarding less informative activations. The second convolution then aggregates these primitives into more complex motifs (e.g. strokes forming a ‘5’). ReLU again introduces non‑linearity, and the second pooling further abstracts the representation. The final feature map fed to a classifier contains a hierarchical representation: low‑level edges, mid‑level strokes, and high‑level character shapes.

Changing any component alters this hierarchy:

• Replacing ReLU with Leaky ReLU during training can help if the network suffers from dying neurons in deeper layers.
• Using average pooling after the first layer may smooth out noise but could also diminish sharp edges, potentially reducing accuracy on fine‑grained tasks.
• Adding a strided convolution in place of the first pooling layer allows the network to learn the down‑sampling pattern, which can be advantageous when the optimal pooling strategy is dataset‑specific.

Concrete example: edge detection vs. texture learning

Suppose we train a CNN on a dataset of natural images with a focus on recognising textures (e.g. wood grain). The first convolutional layer typically learns edge detectors, as these are the building blocks of texture. With ReLU, only positive edge responses are passed forward, sharpening the distinction between different orientations.

If we instead use average pooling after the first layer, the network will aggregate local edge responses, effectively summarising texture patterns over a larger area. This can improve robustness to small shifts but may also blur fine details that are critical for distinguishing similar textures.

Conversely, for a medical imaging task where subtle intensity variations matter (e.g. tumour boundary detection), retaining max pooling preserves the strongest local responses, which might be essential for accurate localisation. In such cases, a deeper network with multiple convolutional layers before pooling allows the model to build a rich representation before down‑sampling.

Best practice recommendations

• Start with standard 3×3 convolutions, ReLU activations, and 2×2 max pooling with stride 2. This configuration balances expressiveness and efficiency.
• Monitor training for vanishing gradients or dying ReLUs; if observed, switch to Leaky ReLU or Swish.
• When computational constraints are tight, consider depthwise separable convolutions or group convolutions, but maintain max pooling to preserve discriminative features.
• For tasks requiring multi‑scale context, integrate spatial pyramid pooling or replace a single pooling layer with multiple strided convolutions.
• Always visualise feature maps at intermediate layers to understand whether the network is learning the intended spatial patterns.

By carefully orchestrating convolution, activation, and pooling, a 2‑D CNN can learn to extract robust spatial features that generalise well across a wide range of visual tasks.

Implications for Hyperspectral Classification

Hyperspectral imagery, with its hundreds of contiguous spectral bands, offers unparalleled spectral resolution for distinguishing materials. However, the high dimensionality of the data often leads to the curse of dimensionality, spectral mixing, and noise that can obscure subtle material differences. Spatial information, traditionally under‑utilised in early hyperspectral classification pipelines, can mitigate these challenges by providing context that is orthogonal to the spectral domain. The advent of 2‑D Convolutional Neural Networks (CNNs) has made it feasible to extract spatial features at multiple scales, thereby augmenting spectral classifiers and improving overall performance.

1. Spectral–Spatial Complementarity

In a purely spectral approach, each pixel is treated as an independent feature vector. Consequently, classifiers such as Support Vector Machines (SVMs) or Random Forests rely solely on the spectral signature, ignoring the fact that a pixel’s physical meaning is also defined by its surrounding pixels. 2‑D CNNs learn convolutional kernels that capture patterns such as edges, textures and shapes across a neighbourhood. When these spatial features are concatenated with the original spectral vector, the classifier receives a richer representation.

For example, consider a scene with intermingled urban and vegetated patches. Spectrally, a pixel from a grass patch may overlap with the reflectance of a low‑rise building due to mixed‑pixel effects. A 2‑D CNN can learn that the pixel is surrounded by a homogeneous texture typical of vegetation, whereas a building pixel is often adjacent to high‑contrast man‑made edges. This spatial context allows the classifier to disambiguate the two cases, reducing misclassifications that would otherwise arise from spectral overlap alone.

2. Mitigating Spectral Mixing and Shadowing

Spectral mixing occurs when a single pixel contains multiple materials, a frequent issue in coarse‑resolution hyperspectral sensors. Traditional unmixing algorithms attempt to deconvolve these mixtures, but they are computationally intensive and sensitive to noise. Spatial CNNs implicitly perform a form of local unmixing by weighting contributions from neighbouring pixels. If a pixel lies in a shadowed area, its spectrum is attenuated. However, the surrounding pixels often retain the true spectral shape of the material. A 2‑D CNN can learn to discount the shadowed pixel or to adjust its feature representation based on the brightness of its neighbours, thereby correcting for illumination effects without explicit shadow detection.

3. Scale‑Dependent Feature Extraction

Spatial features manifest at different scales: fine‑grained textures such as leaf venation appear at high resolutions, whereas larger structural patterns like road networks are visible at coarser scales. A multi‑scale 2‑D CNN architecture, such as a U‑Net or an Inception‑style module, can capture these varied patterns. By integrating features from multiple receptive fields, the network builds a hierarchical representation that aligns with the physical hierarchy of natural scenes.

Comparatively, a single‑scale CNN may miss either fine texture or global structure, leading to sub‑optimal classification. For instance, a single‑scale kernel of 3 × 3 may accurately capture leaf texture but fail to recognise the arrangement of adjacent crops in a field. Conversely, a 7 × 7 kernel captures broader patterns but blurs fine details. Multi‑scale designs circumvent this trade‑off, yielding superior performance across heterogeneous scenes.

4. Reducing Dimensionality While Preserving Discriminative Power

Hyperspectral data often undergo dimensionality reduction techniques such as Principal Component Analysis (PCA) or Linear Discriminant Analysis (LDA) before classification. These methods preserve the most variance or discriminative information but discard subtle spectral nuances that may be relevant for certain classes. Spatial CNNs offer an alternative: by learning spatial filters directly from the data, they can identify and preserve discriminative spectral patterns that are spatially coherent.

For example, in a water‑contamination study, the spectral signature of a polluted water body may differ only slightly from clean water. A PCA may collapse this difference into a low‑variance component. A 2‑D CNN, however, can learn that the polluted region is adjacent to a shoreline with a specific texture, enabling it to retain the subtle spectral difference in the presence of spatial context.

5. Computational Efficiency and Real‑Time Applications

While deep learning models are often criticised for their computational demands, 2‑D CNNs can be optimised for real‑time processing on modern GPUs or even embedded devices. Techniques such as depth‑wise separable convolutions reduce parameter count and FLOPs without sacrificing accuracy. This makes it feasible to deploy spatially aware classifiers in operational settings, such as drone‑based monitoring or satellite data pipelines where rapid turnaround is essential.

In contrast, traditional spectral classifiers may require extensive pre‑processing (e.g. endmember extraction, unmixing) that is computationally heavy and less amenable to real‑time execution. The end‑to‑end nature of 2‑D CNNs eliminates many intermediate steps, offering a streamlined workflow.

6. Comparative Performance Analysis

Empirical studies consistently show that incorporating spatial context via 2‑D CNNs improves classification accuracy. The following table summarises results from a recent benchmark on the Indian Pines hyperspectral dataset, comparing three approaches: a spectral‑only SVM, a spectral‑spatial SVM (using handcrafted texture features), and a 2‑D CNN.

These figures illustrate that the CNN’s ability to learn spatial features automatically outperforms handcrafted feature engineering, while also exceeding the spectral‑only baseline.

7. Limitations and Practical Considerations

Despite their advantages, 2‑D CNNs are not a panacea. They require large labelled datasets for effective training, which can be a bottleneck in remote sensing where ground truth is scarce. Transfer learning from related domains (e.g. natural images) can alleviate this, but careful fine‑tuning is necessary to avoid domain mismatch.

Furthermore, the spatial resolution of the input imagery limits the size of the receptive field. For very low‑resolution sensors, even a 7 × 7 kernel covers only a few metres, potentially missing larger spatial patterns. In such cases, hybrid approaches that combine 2‑D CNNs with 3‑D CNNs or graph‑based methods may offer better performance.

8. Future Directions

Recent research explores the integration of attention mechanisms within 2‑D CNNs, allowing the network to focus selectively on informative spatial regions. Similarly, combining 2‑D CNNs with spectral‑domain networks (e.g. 1‑D CNNs or transformers) yields joint spectral–spatial representations that further boost accuracy.

Another promising avenue is the use of self‑supervised learning to pre‑train 2‑D CNNs on large volumes of unlabelled hyperspectral data. Pre‑training on tasks such as context prediction or spectral reconstruction can imbue the network with generic spatial knowledge, reducing the need for extensive labelled datasets.

In sum, the incorporation of spatial features via 2‑D CNNs represents a significant leap forward for hyperspectral classification. By leveraging contextual information, mitigating spectral mixing, and enabling efficient real‑time inference, these models enhance both the robustness and accuracy of material identification in complex scenes.

Comparative Perspectives: 2‑D vs Other Spatial Methods

In the domain of visual pattern recognition, the spatial dimension of data is a core consideration. Two‑dimensional convolutional neural networks (2‑D CNNs) have become the de‑facto standard for tasks such as object detection, semantic segmentation, and image classification. However, they are not the only approach to extracting spatial structure. Traditional handcrafted filters (e.g. Sobel, Laplacian, Gabor) and three‑dimensional convolutional neural networks (3‑D CNNs) offer alternative mechanisms, each with distinct strengths and weaknesses. This section provides a focused comparison, illustrating when and why one might favour one method over another.

Traditional Hand‑crafted Filters

Before the deep learning revolution, image processing relied heavily on manually designed kernels that emphasise particular spatial patterns. The Sobel operator, for instance, approximates the first derivative, highlighting edges in a specific orientation. The Laplacian of Gaussian (LoG) responds to areas of rapid intensity change, useful for blob detection. Gabor filters, with oscillatory patterns modulated by a Gaussian envelope, resemble the receptive fields of simple cells in the visual cortex and are adept at capturing specific frequencies and orientations.

These techniques possess several notable attributes:

• Computational Simplicity: Convolution with a small fixed kernel is inexpensive and can be implemented efficiently on CPUs or even hardware like FPGAs.
• Interpretability: The mathematical form of the filter is explicit, allowing a clear understanding of what spatial features are being emphasised.
• Domain Knowledge Encapsulation: By choosing filter parameters (e.g. orientation, scale), one can embed expert knowledge directly into the processing pipeline.

However, these strengths come with limitations that become apparent as image complexity grows:

• lack adaptability: Hand‑crafted filters are fixed; they cannot learn from data to optimise for a specific task. Consequently, they are often sub‑optimal when the statistical properties of the dataset differ from the assumptions underlying the filter design.
• Limited Feature Hierarchy: Traditional filters typically produce a single or a small set of response maps. They cannot build deep hierarchies of increasingly abstract features without additional engineering.
• Sensitivity to Noise and Variations: Hand‑crafted filters can be brittle, with performance degrading under changes in illumination, scale, or viewpoint unless combined with elaborate pre‑processing steps.

2‑D Convolutional Neural Networks

In contrast, 2‑D CNNs learn a bank of filters directly from data. Each convolutional layer comprises many learnable kernels, each producing a feature map. Subsequent layers stack these feature maps, enabling the network to capture increasingly complex spatial patterns.

Key advantages of 2‑D CNNs include:

• Data‑Driven Feature Learning: The network automatically discovers the most discriminative patterns for the task, often outperforming handcrafted counterparts.
• Hierarchical Representation: Deeper layers capture abstract concepts (e.g. object parts, shapes) that are invariant to low‑level variations.
• End‑to‑End Training: The entire pipeline, from raw pixels to final predictions, can be optimised jointly, ensuring that feature extraction is tailored to the loss function.

Nevertheless, 2‑D CNNs exhibit challenges when confronted with data that possess an inherent third spatial dimension, such as volumetric scans or stacked image sequences.

3‑D Convolutional Neural Networks

3‑D CNNs extend the convolution operation into three dimensions, operating on volumes rather than planar images. The convolutional kernel slides over depth, height, and width, producing feature maps that encapsulate volumetric context.

Typical applications of 3‑D CNNs include:

• Medical Imaging: CT, MRI, and PET scans are naturally volumetric; 3‑D CNNs can capture spatial correlations across slices.
• Video Analysis: By treating time as the third dimension, 3‑D CNNs can learn spatio‑temporal patterns, useful for action recognition.
• Multichannel Data Fusion: When multiple modalities (e.g. RGB, depth, infrared) are aligned in a 3‑D grid, 3‑D convolutions can jointly process them.

Despite their expressive power, 3‑D CNNs entail substantial costs:

• Computational Expense: The number of parameters and floating‑point operations grows dramatically with kernel depth, leading to longer training times and higher memory usage.
• Data Requirements: Learning 3‑D filters effectively requires larger datasets to avoid overfitting, a constraint in domains where annotated volumetric data are scarce.
• Architectural Complexity: Designing efficient 3‑D architectures (e.g. balancing depth vs. width, choosing appropriate pooling strategies) is more challenging than 2‑D counterparts.

Comparative Analysis: Strengths and Weaknesses

“The choice between 2‑D and 3‑D convolutions is not a binary decision but a spectrum of trade‑offs that must be calibrated against data characteristics, computational resources, and the specific task at hand.” — Dr. A. Patel, CVPR 2023

Below we summarise key contrasts in a tabular format, drawing on concrete examples.

Concrete Example: Edge Detection vs. Scene Understanding

Consider the task of detecting the boundary of a car in a street scene. A Sobel filter can quickly produce an edge map highlighting high‑frequency changes. However, to distinguish the car from adjacent traffic lights or pedestrians, a 2‑D CNN trained on thousands of annotated images can learn higher‑level features such as colour, shape, and context, yielding a more precise segmentation mask. If the same scene were captured as a short video clip, a 3‑D CNN could utilise temporal continuity to reduce false positives: a moving car will exhibit consistent motion patterns across consecutive frames, whereas a static object will not.

Hybrid Strategies

In practice, developers often blend these approaches to maximise performance while managing resources:

• Pre‑processing with Hand‑crafted Filters: Applying a Sobel or Laplacian filter before feeding data to a 2‑D CNN can act as a feature enhancer, reducing the burden on the network to learn basic edge detectors.
• 2‑D CNNs with Temporal Aggregation: For video, one can process each frame with a 2‑D CNN and then aggregate features using recurrent units (LSTM) or temporal pooling, avoiding the full 3‑D convolution overhead.
• Sparse 3‑D Convolutions: In medical imaging, techniques such as voxel‑based subsampling or use of 3‑D atrous convolutions can reduce computational cost while retaining volumetric context.

Decision Guideline

1. Do you have large, labelled 3‑D datasets? Yes → Consider 3‑D CNNs; No → Prefer 2‑D CNNs with multi‑view aggregation.
2. Is computational budget tight (e.g. embedded devices)? Yes → Use traditional filters or lightweight 2‑D CNNs (e.g. MobileNet). No → Full‑scale 2‑D or 3‑D models are viable.
3. Do you require explicit interpretability of low‑level features? Yes → Hand‑crafted filters provide clear visualisations; 2‑D CNNs can be inspected via saliency maps. No → 3‑D CNNs can be used where interpretability is secondary to performance.
4. Is the spatial structure inherently 3‑D (e.g. depth, time)? Yes → 3‑D CNNs or hybrid 2‑D/3‑D pipelines. No → 2‑D CNNs suffice.

In sum, 2‑D CNNs strike a balance between learning capacity and computational feasibility for most 2‑D image tasks. Traditional filters remain valuable for quick, interpretable pre‑processing or when data are extremely limited. 3‑D CNNs excel when volumetric context is essential, albeit at a higher cost. By understanding these trade‑offs, practitioners can design spatial feature extraction pipelines that are both effective and efficient for their specific application domains.

Further Reading

• Goodfellow, I., Bengio, Y., & Courville, A. (2016) — Deep Learning. A foundational text covering CNNs and their application to spatial data. Provides theoretical background and practical examples relevant to hyperspectral image analysis.
• Shi, L., & Pan, H. (2020) — Convolutional Neural Networks for Hyperspectral Image Classification. IEEE Geoscience and Remote Sensing Magazine, 8(2), 72‑83. Discusses specialised CNN architectures adapted for high‑dimensional spectral data, including dimensionality reduction techniques.
• Jiang, Y., et al. (2018) — Deep Residual Learning for Hyperspectral Image Classification. Remote Sensing, 10(9), 1516. Demonstrates the effectiveness of residual connections in 2‑D CNNs for extracting spatial–spectral features in hyperspectral scenes.
• National Institute of Standards and Technology (NIST) (2022) — Hyperspectral Imaging System Design Guide. Official documentation outlining best practices for data acquisition, preprocessing, and CNN-based analysis pipelines.
• Li, B., et al. (2021) — A Survey on Deep Learning for Remote Sensing Image Classification. IEEE Geoscience and Remote Sensing Magazine, 9(1), 18‑33. Provides a comprehensive review of CNN variants used in remote sensing, with emphasis on spatial feature extraction.