Lesson 4.2.3: Statistical features for disease classification

Definition of Statistical Features in Spectral Analysis

In the context of disease classification, spectral analysis of biomedical signals—such as electroencephalography (EEG), magnetic resonance spectroscopy (MRS), or mass spectrometry—provides a high‑dimensional representation of underlying physiological processes. However, raw spectra are often noisy, redundant, and difficult to interpret directly. Statistical features summarise key aspects of spectral content, reducing dimensionality while preserving discriminative information. This section introduces the most commonly employed statistical descriptors, discusses their mathematical foundations, and compares alternative approaches through concrete examples.

1. Mean Power and Spectral Centroid

The simplest descriptor is the arithmetic mean of power spectral density (PSD) values across a frequency band of interest. For a discrete spectrum P(f_i) sampled at frequencies f_i, i = 1…N, the mean power is

mean_power = sum(P) / N

This yields an overall energy measure. A more refined variant is the spectral centroid, defined as the frequency‑weighted mean:

centroid = sum(f_i * P(f_i)) / sum(P(f_i))

Centroid captures the “centre of mass” of the spectrum and is especially informative in distinguishing between diseases that shift spectral power towards lower or higher frequencies. For instance, in Parkinson’s disease, the centroid of the EEG alpha band often shifts downward relative to healthy controls.

2. Variance, Standard Deviation, and Spread

While mean values describe central tendency, variance quantifies dispersion:

variance = sum((P - mean_power)**2) / N
std_dev = sqrt(variance)

A higher variance indicates a more heterogeneous spectral profile. In multiple sclerosis, studies have reported increased PSD variance in the beta band, reflecting irregular neural conduction.

3. Skewness and Kurtosis

These higher‑order moments assess asymmetry and peakedness of the spectral distribution. Skewness is defined as the third central moment normalised by the cube of the standard deviation:

skew = sum((P - mean_power)**3) / (N * std_dev**3)

Kurtosis measures the fourth central moment normalised by the fourth power of the standard deviation:

kurtosis = sum((P - mean_power)**4) / (N * std_dev**4)

Skewness can reveal whether spectral power is biased toward lower or higher frequencies, while kurtosis indicates the presence of sharp peaks. In epilepsy, the PSD of interictal EEG often exhibits a pronounced right‑skew, reflecting occasional high‑frequency bursts.

4. Spectral Entropy

Entropy quantifies the unpredictability or complexity of a spectrum. Normalising the PSD to obtain a probability distribution p_i = P(f_i) / sum(P), the Shannon entropy is

entropy = -sum(p * log2(p))

Higher entropy signals a flatter spectrum with less dominant frequency components. In Alzheimer’s disease, reduced entropy in the theta band has been linked to impaired neural synchronisation.

5. Peak‑to‑Valley Ratio (PVR)

PVR captures the prominence of spectral peaks relative to the surrounding troughs:

peak = max(P)
valley = min(P)
pvr = peak / valley

A high PVR indicates a strongly dominant frequency band. For example, the 6‑Hz peak in the basal ganglia spectrum of Parkinsonian patients yields a markedly higher PVR than in healthy subjects.

6. Band‑Specific Energy Ratios

Diseases often affect specific frequency bands. Ratios such as the alpha‑to‑beta energy ratio provide a simple yet powerful discriminator:

alpha_energy = sum(P_alpha)
beta_energy = sum(P_beta)
ratio = alpha_energy / beta_energy

In schizophrenia, studies report an elevated gamma‑to‑theta ratio, reflecting aberrant high‑frequency activity.

7. Correlation Coefficients Across Bands

Cross‑band relationships can be summarised by computing Pearson’s correlation between power in two bands across subjects or time windows:

corr = corrcoef(P_band1, P_band2)[0,1]

In autism spectrum disorder, a negative correlation between delta and gamma power has been observed, suggesting compensatory mechanisms.

8. Alternative Approaches: Non‑Linear and Time‑Frequency Features

While the above features are strictly statistical, alternative descriptors capture different aspects of spectral data:

• Wavelet‑based moments: Decompose signals into time‑frequency atoms, then compute statistics on wavelet coefficients. This preserves temporal localisation.
• Hjorth parameters (activity, mobility, complexity) summarise the instantaneous spectral slope and curvature.
• Fractal dimensions estimate self‑similarity, useful in characterising pathological oscillations.

Comparatively, purely statistical features are computationally inexpensive and interpretable, making them attractive for real‑time clinical applications. Non‑linear descriptors often provide complementary information but may introduce additional complexity and require larger datasets to avoid overfitting.

9. Practical Example: Classifying Parkinson’s vs. Healthy Controls

Consider an EEG dataset sampled at 256 Hz, filtered to isolate the 4–30 Hz band. For each recording, we extract the following features:

1. Mean power in the alpha band (8–12 Hz)
2. Spectral centroid in the 4–30 Hz band
3. Standard deviation of the PSD across the entire band
4. Spectral entropy (normalised to the number of frequency bins)
5. Alpha‑to‑beta energy ratio (8–12 Hz vs. 13–30 Hz)

A logistic regression model trained on these five features achieved an accuracy of 87 % in distinguishing Parkinsonian patients from healthy controls, with the spectral centroid and alpha‑to‑beta ratio contributing the most to the decision boundary.

10. Summary of Key Takeaways

Statistical features provide a concise, interpretable summary of spectral data, essential for disease classification pipelines. Mean, variance, skewness, kurtosis, entropy, and band‑specific ratios each capture distinct facets of spectral organisation. When combined, they often yield robust classifiers, though care must be taken to avoid redundancy and over‑fitting. Alternative non‑linear descriptors can supplement these features, particularly when temporal dynamics or self‑similarity play a crucial role in the pathology under investigation.

Mechanisms of Feature Extraction from Spectral Data

In the context of disease classification, raw spectra are typically high‑dimensional, noisy and rife with redundant information. Feature extraction transforms these spectra into a compact representation that preserves the discriminative structure necessary for subsequent machine learning. This section outlines the principal computational steps and algorithms used to derive statistical features, compares alternative approaches, and illustrates each with concrete numerical examples.

Pre‑processing: Normalisation and Baseline Correction

Before any statistical descriptor can be computed, the spectra must be stabilised. Two common pre‑processing stages are normalisation and baseline correction.

1. Vector Normalisation: Each spectrum \(x = (x_1, x_2, \dots, x_n)\) is divided by its Euclidean norm \[ x' = \frac{x}{\|x\|_2}, \] ensuring that variances arise from shape rather than intensity.
2. Baseline Correction: Low‑frequency drift is removed using polynomial fitting or asymmetric least squares. For example, a second‑order polynomial \(p(\lambda) = a\lambda^2 + b\lambda + c\) is fitted to the baseline and subtracted, yielding the corrected spectrum \(x'' = x' - p\).

Both steps reduce inter‑sample variability that is unrelated to pathology.

Statistical Descriptors

Statistical features quantify global and local characteristics of the spectrum. They are broadly divided into first‑order (univariate) and higher‑order (multivariate) statistics.

First‑Order Features

These are computed on the intensity values alone, ignoring any correlation structure.

For instance, a spectrum with a prominent absorption band at 1650 cm⁻¹ will exhibit a higher mean and RMS compared to a flat spectrum, while a sharp peak will increase the kurtosis.

Higher‑Order Features

These capture relationships between different spectral points, often revealing subtle pattern differences.

Peak‑Based Features

Identify local maxima and compute attributes such as peak height, width, area and position.

import scipy.signal as sig
peaks, _ = sig.find_peaks(x, height=0.5)
peak_heights = x[peaks]
peak_widths = sig.peak_widths(x, peaks, rel_height=0.5)[0]

In a Raman spectrum of malignant tissue, the 1585 cm⁻¹ band may be broader and higher than in healthy tissue, providing a discriminative cue.

Correlation‑Based Features

The covariance matrix \(\Sigma\) of the spectrum (or of a selected subset of wavelengths) captures inter‑wavelength relationships.

• Eigenvalues of \(\Sigma\) quantify variance along principal components.
• The trace \(\operatorname{tr}(\Sigma)\) equals the total variance.
• Off‑diagonal elements indicate coupling between wavenumbers.

For example, a covariance between 1450 cm⁻¹ and 1650 cm⁻¹ that is higher in diseased samples suggests a pathological shift in protein conformation.

Entropy Measures

Shannon entropy, \[ H = -\sum_{i=1}^{n} p_i \log p_i, \] where \(p_i = x_i / \sum_j x_j\), estimates spectral disorder. A more heterogeneous spectrum yields higher entropy, often seen in tumours.

Dimensionality‑Reduction‑Based Features

Instead of hand‑crafted statistics, one can derive features through linear or non‑linear dimensionality reduction.

Principal Component Analysis (PCA)

PCA projects the spectrum onto orthogonal axes of maximal variance. The first few principal component (PC) scores serve as features.

from sklearn.decomposition import PCA
pca = PCA(n_components=3)
pc_scores = pca.fit_transform(x.reshape(1, -1))

In practice, the first PC often captures the overall intensity trend, while the second may capture subtle band shifts.

Linear Discriminant Analysis (LDA)

LDA maximises class separability by projecting onto directions that maximise between‑class variance relative to within‑class variance. The resulting discriminant scores are highly informative for classification.

from sklearn.discriminant_analysis import LinearDiscriminantAnalysis
lda = LinearDiscriminantAnalysis()
lda_scores = lda.fit_transform(x.reshape(1, -1), label)

Because LDA is supervised, it may produce more compact features than PCA for disease classification.

Autoencoders

Neural network autoencoders learn non‑linear embeddings. The bottleneck layer provides dense features that capture complex spectral patterns.

import tensorflow as tf
input_dim = x.shape[0]
encoding_dim = 10
input_vec = tf.keras.Input(shape=(input_dim,))
encoded = tf.keras.layers.Dense(encoding_dim, activation='relu')(input_vec)
decoded = tf.keras.layers.Dense(input_dim, activation='sigmoid')(encoded)
autoencoder = tf.keras.Model(input_vec, decoded)
encoder = tf.keras.Model(input_vec, encoded)
autoencoder.compile(optimizer='adam', loss='mse')
autoencoder.fit(x.reshape(1, -1), x.reshape(1, -1), epochs=100, verbose=0)
latent_features = encoder.predict(x.reshape(1, -1))

Autoencoders can capture non‑linear interactions that linear methods miss, albeit at the cost of increased computational complexity.

Feature Selection and Evaluation

High‑dimensional feature sets risk over‑fitting. Two common strategies are:

• Filter Methods: Compute a univariate relevance score (e.g., mutual information) for each feature and retain the top‑k. For example, mutual information between a peak height and class label can rank its importance.
• Wrapper Methods: Use a classifier (e.g., Random Forest) to evaluate feature subsets via cross‑validation. Recursive Feature Elimination (RFE) systematically removes the least important features.

Both approaches have trade‑offs: filter methods are fast but ignore feature interactions; wrappers are accurate but computationally intensive.

Comparative Analysis of Feature Types

Below is a concise comparison of the feature families discussed.

Concrete Example: Differentiating Benign vs. Malignant Breast Tissue via FT‑IR

Suppose we have FT‑IR spectra from 100 samples (50 benign, 50 malignant) measured between 4000–400 cm⁻¹ at 4 cm⁻¹ resolution. The workflow proceeds as follows:

1. Pre‑processing: Vector normalise each spectrum and apply asymmetric least squares baseline correction.
2. Feature extraction:
   • Compute mean, RMS, skewness, kurtosis.
   • Detect peaks at 1650 cm⁻¹ (amide I) and 1540 cm⁻¹ (amide II); record height, width, area.
   • Calculate the covariance between 1450 cm⁻¹ and 1650 cm⁻¹.
   • Derive the first three PCA scores.
3. Feature selection: Use mutual information to rank features; retain the top 12.
4. Classification: Train a Support Vector Machine with a radial basis function kernel, tuned via grid search.

Cross‑validation yields an accuracy of 92 %, sensitivity of 90 % and specificity of 94 %, demonstrating that a modest set of statistically grounded features can achieve clinically relevant performance.

Conclusion

Deriving statistical features from raw spectra is a multi‑step, algorithmic process that balances computational feasibility, interpretability and discriminative power. By carefully selecting and combining first‑order, peak‑based, correlation‑based and dimensionality‑reduction techniques, practitioners can construct robust feature sets that enable accurate disease classification while retaining biological insight.

Implications for Disease Classification Performance

In the context of disease classification, the statistical features we extract from raw clinical data are the raw material upon which machine‑learning models build their predictive power. The choice of features is therefore not a purely technical decision; it carries direct consequences for model accuracy, robustness to noise and variation, and ultimately for the clinical utility of the system. The following discussion unpacks these implications, compares common feature families, and illustrates with concrete examples drawn from cardiovascular, oncological and infectious disease studies.

1. Accuracy Enhancement Through Feature Informativeness

Accuracy is the most immediately measurable metric, but it is highly contingent on the signal‑to‑noise ratio that the chosen features provide. Features that capture distinct pathophysiological signatures will enable a classifier to separate classes with higher fidelity. Consider the case of myocardial infarction (MI) detection from electrocardiogram (ECG) signals. A feature set comprising the RR interval, QRS complex duration, and ST‑segment elevation can achieve >95 % sensitivity and specificity in a random forest model. In contrast, a feature set limited to heart rate alone falls to <80 % sensitivity, even if the same algorithm is used.

2. Robustness to Variability and Noise

Robustness refers to a model’s ability to maintain performance when confronted with data that differ from the training distribution.

• Normalised features – Scaling features to zero mean and unit variance reduces sensitivity to absolute measurement units. In a lung‑cancer CT classification task, normalising radiomic texture measures by the mean intensity of the tumour mask reduced performance variability across scanners.
• Aggregated statistics – Summarising a time series by mean and standard deviation rather than raw points reduces the impact of transient artefacts. For heart‑sound analysis, using the mean heart‑sound amplitude over a 30‑second window preserved 92 % of the classification accuracy that a 5‑second window achieved, while eliminating outliers caused by patient movement.
• Robust distance metrics – Employing median absolute deviation (MAD) instead of standard deviation for spread estimation increases resilience to outliers. In sepsis prediction from bedside monitors, a model that used MAD of lactate levels retained 4 % higher AUC when tested on a noisy ICU dataset.

3. Clinical Relevance and Interpretability

Beyond statistical performance, clinicians require that features be interpretable and actionable. A model that relies on opaque, high‑dimensional embeddings may achieve high accuracy but face resistance in clinical deployment.

Tabular features derived from routine laboratory tests (e.g. haemoglobin, C‑reactive protein) are easily understood by clinicians. In a study of pneumonia severity scoring, the inclusion of CRP as a feature not only improved the AUC from 0.82 to 0.87 but also allowed clinicians to trust the model’s recommendations, as evidenced by a 35 % reduction in unnecessary antibiotic prescriptions.

4. Feature Redundancy and Dimensionality Reduction

High dimensionality can lead to over‑fitting, especially with limited sample sizes. Redundant features—those that convey the same information—do not improve accuracy and may even degrade robustness.

from sklearn.decomposition import PCA
pca = PCA(n_components=0.95)  # retain 95% variance
X_reduced = pca.fit_transform(X)

Principal Component Analysis (PCA) is a common technique to compress correlated features into orthogonal components. In a breast cancer prognosis study, reducing 120 radiomic features to 15 principal components maintained AUC at 0.89 while reducing training time by 70 %. However, the loss of interpretability can be significant, so PCA is often used in conjunction with other feature selection methods that preserve clinically meaningful variables.

5. Comparative Analysis of Feature Families

6. Integration into Clinical Decision Support Systems

When a model is to be embedded within an electronic health record (EHR), the computational cost of feature extraction matters. Features that require minimal pre‑processing—such as mean blood pressure or total cholesterol—are favoured for real‑time decision support. In contrast, features that entail extensive image processing may be reserved for offline batch analysis or specialist consultation.

7. Future Directions and Emerging Trends

Recent advances in transfer learning for medical imaging allow pre‑trained convolutional neural networks to extract features that capture subtle disease patterns while requiring fewer labelled examples. These learned features often outperform hand‑crafted radiomic descriptors in terms of accuracy but pose new challenges for interpretability.

Another trend is the utilisation of longitudinal feature trajectories. Rather than summarising a single snapshot, models can incorporate the slope and variability of biomarkers over time, which has been shown to improve prognostication in chronic kidney disease by 5 % AUC.

Finally, the incorporation of multi‑modal features—combining imaging, genomics, and clinical data—has the potential to capture complementary disease dimensions. Early studies in breast cancer risk stratification report up to a 12 % increase in predictive accuracy when genomic risk scores are combined with radiomic texture features, although the added complexity necessitates careful feature selection to avoid over‑fitting.

In summary, the statistical features