Implementing adversarial training in ML pipelines

1) What “adversarial training” means in a real pipeline

Adversarial training hardens a model by training it on adversarial examples in addition to clean data. Instead of learning a decision boundary that works only for “normal” inputs, the model learns to be stable under small, worst-case perturbations (within a chosen budget ε).

In an ML pipeline, adversarial training is not just “add FGSM once.” It’s an engineering pattern that includes:

• Threat model definition (what attacker can do)

• Attack generation (FGSM/PGD, norm, ε, steps)

• Training strategy (mixing ratio, schedules, loss design)

• Robust evaluation (clean vs adversarial metrics)

• Deployment controls (monitoring, drift checks, regression tests)

2) Pipeline design: where adversarial training plugs in

A standard pipeline becomes:

1. Data ingestion → validation

2. Preprocessing → augmentation

3. Model training

4. Adversarial example generation (inside training loop)

5. Robustness evaluation (attack suite)

6. Model packaging + deployment

7. Post-deployment monitoring + periodic robustness re-tests

Key idea: adversarial examples must be generated on-the-fly (or at least refreshed regularly) to avoid overfitting to a fixed adversarial set.

3) Threat model checklist (must be defined before training)

Before writing code, define:

• Norm constraint: L∞L_\inftyL∞ or L2L_2L2

• Perturbation budget: ε (e.g., 0.1, 0.2, 0.3 for MNIST-like scaled inputs)

• Attack method: FGSM (fast) or PGD (stronger)

• Attack strength during training: steps, step size

• Evaluation attack strength: usually stronger than training attack

• Target: untargeted vs targeted

Rule of thumb: Train with PGD-5/10 (fast enough), evaluate with PGD-20/50 (stronger).

4) Implementation patterns (3 common ways)

Pattern A — Batch mixing (clean + adversarial)

For each batch:

• generate adversarial examples from clean inputs

• concatenate clean + adversarial

• train on combined batch

Pros: simple, effective baseline
Cons: doubles batch compute

Pattern B — Adversarial-only batches

Train only on adversarial examples.

Pros: pushes robustness strongly
Cons: can degrade clean accuracy noticeably

Pattern C — Curriculum / schedule

Start training on clean data, gradually increase adversarial ratio or ε.

Pros: more stable optimization
Cons: more tuning

5) Hands-on: PGD-based adversarial training (TensorFlow 2 + CleverHans)

This lab-style code shows:

• build & train a CNN

• implement adversarial training in the training loop

• evaluate robustness using a strong PGD attack

5.1 Setup and baseline training (MNIST)

import tensorflow as tf
import numpy as np

# Data
(x_train, y_train), (x_test, y_test) = tf.keras.datasets.mnist.load_data()
x_train = (x_train.astype("float32")/255.0)[..., None]
x_test  = (x_test.astype("float32")/255.0)[..., None]
y_train_oh = tf.one_hot(y_train, 10)
y_test_oh  = tf.one_hot(y_test, 10)

# Model (logits output)
model = tf.keras.Sequential([
    tf.keras.layers.Conv2D(32, 3, activation="relu", input_shape=(28,28,1)),
    tf.keras.layers.MaxPool2D(),
    tf.keras.layers.Conv2D(64, 3, activation="relu"),
    tf.keras.layers.MaxPool2D(),
    tf.keras.layers.Flatten(),
    tf.keras.layers.Dense(128, activation="relu"),
    tf.keras.layers.Dense(10)  # logits
])

loss_fn = tf.keras.losses.CategoricalCrossentropy(from_logits=True)
optimizer = tf.keras.optimizers.Adam(1e-3)

# Baseline training (clean only)
model.compile(optimizer=optimizer, loss=loss_fn,
              metrics=[tf.keras.metrics.CategoricalAccuracy(name="acc")])
model.fit(x_train, y_train_oh, epochs=2, batch_size=128, validation_split=0.1)

print("Clean test accuracy (baseline):", model.evaluate(x_test, y_test_oh, verbose=0)[1])

5.2 Adversarial example generator (PGD)

from cleverhans.tf2.attacks.projected_gradient_descent import projected_gradient_descent

def pgd_attack(model, x, eps=0.3, eps_iter=0.03, nb_iter=10):
    # L-infinity PGD
    return projected_gradient_descent(
        model_fn=model,
        x=x,
        eps=eps,
        eps_iter=eps_iter,
        nb_iter=nb_iter,
        norm=np.inf,
        targeted=False
    )

5.3 Adversarial training loop (clean + adversarial mixing)

train_ds = tf.data.Dataset.from_tensor_slices((x_train, y_train_oh))\
                          .shuffle(10000).batch(128)

@tf.function
def adv_train_step(x, y, eps=0.3, eps_iter=0.03, nb_iter=5, adv_ratio=0.5):
    """
    adv_ratio=0.5 means 50% adversarial + 50% clean per batch.
    """
    batch_size = tf.shape(x)[0]
    adv_count = tf.cast(tf.round(tf.cast(batch_size, tf.float32) * adv_ratio), tf.int32)

    # Split batch into a clean part and a part to adversarially perturb
    x_adv_src = x[:adv_count]
    y_adv_src = y[:adv_count]
    x_clean   = x[adv_count:]
    y_clean   = y[adv_count:]

    # Generate adversarial examples for part of the batch
    x_adv = pgd_attack(model, x_adv_src, eps=eps, eps_iter=eps_iter, nb_iter=nb_iter)

    # Mix clean + adversarial
    x_mix = tf.concat([x_adv, x_clean], axis=0)
    y_mix = tf.concat([y_adv_src, y_clean], axis=0)

    with tf.GradientTape() as tape:
        logits = model(x_mix, training=True)
        loss = loss_fn(y_mix, logits)

    grads = tape.gradient(loss, model.trainable_variables)
    optimizer.apply_gradients(zip(grads, model.trainable_variables))
    return loss

5.4 Run adversarial training

for epoch in range(2):
    losses = []
    for xb, yb in train_ds:
        l = adv_train_step(xb, yb, eps=0.3, eps_iter=0.03, nb_iter=5, adv_ratio=0.5)
        losses.append(float(l))
    print(f"Adv-training epoch {epoch+1} | loss={np.mean(losses):.4f}")

print("Clean test accuracy (after adv training):", model.evaluate(x_test, y_test_oh, verbose=0)[1])

5.5 Robust evaluation (stronger PGD than training)

# Evaluate against a stronger PGD attack
x_adv_test = pgd_attack(model, x_test, eps=0.3, eps_iter=0.03, nb_iter=20)
adv_acc = model.evaluate(x_adv_test, y_test_oh, verbose=0)[1]
print("PGD-20 adversarial accuracy:", adv_acc)

What learners should notice:

• Clean accuracy may drop slightly

• Adversarial accuracy under PGD should improve compared to the baseline model

6) Practical best practices (production mindset)

A) Train vs Eval attack strength

• Train: PGD-5/10 (fast)

• Eval: PGD-20/50 + at least one alternative attack family

B) Track the right metrics

• Clean accuracy

• Robust accuracy (under chosen attacks)

• Attack Success Rate (ASR)

• Per-class robustness (some classes break first)

• Calibration (confidence leakage matters for safety)

C) Make preprocessing consistent

If training uses normalization/augmentations, inference must match. Inconsistency can create “ghost vulnerabilities.”

D) Version everything

Store:

• ε, steps, step size, norm

• code version + framework versions

• dataset version + preprocessing hash

• robustness report artifacts

7) Common failure modes to teach explicitly

1. Training with FGSM only, evaluating with FGSM only
   → false security. Always evaluate with stronger attacks.

2. Overfitting to a single adversary
   → robust to PGD-10 but fails elsewhere. Use a small suite.

3. Compute blow-up
   → PGD training is expensive. Use fewer steps, mixed batches, or curriculum schedules.

4. Robustness-accuracy trade-off shock
   → stakeholders panic when clean accuracy drops slightly. Set acceptable thresholds upfront.