Common attack vectors in ML systems

1) What “attack vectors” mean in ML systems

An attack vector is the practical path an adversary uses to compromise an ML system—by manipulating inputs, training data, outputs, infrastructure, or feedback loops.

In ML, attack vectors span three layers:

1. ML layer: evasion, poisoning, backdoors, extraction, privacy attacks
2. Data layer: pipeline tampering, label corruption, feature store injection
3. Software/infra layer: API abuse, CI/CD compromise, misconfigurations, DoS

2) Attack vectors by lifecycle stage (most useful structure)

A) Training-time attack vectors

1) Data poisoning

Attacker injects or modifies training samples to degrade performance or bias outcomes.

• Untargeted poisoning: reduces overall accuracy
• Targeted poisoning: breaks a specific class or subgroup
• Real-world example: fake user reviews, fraudulent transactions, manipulated sensor logs

Mitigations: secure ingestion, dataset versioning, anomaly/outlier detection, provenance, robust training.

2) Label manipulation (label flipping / label noise injection)

Attacker corrupts labels (manually or via compromised labeling tools).

• stealthy if done on ambiguous samples or rare classes

Mitigations: gold set auditing, multi-annotator agreement, label versioning + audit logs, reviewer escalation.

3) Backdoor (Trojan) attacks

Attacker introduces a “trigger” pattern during training so the model behaves normally unless the trigger appears.

• Image: sticker/patch → forced class
• Text: rare token/phrase → forced sentiment
• Tabular: feature combination → forced approval

Mitigations: data sanitization, trigger detection (clustering/spectral methods), strict source control, retrain from clean snapshots.

4) Supply-chain attacks in training

Attacker compromises:

• datasets, dependency packages, pretrained checkpoints
• training scripts or CI/CD artifacts

Mitigations: signed artifacts, dependency pinning + scanning, reproducible builds, least-privilege CI/CD, model registry integrity checks.

B) Inference-time attack vectors

5) Evasion attacks (adversarial examples)

Attacker crafts inputs that look normal to humans but cause misclassification.

• White-box: attacker knows model, uses gradients
• Black-box: attacker queries API and searches input space

Mitigations: adversarial training, robust preprocessing (carefully evaluated), monitoring + throttling, input validation, fallback policies.

6) Adversarial patches & physical-world attacks

Attacker uses a localized patch/sticker or real-world manipulation (lighting, angles) to fool perception systems.

Mitigations: patch training, detection, sensor fusion, physical robustness testing, safety constraints downstream.

7) Prompt injection (LLM-based systems)

Attacker embeds instructions in user input (or retrieved content) to override system rules.

Common outcomes:

• data exfiltration from tools/connectors
• unsafe tool execution
• policy bypass in RAG systems

Mitigations: strict tool permissioning, input/output filtering, prompt isolation, least privilege tool scopes, audit logs, “deny by default” tool calls.

8) Model extraction / model stealing

Attacker repeatedly queries the model to approximate it (build a surrogate), then:

• steals IP
• improves evasion attacks
• clones your service cheaper

Mitigations: rate limits, output shaping (avoid logits), watermarking (where applicable), anomaly detection for probing behavior.

9) Privacy attacks (membership inference & model inversion)

Attacker uses outputs to infer:

• whether a record was in training data (membership)
• sensitive attributes or approximate training examples (inversion)

Mitigations: output restriction, differential privacy (training), regularization, access control, auditing, reduce confidence granularity.

C) Post-deployment attack vectors (the “long game”)

10) Feedback loop poisoning

Attacker manipulates feedback signals used for retraining:

• user reports, clicks, human review labels, outcomes

Mitigations: quarantine new feedback, trust scoring, gold tasks, delayed training ingestion, anomaly detection on feedback patterns.

11) Data drift exploitation

Attacker rides natural drift (or induces drift) so model behavior becomes unsafe—without needing explicit adversarial perturbations.

Mitigations: drift monitoring (PSI/KL), slice-level monitoring, retraining governance, canary deployments.

12) Monitoring evasion

Attacker stays below alert thresholds:

• low-rate probing
• slow poisoning
• distributed requests across many identities

Mitigations: behavioral analytics, long-window anomaly detection, correlation across identities, rate limits by organization/ASN, incident playbooks.

3) Infrastructure and API attack vectors (cross-cutting)

These are not ML-specific but frequently used against ML systems:

• DoS / resource exhaustion: large payloads, expensive queries, GPU starvation
• Auth bypass / key theft: stolen API keys, weak IAM
• Misconfiguration: public buckets, open model registries
• Unsafe deserialization: loading untrusted model artifacts
• Log injection / PII leakage: malicious payloads in logs

Mitigations: WAF, strict request caps, secrets management, signed artifacts, least privilege, secure logging.

4) Practical mapping: vector → signal → mitigation