Identifying attack surfaces in deployment stages

1) What “attack surface” means for deployed ML

In deployment, the attack surface is every place an adversary can influence:

• inputs (what the model sees)
• outputs (what the system reveals)
• model artifacts (weights, configs, prompts, feature pipelines)
• infrastructure (APIs, storage, CI/CD, monitoring)
• feedback loops (logging, human review, retraining signals)

Unlike offline training, deployment is “live”—attackers can probe your system continuously, learn its behavior, and adapt.

2) Deployment stages and their typical attack surfaces

A) Client / edge layer (user device, sensor, browser, app)

Attack surfaces

• input manipulation (camera spoofing, sensor noise injection, adversarial patches)
• replay attacks (reusing old legitimate inputs)
• client-side tampering (modified app, rooted device)

Controls

• device attestation (where possible)
• input signing / secure channels
• replay protection (nonces, timestamps)
• edge anomaly detection (sensor sanity checks)

B) Data ingestion and request handling (API gateway → inference service)

Attack surfaces

• malformed inputs to crash parsers (DoS)
• payload flooding (rate-based DoS)
• adversarial example probing (query-based black-box attacks)
• prompt injection (for LLM-based systems)
• schema abuse (wrong types, extreme lengths, unexpected encodings)

Controls

• strict schema validation + size limits
• WAF / API gateway protections
• rate limiting and quotas per key/user/IP
• request authentication + authorization
• structured logging + anomaly alerts

C) Feature transformation layer (online feature store, preprocessing)

Attack surfaces

• feature skew (training vs inference mismatch)
• feature poisoning (compromised feature store updates)
• time-of-check/time-of-use bugs (stale features, wrong joins)
• data leakage through features (sensitive attributes)

Controls

• versioned feature definitions
• online/offline parity tests
• write protection on feature store (least privilege)
• drift monitoring + integrity checksums for critical features

D) Model serving runtime (container, model server, GPU/CPU runtime)

Attack surfaces

• model extraction (stealing model behavior via queries)
• side-channel leakage (timing, error messages)
• dependency vulnerabilities (model server CVEs)
• container escape / misconfigurations
• unsafe deserialization (loading malicious model files)

Controls

• hardened container images, minimal base images
• restrict filesystem/network permissions
• secure model loading (signed artifacts)
• patch management + vulnerability scanning
• suppress verbose errors in production

E) Output layer (predictions, scores, explanations, APIs)

Attack surfaces

• membership inference (training data presence leakage via confidence)
• model inversion (reconstructing sensitive patterns)
• over-informative explanations (leaking features/logic)
• response shaping attacks (manipulate downstream decision thresholds)

Controls

• limit output granularity (top-1 label vs full probabilities where possible)
• add noise / calibration controls where appropriate
• access control for explanations (not public by default)
• consistent, non-revealing error handling

F) Monitoring, logging, and analytics (telemetry pipeline)

Attack surfaces

• poisoning feedback loops (attackers generate “ground truth” feedback)
• log injection (malicious strings in logs, dashboards)
• PII leakage via logs
• alert fatigue (attackers flood alerts to hide real signals)

Controls

• sanitize logs, escape untrusted strings
• separate security logs from product analytics
• PII redaction, tokenization
• anomaly detection on feedback channels
• alert rate-limiting and severity routing

G) Human-in-the-loop and moderation queues

Attack surfaces

• social engineering (tricking reviewers)
• label manipulation through coordinated reporting
• queue flooding to delay genuine review

Controls

• reviewer training + playbooks
• sampling + audit (gold tasks)
• queue throttling and trust scoring
• separation of duties (review vs approve vs deploy)

H) Retraining and CI/CD (MLOps pipeline)

Attack surfaces

• training data poisoning via feedback loops
• compromised build pipeline / model artifact replacement
• dependency supply-chain attacks
• rollback sabotage (preventing clean recovery)

Controls

• signed model artifacts + provenance
• reproducible training runs
• approval gates for dataset/model promotion
• automated robustness regression tests
• secure secrets management + least privilege

3) How to systematically identify attack surfaces (a practical method)

Step 1 — Draw your deployment dataflow diagram

Include:

• clients
• gateways
• feature store
• model server
• downstream decision systems
• logs/monitoring
• retraining pipelines

Step 2 — For each node, list “attacker touchpoints”

For each component ask:

• What inputs can be controlled by users?
• What outputs are visible externally?
• What secrets or artifacts are stored here?
• What dependencies or plugins execute code?

Step 3 — Map threats using STRIDE (quick ML adaptation)

• Spoofing: fake users/devices, replay
• Tampering: input/feature/model tampering
• Repudiation: missing logs/audit trails
• Information disclosure: confidence leakage, inversion
• Denial of service: floods, parser crashes, GPU exhaustion
• Elevation of privilege: container escapes, IAM over-permission

Step 4 — Prioritize by risk

Use a simple score:

• Impact (safety/financial/privacy)
• Likelihood (exposure + attacker skill)
• Detectability (how quickly you’d notice)

4) “High-risk” deployment attack surfaces (most common in real systems)

1. Public inference APIs (black-box probing, extraction, DoS)
2. Overly detailed outputs (probabilities, explanations → privacy leakage)
3. Feature store write paths (poisoning)
4. Training feedback loops (data poisoning through user feedback)
5. Preprocessing mismatches (skew exploited to break predictions)
6. Weak monitoring (attacks persist undetected)