Challenges in federated learning models

1) Why federated learning (FL) changes the threat landscape

Federated Learning trains a global model across many clients (phones, hospitals, edge devices) without centralizing raw data. This helps privacy, but it also introduces new security challenges:

• training is distributed (many participants, varying trust)
• updates come from devices you may not fully control
• attackers can hide inside the crowd
• monitoring is harder because raw data is not visible centrally

In FL, the question isn’t only “Can the model be attacked?” but also:

“Can the training process be manipulated through malicious clients or updates?”

2) Where FL is vulnerable (pipeline view)

A simplified FL pipeline:

1. Server sends global model wtw_twt to selected clients
2. Clients train locally on their data → produce update Δwi\Delta w_iΔwi
3. Server aggregates updates (FedAvg, etc.) → new global model wt+1w_{t+1}wt+1
4. Repeat across rounds

Attack surfaces:

• client devices (can be compromised)
• client updates (can be poisoned)
• aggregation mechanism (can be manipulated)
• communication channel (eavesdropping, tampering)
• model outputs (privacy leakage in updates)

3) Core adversarial challenges in federated learning

A) Non-IID data makes security harder

FL clients often have data that is:

• highly skewed (non-IID)
• different in quality
• different in volume

Why this is a security problem:
Robust aggregation assumes “outliers are malicious,” but in FL, outliers can simply be honest clients with unusual data. This creates a hard problem:

• Too strict filtering → you reject good clients
• Too loose filtering → poisoned updates slip through

B) Poisoning via malicious client updates

Attackers can participate as clients (or compromise clients) and send poisoned updates:

1) Untargeted poisoning

Goal: degrade global model performance.

2) Targeted poisoning

Goal: make the global model fail on a specific class or subgroup.

3) Backdoor attacks (very common in FL)

Goal: make model behave normally except for a trigger.

• Example: when a specific pattern appears, model outputs attacker-chosen label.

Why backdoors are effective in FL:
A single malicious client can scale its update (“model replacement”) to dominate aggregation in one round.

C) Model replacement and update scaling

Some attacks work by amplifying the malicious update magnitude so it overwhelms the average.

Challenge: The server often trusts clients’ updates unless:

• update norm constraints exist
• robust aggregation is used
• anomaly detection is applied

D) Sybil attacks (fake clients)

Attacker creates many fake clients (Sybil identities) to gain majority influence in aggregation.

Why it’s dangerous:
Even robust aggregation can fail if a large fraction of participants are malicious.

Challenge: In real FL settings, client identity can be hard to verify (especially at scale).

E) Privacy leakage from gradients/updates

Even if raw data isn’t shared, updates can leak information:

• membership inference (“Was this patient/device data used?”)
• property inference (“Does this client have data with attribute X?”)
• reconstruction/inversion (recovering input patterns from gradients)

FL reality:
Privacy is not guaranteed just because data stays local.

F) Secure aggregation vs security monitoring trade-off

Secure aggregation encrypts updates so the server can only see the aggregate. Great for privacy—but it reduces visibility.

Trade-off:

• More privacy → harder to detect malicious clients
• More monitoring → more exposure of client updates

This creates a design tension between privacy and security.

G) Client heterogeneity and unreliable devices

Clients differ in:

• compute power
• connectivity
• OS versions and security posture

This increases:

• susceptibility to compromise
• inconsistency in training behavior
• risk of partial participation (which attackers can exploit)

H) Evaluation and accountability are harder

In centralized ML:

• you can inspect data
• reproduce training conditions
• run forensic analysis on logs

In FL:

• you often cannot inspect client data
• “who caused the drift” is harder to prove
• poisoning can be gradual and distributed

4) Practical mitigation directions (high-level overview)

A) Robust aggregation

Methods that reduce influence of abnormal updates:

• trimmed mean / median-based aggregation
• norm clipping + aggregation
• outlier detection on update statistics

B) Client attestation and identity hardening

• device attestation where possible
• enforce unique client identity and prevent Sybils
• reputation scoring (careful: can bias against rare data)

C) Backdoor detection strategies

• monitor global model behavior on trigger probes (synthetic tests)
• inspect representation anomalies
• periodic centralized red-team evaluations on held-out sets

D) Differential privacy (DP) and secure aggregation

• DP reduces leakage from updates
• secure aggregation reduces server-side visibility of individual updates
  But both can reduce model utility and complicate attack detection.

E) Controlled training participation

• limit number of clients per round
• random sampling and diversity constraints
• quarantine suspicious clients/updates