The Power of Bias: Optimizing Client Selection in Federated Learning With Heterogeneous Differential Privacy

IEEE Transactions on Dependable and Secure Computing（TDSC）

Jiating Ma¹,  Yipeng Zhou²,  Qi Li³,  Quan Z. Sheng²,  Laizhong Cui^1*, Jiangchuan Liu⁴

¹Shenzhen University

²Macquarie University

³Tsinghua University

⁴Simon Fraser University

Abstract

To preserve the data privacy, the federated learning (FL) paradigm emerges in which clients only expose model gradients rather than original data for conducting model training. To enhance the protection of model gradients in FL, differentially private federated learning (DPFL) is proposed which incorporates differentially private (DP) noises to obfuscate gradients before they are exposed. Yet, an essential but largely overlooked problem in DPFL is the heterogeneity of clients’privacy requirement, which can vary significantly between clients and extremely complicates the client selection problem in DPFL. In other words, both the data quality and the influence of DP noises should be taken into account when selecting clients. To address this problem, we conduct convergence analysis of DPFL under heterogeneous privacy, a generic client selection strategy, popular DP mechanisms and convex loss. Based on convergence analysis, we formulate the client selection problem to minimize the value of loss function in DPFL with heterogeneous privacy, which is a convex optimization problem and can be solved efficiently. Accordingly, we propose the DPFL-BCS (biased client selection) algorithm. The extensive experiment results with real datasets under both convex and non-convex loss functions indicate that DPFL-BCS can remarkably improve model utility compared with the SOTA baselines.

Figure 1: Model utility comparison of different algorithms under fixed privacy heterogeneity (GM = Gaussian Mechanism, LM = Laplace Mechanism).

Figure 2: Final model utility comparison of different algorithms by varying privacy heterogeneity  (GM = Gaussian Mechanism, LM = Laplace Mechanism).

Figure 3: Final model utility comparison of different algorithms by varying data heterogeneity α (GM = Gaussian Mechanism, LM = Laplace Mechanism).

Figure 4: Final model utility comparison of different algorithms by varying number of total iterations T (GM = Gaussian Mechanism, LM = Laplace Mechanism).

Figure 5: Runtime comparison of different algorithms on different datasets with the Gaussian mechanism.

Acknowledgement

This work are supported by: (i) National Natural Science Foundation of China (Grant U23B2026 and Grant 62372305); (ii) Guangdong Basic and Applied Basic Research Foundation (Grant 2024B1515040012); (iii) Shenzhen Science and Technology Program (Grant KJZD20230923114809020); (iv) Research Team Cultivation Program of Shenzhen University (Grant 2023QNT015); (v) Key Programs of Ningbo Municipal Natural Science Foundation (Grant 2024J021); (vi) Australian Research Council Discovery Project (Grant DP210101723).

Bibtex

@ARTICLE{11010143,

  author={Ma, Jiating and Zhou, Yipeng and Li, Qi and Sheng, Quan Z. and Cui, Laizhong and Liu, Jiangchuan},

  journal={IEEE Transactions on Dependable and Secure Computing},

  title={The Power of Bias: Optimizing Client Selection in Federated Learning With Heterogeneous Differential Privacy},

  year={2025},

  volume={22},

  number={5},

  pages={5672-5687},

  doi={10.1109/TDSC.2025.3572527}}

Downloads