Enhancing Privacy and Efficiency Techniques in Federated Learning Systems: Applications in Healthcare, Finance, and Smart Devices

Document Type : Research Paper

Author

Department of Computer Science, College of Computing and Informatics, Saudi Electronic University, Saudi Arabia.

10.22059/jitm.2025.102921

Abstract

Federated Learning (FL) has emerged as a revolutionary technique for distributed machine learning for training a model on shared data without sharing the data itself. Nevertheless, privacy-related concerns and scalability difficulties remain a problem. This paper discusses the state-of-the-art works to improve the privacy and convergence at FL frameworks for targeted healthcare and financial applications, as well as smart devices. It focuses on methodologies that preserve user privacy, such as differential privacy, homomorphic encryption, secure multi-party computation, and methods that enhance the model’s efficiency, including model compression, communication optimization, and adaptive optimization algorithms. To overcome these challenges, this study helps in the future design of FL systems for vital domains with high scalability.

Keywords

References

Abadi, M., Chu, A., Goodfellow, I., McMahan, H. B., Mironov, I., Talwar, K., & Zhang, L. (2016). Deep learning with differential privacy. Proceedings of the 2016 ACM SIGSAC Conference on Computer and Communications Security (CCS), 308–318. https://doi.org/10.1145/2976749.2978318
Bonawitz, K., Eichner, H., Grieskamp, W., Huba, D., Ingerman, A., Ivanov, V., ... & van Overveldt, T. (2019). Towards federated learning at scale: System design. Proceedings of the 2nd SysML Conference. Retrieved from https://arxiv.org/abs/1902.01046
Bonawitz, K., Ivanov, V., Kreuter, B., Marcedone, A., McMahan, H. B., Patel, S., ... & Seth, K. (2017). Practical secure aggregation for federated learning on user-held data. Advances in Neural Information Processing Systems (NeurIPS), 30, 1–12. Retrieved from https://arxiv.org/abs/1708.06689
Caldas, S., Konečný, J., McMahan, H. B., & Talwalkar, A. (2018). Expanding the reach of federated learning by reducing client resource requirements. arXiv preprint. Retrieved from https://arxiv.org/abs/1812.07210
Chen, Y., Xie, Y., Kairouz, P., & Song, L. (2020). Understanding model averaging in federated learning with structured optimization. Advances in Neural Information Processing Systems (NeurIPS), 33, 1–11. Retrieved from https://arxiv.org/abs/2003.10417
Geyer, R. C., Klein, T., & Nabi, M. (2017). Differentially private federated learning: A client level perspective. Advances in Neural Information Processing Systems (NeurIPS) Workshop. Retrieved from https://arxiv.org/abs/1712.07557
Hardy, S., Berkovsky, S., Penschuck, M., Germanakos, P., & Bellotti, F. (2020). Federated learning for user modeling. Proceedings of the 28th ACM Conference on User Modeling, Adaptation, and Personalization (UMAP), 1–11. https://doi.org/10.1145/3340631.3394868
Hynes, N., Dao, D., Yan, C., Cheng, F., Song, D., & Popa, R. A. (2018). A demonstration of Sterling: A privacy-preserving data marketplace. Proceedings of the VLDB Endowment, 11(12), 2086–2089. https://doi.org/10.14778/3229863.3236267
Kairouz, P., McMahan, H. B., Avent, B., Bellet, A., Bennis, M., Bhagoji, A. N., ... & Zhao, S. (2021). Advances and open problems in federated learning. Foundations and Trends® in Machine Learning, 14(1–2), 1–210. https://doi.org/10.1561/2200000083
Konečný, J., McMahan, H. B., Yu, F. X., Richtárik, P., Suresh, A. T., & Bacon, D. (2016). Federated learning: Strategies for improving communication efficiency. arXiv preprint. Retrieved from https://arxiv.org/abs/1610.05492
Li, Q., Wen, Z., Wu, Z., Hu, S., Wang, N., Li, Y., & He, B. (2019). A survey on federated learning systems: Vision, hype, and reality for data privacy and protection. arXiv preprint. Retrieved from https://arxiv.org/abs/1902.04885
Li, T., Sahu, A. K., Talwalkar, A., & Smith, V. (2020). Federated learning: Challenges, methods, and future directions. IEEE Signal Processing Magazine, 37(3), 50–60. https://doi.org/10.1109/MSP.2020.2975749
Li, X., Huang, K., Yang, W., Wang, S., & Zhang, Z. (2020). On the convergence of FedAvg on non-IID data. International Conference on Learning Representations (ICLR). Retrieved from https://arxiv.org/abs/1907.02189
Luo, C., Zhang, F., Zhou, Y., & Huang, L. (2021). Privacy-preserving federated learning for speech recognition. IEEE/ACM Transactions on Audio, Speech, and Language Processing, 29, 1495–1507. https://doi.org/10.1109/TASLP.2021.3050296
Mangal, A., Garg, H., & Bhatnagar, C. (2023). Assessing the performance of saliency detection method using various deep neural networks. Journal of Information Technology Management, 15(Special Issue), 23–34. https://doi.org/10.22059/jitm.2023.95243
McMahan, H. B., Moore, E., Ramage, D., Hampson, S., & Arcas, B. A. Y. (2017). Communication-efficient learning of deep networks from decentralized data. Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), 54, 1273–1282. Retrieved from https://arxiv.org/abs/1602.05629
Mohri, M., Sivek, G., & Suresh, A. T. (2019). Agnostic federated learning. International Conference on Machine Learning (ICML), 4661–4670. Retrieved from https://arxiv.org/abs/1902.00146
Nguyen, D. C., Ding, M., Pathirana, P. N., & Seneviratne, A. (2021). Federated learning for Internet of Things: A comprehensive survey. IEEE Communications Surveys & Tutorials, 23(3), 1622–1658. https://doi.org/10.1109/COMST.2021.3075439
Reisizadeh, A., Mokhtari, A., Hassani, H., Jadbabaie, A., & Pedarsani, R. (2020). FedPAQ: A communication-efficient federated learning method with periodic averaging and quantization. Proceedings of the 23rd International Conference on Artificial Intelligence and Statistics (AISTATS), 2020, 2021–2029. Retrieved from https://arxiv.org/abs/1909.06335
Sattler, F., Wiedemann, S., Müller, K.-R., & Samek, W. (2019). Robust and communication-efficient federated learning from non-i.i.d. data. IEEE Transactions on Neural Networks and Learning Systems, 31(9), 3400–3413. https://doi.org/10.1109/TNNLS.2019.2944482
Shokri, R., & Shmatikov, V. (2015). Privacy-preserving deep learning. Proceedings of the 22nd ACM SIGSAC Conference on Computer and Communications Security (CCS), 1310–1321. https://doi.org/10.1145/2810103.2813687
Smith, V., Chiang, C.-K., Sanjabi, M., & Talwalkar, A. (2017). Federated multi-task learning. Advances in Neural Information Processing Systems (NeurIPS), 30, 4424–4434. Retrieved from https://arxiv.org/abs/1705.10467
Truex, S., Baracaldo, N., Anwar, A., Steinke, T., Ludwig, H., Weber, B., & Pasupuleti, V. (2019). A hybrid approach to privacy-preserving federated learning. Proceedings of the 12th ACM Workshop on Artificial Intelligence and Security (AISec), 1–11. https://doi.org/10.1145/3338501.3357370
Wang, H., Yurochkin, M., Sun, Y., Papailiopoulos, D., & Khazaeni, Y. (2020). Federated learning with matched averaging. International Conference on Learning Representations (ICLR). Retrieved from https://arxiv.org/abs/2002.06440
Wang, J., Zhao, Y., Yao, Q., Kwok, J. T., & Ni, L. M. (2019). Federated learning with matched averaging. International Joint Conferences on Artificial Intelligence (IJCAI), 5050–5056. https://doi.org/10.24963/ijcai.2019/703
Yang, Q., Liu, Y., Cheng, Y., Kang, Y., Chen, T., & Yu, H. (2019). Federated learning. Synthesis Lectures on Artificial Intelligence and Machine Learning, 13(3), 1–207. https://doi.org/10.2200/S00960ED1V01Y201910AIM043
Yang, T., Andrew, G., Eichner, H., Sun, H., Li, W., Kong, N., ... & McMahan, H. B. (2018). Applied federated learning: Improving Google keyboard query suggestions. arXiv preprint. Retrieved from https://arxiv.org/abs/1812.02903
Zhang, Z., Liang, P. P., Zhang, C., & Gitter, A. (2021). Federated multi-modal learning with missing data. Proceedings of the 38th International Conference on Machine Learning (ICML), 2021. Retrieved from https://arxiv.org/abs/2009.06186
Zhao, Y., Li, M., Lai, L., Suda, N., Civin, D., & Chandra, V. (2018). Federated learning with non-iid data. arXiv preprint. Retrieved from https://arxiv.org/abs/1806.00582
Zhu, H., Zhou, J., & Xu, S. (2021). Data-free knowledge distillation for heterogeneous federated learning. Proceedings of the 39th International Conference on Learning Representations (ICLR). Retrieved from https://arxiv.org/abs/2012.05387
Zhu, L., Liu, Z., & Han, S. (2019). Deep leakage from gradients. Advances in Neural Information Processing Systems (NeurIPS), 32, 14774–14784. Retrieved from https://arxiv.org/abs/1906.08935
Journal of Information Technology Management
Volume 17, Special Issue
2025
Pages 45-62

Files

Share

How to cite

Statistics