Infrequent Synchronization in Distributed AdaBoost
DOI:
https://doi.org/10.51408/1963-0141Keywords:
Distributed AdaBoost, Infrequent Synchronization, Ensemble Learning, Communication-Efficient Learning, Federated Boosting, Weak Learners, Scalability, Fault Tolerance, Real-World DeploymentAbstract
Distributed machine learning has become increasingly vital as data sources continue to expand geographically. Traditional ensemble methods such as AdaBoost demonstrate impressive predictive capabilities but often require frequent synchronization across nodes, resulting in significant communication overhead. This paper introduces a novel paradigm of infrequent synchronization in which nodes perform multiple rounds of local AdaBoost before exchanging partial or complete model updates. The potential advantages include reduced communication costs, the ability to handle intermittent connectivity, and competitive accuracy compared to fully synchronized approaches. A real-world use case in the trucking industry is presented to demonstrate the feasibility and value of this new approach. The paper concludes by outlining future directions and the expected impact on communication-efficient distributed learning.
References
Y. Freund and R. E. Schapire, “A decision-theoretic generalization of on-line learning and an application to boosting”, Journal of Computer and System Sciences, vol. 55, no. 1, pp. 119–139, 1997.
Y. Zhang and J. Huan, “Inductive multi-task learning with multiple view data”, Proceedings of the 17th ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, pp. 543–551, 2011.
L.-M. Ang, K. P. Seng et al., “Big sensor data systems for smart cities”, IEEE Internet of Things Journal, vol. 5, no. 2, pp. 468–476, 2018.
J. Hamer, G. Quinlan and S. Marshland, “FedBoost: Communication-efficient federated learning with boosting”, Proceedings of the International Conference on Distributed Machine Learning, pp. 1–8, 2020.
H. B. McMahan, E. Moore, D. Ramage, S. Hampson and B. A. y Arcas, “Communication-efficient learning of deep networks from decentralized data”, Proceedings of the 20th International Conference on Artificial Intelligence and Statistics (AISTATS), pp. 1273–1282, 2017.
K. Cheng, T. Fan, Y. Jin et al., “SecureBoost: A lossless federated learning framework”, IEEE Transactions on Big Data, vol. 7, no. 4, pp. 776–788, 2021.
M.-F. Balcan, A. Blum and S. Fine, “Communication efficient distributed learning”, Proceedings of the 25th Annual Conference on Learning Theory, pp. 35.1–35.22, 2012.
T. Li, A. K. Sahu, M. Zaheer et al., “Federated optimization in heterogeneous networks”, Proceedings of Machine Learning and Systems, vol. 2, pp. 429–450, 2020.
Y.-Y. Chiang, C.-J. Hsieh and I. S. Dhillon, “Communication-efficient distributed boosting algorithms”, Proceedings of the International Conference on Machine Learning (ICML), pp. 139–147, 2014.
P. Kairouz, H. B. McMahan et al., “Advances and open problems in federated learning”, Foundations and Trends in Machine Learning, vol. 14, no. 1–2, pp. 1–210, 2021.
S. Gilbert and N. Lynch, “Brewer's conjecture and the feasibility of consistent, available, partition-tolerant web services”, ACM SIGACT News, vol. 33, no. 2, pp. 51–59, 2002.
M. Li, D. G. Andersen, A. J. Smola and K. Yu, “Communication efficient distributed machine learning with the parameter server”, Advances in Neural Information Processing Systems, vol. 27, pp. 19–27, 2014.
A. Karbasi and K. G. Larsen, “Parallel boosting algorithms: Limitations and possibilities”, Journal of Machine Learning Research, vol. 25, pp. 1–34, 2024.
Y. Fraboni, R. Vidal, L. Kameni and M. Lorenzi, “FedBuff: Asynchronous federated learning with buffered updates”, IEEE Transactions on Neural Networks and Learning Systems, vol. 34, no. 1, pp. 102–114, 2023.
L. Huang, A. L. Shea, H. Qian, A.Masurkar, H. Deng and D. Liu, “Patient clustering improves efficiency of federated machine learning to predict mortality and hospital stay time using distributed electronic medical records”, Journal of Biomedical Informatics, vol. 99, p. 103291, 2020.
M. Polato, A. Sperduti and F. Chierichetti, “AdaBoost.F: Federated boosting with decision trees”, IEEE Transactions on Knowledge and Data Engineering, vol. 35, no. 3, pp. 2485–2496, 2023.
Downloads
Published
How to Cite
Issue
Section
License
Copyright (c) 2025 Arthur K. Oghlukyan & Luis Fernando de Mingo López
This work is licensed under a Creative Commons Attribution-NonCommercial 4.0 International License.
