How does Federated Learning work in the real world?

Federated learning (FL) is a novel paradigm enabling distributed machine learning (ML) model training, while ensuring that training data remains on individual clients. The increasing need for privacy makes FL a highly promising method spearheading the future of ML. Although theoretically elegant, FL faces significant hurdles when it comes to real-world implementation. The key obstacle towards wider proliferation of FL is an inherent heterogeneity of real-world settings. Clients that may participate in FL range from computationally limited Internet of Things (IoT) devices, over low-end smartphones, to high-end multicore GPU-powered smartphones and beyond. Furthermore, devices may host a range of different applications that are simultaneously competing for hosts’ limited resources. In this work we will for the first time quantify the effects of heterogeneity on FL performance in a real-world testbed. We will first conduct a survey of edge devices that are suitable for FL and assemble a representative, yet diverse, testbed at our KAUST premises. Building upon the existing open-source solutions, we will then program a FL training framework enabling rapid prototyping, experimentation, and measurements within our testbed. We will then conduct a set of experiments geared towards profiling the effect of device hardware, software, and usage heterogeneity of FL training accuracy, convergence time, fairness, and energy use. Finally, we will analyze the results and extract guidelines for efficient FL in heterogeneous environments.

- Physical testbed comprising a range of IoT and mobile devices. - Programming framework for FL experimentation and evaluation. - Executable scenarios encompassing different use-cases and levels of the underlying system heterogeneity. - Report on the experimental results with metrics designed to profile the effect of system heterogeneity on holistic performance of FL. - Guidelines for advancing FL in heterogeneous environments.