Introduction

Introduction of the Tutorial(5 min.) by Dr. Sheng Di.

Session 1

Background of Error Bounded Lossy Compression and Large-Scale Model Training in Federated Environments (45 min.), by Dr. Sheng Di, Dr. Zhaorui Zhang, and Prof. Xiaodong Yu.

• The introduction of the error-bounded lossy compression: includes the motivation of lossy compression; what is the error-bounded lossy compression; the most popular errorbounded lossy compressors; their use-cases;
• The introduction of the large-scale model training in federated learning environments: includes what is federated learning; the parameter aggregation strategies; the bottlenecks of such tasks; the limitations of existing solutions; and the open frameworks for federated learning.
• The introduction of differential privacy (DP): includes what differential privacy is, how differential privacy protects the data, and how to apply DP in federated learning.

Session 2

Error-Bounded Lossy Compression for Communication Reduction for Large-Scale Model Training in Federated Environments ( 20 min.), by Dr. Zhaorui Zhang.

• The motivation why we want to using error-bounded lossy compression to compress the parameters for large-scale model training in federated learning systems.
• The algorithm to estimate an appropriate error bound that ensures the model accuracy while applying error-bounded lossy compression to model parameters.
• The resource utilization (CPUs, GPUs, and network) analysis and system performance optimization.
• Experiment results analysis about the error-bounded lossy parameter compression.

Session 3

Towards Fairness and Communication Efficiency for Large-Scale Model Training in Federated Environments based on Error-Bounded Lossy Parameter Compression ( 20 min.), by Dr. Zhaorui Zhang.

• Motivation to use the error-bounded lossy compression to compress parameters to achieve the communication efficiency and fairness simultaneously.
• The algorithm to estimate an appropriate error bound for error-bounded lossy compressors that can guarantee the model accuracy and fairness simultaneously.
• Theoretical analysis for the aggregated error caused by the lossy parameter compression.
• Experiment results analysis about the error-bounded lossy parameter compression.

BREAK(30 min)

Session 4

Differentially Private Quantization for PrivacyPreserving and Communication-Efficient Federated Training ( 20 min.), by Prof. Xiaodong Yu.

• The motivation why differential privacy (DP) should be integrated into quantization, and an overview of state-of-the-art DP-quantization approaches.
• Generic guidelines for injecting DP noise into the quantization pipeline and a structured categorization of existing DP-quantization methods.
• Introduction of representative DP-quantization techniques and our unified framework with standardized DP proofs and consistent utility analysis.
• Key findings from reproduction/replication studies and practical implications for designing next-generation DP-quantization mechanisms.

Session 5

APPFL: Open-Source Software Framework for Privacy-Preserving Federated Learning (20 min.), by Prof. Xiaoyi Lu.

• We will introduce APPFL, a highly modular, open-source Python framework for privacy-preserving federated learning that allows plug-and-play combination of FL algorithms, differential privacy mechanisms, communication protocols (MPI and gRPC), models, and datasets.
• We will introduce a new communication-efficient algorithm called IIADMM of APPFL, which eliminates the need to transfer dual variables from clients to server while achieving equal or better accuracy than FedAvg and ICEADMM.
• We will introduce how the APPFL integrates Laplace-based output perturbation differential privacy and the experiment results of APPFL on multiple datasets that IIADMM consistently outperforms existing methods under strong privacy budgets (ε = 10^-3).

Session 6

FedDES: Discrete Event Performance Simulation for Large-Scale Federated Learning ( 20 min.), by Prof. Xiaoyi Lu.

• The motivation why we want to proposing FedDES.
• The key design principle of the FedDES. How the FedDES simulates the large-scale federated learning.
• What kinds of training strategies can FedDES simulate for federated learning?
• The key results that FedDES achieved.

Session 7

Hands-On Exercises on Useful Federated Training Frameworks and Compression Tools. ( 25 min.), by Prof. Xiaoyi Lu and Dr. Zhaorui Zhang.

• How to deploy APPFL for federated learning;
• How to deploy FedDES, which is the simulator for federated learning and used for system performance analysis for largescale model training.
• How to integrate the error-bounded lossy compressors (such as LibPressio) into federated learning frameworks for communication reduction and privacy preserving.

Conclusion(5 min.)

Give a brief summary and conclude the tutorial, by ALL.