1 of 79

1

Federated Learning Can Find Friends �That Are Advantageous �and Help with Low-Resource Machine Translation

Eduard Gorbunov

MBZUAI

October 16, 2024

Federated Learning One-World Seminar

2 of 79

2

Outline

  1. Federated Learning

​

​

  • Federated Learning with a Target Client

​

​

  • Low-Resource Machine Translation

3 of 79

3

Federated Learning

4 of 79

4

The Problem

model parameters

# of parameters

. . .

5 of 79

5

The Problem

model parameters

# of parameters

# of workers/clients

 

. . .

6 of 79

6

The Problem

model parameters

# of parameters

# of workers/clients

 

. . .

 

7 of 79

7

The Problem

model parameters

# of parameters

# of workers/clients

 

. . .

 

Key features:

  • Data is heterogeneous among clients
  • Data is private

8 of 79

8

Parallel SGD

. . .

 

9 of 79

9

Parallel SGD

. . .

 

10 of 79

10

Parallel SGD

. . .

 

11 of 79

11

Parallel SGD: (Some) Good Properties

  • Simple method

​

  • Provable convergence

​

  • Cheap iterations

​

  • Can be improved in terms of communication efficiency

12 of 79

12

Federated Learning�with a Target Client

13 of 79

13

Samuel Horváth�Assistant Professor �MBZUAI

N. Tupitsa, S. Horváth, M. Takáč, E. Gorbunov. Federated Learning Can Find Friends That Are Advantageous (arXiv:2402.05050)

Martin Takáč�Associate Professor �MBZUAI

Nazarii Tupitsa�Research Assistant �MBZUAI

14 of 79

14

Federated Learning with a Target Client

Standard Federated Learning

 

15 of 79

15

Federated Learning with a Target Client

Standard Federated Learning

 

Federated Learning with a Target Client

 

16 of 79

16

Federated Learning with a Target Client

Standard Federated Learning

 

Federated Learning with a Target Client

 

  • Eduard, wait a second… Isn’t it trivial?

17 of 79

17

Federated Learning with a Target Client

Standard Federated Learning

 

Federated Learning with a Target Client

 

  • Eduard, wait a second… Isn’t it trivial?
  • Workers have only finite number of samples
  • Maybe, some workers can be helpful
  • But we don’t know which ones

18 of 79

18

Federated Learning with a Target Client

Standard Federated Learning

Personalized Federated Learning

 

 

Federated Learning with a Target Client

 

  • Eduard, wait a second… Isn’t it trivial?
  • Workers have only finite number of samples
  • Maybe, some workers can be helpful
  • But we don’t know which ones

19 of 79

19

Is Parallel SGD Good for FL with a Target Client?

Parallel SGD:

, where

20 of 79

20

Is Parallel SGD Good for FL with a Target Client?

Parallel SGD:

, where

 

21 of 79

21

Is Parallel SGD Good for FL with a Target Client?

Parallel SGD:

, where

 

 

22 of 79

22

Is Parallel SGD Good for FL with a Target Client?

Parallel SGD:

, where

 

 

23 of 79

23

Naive Approach

Use target’s client data only:

 

24 of 79

24

Naive Approach

Use target’s client data only:

 

  • Poor generalization if the target client’s dataset is small
  • Small mini-batch 🡪 slow convergence for some tasks

25 of 79

25

Non-Implementable Approach

Use data from clients with the same data distribution:

26 of 79

26

Non-Implementable Approach

Use data from clients with the same data distribution:

subset of clients with�the same data distribution�as for the target client

27 of 79

27

Non-Implementable Approach

Use data from clients with the same data distribution:

 

subset of clients with�the same data distribution�as for the target client

28 of 79

28

Non-Implementable Approach

Use data from clients with the same data distribution:

 

subset of clients with�the same data distribution�as for the target client

29 of 79

29

New Method: MeritFed

  • Update rule:

30 of 79

30

New Method: MeritFed

aggregation weights

  • Update rule:

31 of 79

31

New Method: MeritFed

aggregation weights

  • Update rule:
  • Key novelty:

32 of 79

32

New Method: MeritFed

aggregation weights

  • Update rule:
  • Key novelty:

probability simplex

33 of 79

33

New Method: MeritFed

aggregation weights

  • Update rule:
  • Key novelty:

How to solve the above auxiliary problem?

probability simplex

34 of 79

34

MeritFed: How to Solve Auxiliary Problem

Approach 1: use fresh data (if one can get fresh samples)

35 of 79

35

MeritFed: How to Solve Auxiliary Problem

Approach 1: use fresh data (if one can get fresh samples)

  • Standard stochastic optimization on a simplex – can be solved via Stochastic Mirror Descent
  • In practice, data is limited 🡪 we don’t have fresh data

36 of 79

36

MeritFed: How to Solve Auxiliary Problem

Approach 1: use fresh data (if one can get fresh samples)

  • Standard stochastic optimization on a simplex – can be solved via Stochastic Mirror Descent
  • In practice, data is limited 🡪 we don’t have fresh data

 

37 of 79

37

MeritFed: How to Solve Auxiliary Problem

Approach 1: use fresh data (if one can get fresh samples)

  • Standard stochastic optimization on a simplex – can be solved via Stochastic Mirror Descent
  • In practice, data is limited 🡪 we don’t have fresh data

 

38 of 79

38

MeritFed: How to Solve Auxiliary Problem

Approach 1: use fresh data (if one can get fresh samples)

  • Standard stochastic optimization on a simplex – can be solved via Stochastic Mirror Descent
  • In practice, data is limited 🡪 we don’t have fresh data

 

extra validation/training dataset

 

39 of 79

39

Mirror Descent

on a probability simplex with Kulback-Leibler distance as a Bregman divergence

Problem:

Method:

  • Exponent and product are computed component-wise
  • One can use a zeroth-order version of it for the auxiliary problem in MeritFed (for privacy)

Amir Beck. 2017. First-order methods in optimization. SIAM.

40 of 79

40

Convergence Theory

Assumptions:

 

41 of 79

41

Convergence Theory

Assumptions:

 

42 of 79

42

Convergence Theory

Assumptions:

 

43 of 79

43

Convergence Theory

Assumptions:

 

 

44 of 79

44

Convergence Theory

Assumptions:

 

 

the same bound as for “Ideal” Parallel SGD that uses only clients with the same data distribution

45 of 79

45

Convergence Theory

Assumptions:

 

 

the same bound as for “Ideal” Parallel SGD that uses only clients with the same data distribution

Term caused by �inaccurate solution�of aux. problem

46 of 79

46

Numerical Results: Problems and Baselines

  • Mean estimation

​

  • Image classification: ResNet18 @ CIFAR10�
  • Emotions classification: BERT @ GoEmotions

We tested MeritFed on several tasks:

47 of 79

47

Numerical Results: Problems and Baselines

  • Mean estimation

​

  • Image classification: ResNet18 @ CIFAR10�
  • Emotions classification: BERT @ GoEmotions�
  • ResNet18 @ MedMNIST

We tested MeritFed on several tasks:

Baselines:

 

  • Wu, H., & Wang, P. (2021). Fast-convergent federated learning with adaptive weighting. IEEE Transactions on Cognitive Communications and Networking
  • Chen, S., Crammer, K., He, H., Roth, D., & Su, W. J. (2021). Weighted training for cross-task learning. arXiv preprint arXiv:2105.14095.
  • Li, T., Sahu, A. K., Zaheer, M., Sanjabi, M., Talwalkar, A., & Smith, V. (2020). Federated optimization in heterogeneous networks.�Proceedings of Machine learning and systems.

48 of 79

48

Mean Estimation

Setup:

Problem:

 

49 of 79

49

Mean Estimation

Setup:

Problem:

 

50 of 79

50

Mean Estimation

Setup:

Problem:

 

51 of 79

51

ResNet18 @ CIFAR10: Setup

 

52 of 79

52

ResNet18 @ CIFAR10: Results with Extra Data

53 of 79

53

ResNet18 @ CIFAR10: Results without Extra Data

MeritFed works even without extra data!

54 of 79

54

BERT @ GoEmotions: Setup

 

55 of 79

55

BERT @ GoEmotions: Results with Extra Data

56 of 79

56

BERT @ GoEmotions: Results with Extra Data

57 of 79

57

Bonus: Byzantine-Robustness for Free

Setup:

  • Mean estimation with 55 clients�
  • 5 clients have the same distribution as a target one�
  • 50 clients know the target distribution and perform a Byzantine-attack
  • Observation: MeritFed will work even if other workers send (intentionally) “harmful” vectors
  • In other words, MeritFed is Byzantine-robust

Attacks:

 

  • Baruch, G., Baruch, M., & Goldberg, Y. A little is enough: Circumventing defenses for distributed learning. NeurIPS 2019.
  • Xie, C., Koyejo, O., & Gupta, I. Fall of empires: Breaking byzantine-tolerant sgd by inner product manipulation. UAI 2020.

58 of 79

58

Bonus: Byzantine-Robustness for Free

59 of 79

59

Low-Resource Machine Translation

60 of 79

60

Samuel Horváth�Assistant Professor MBZUAI

V. Moskvoretskii, N. Tupitsa, C. Biemann, S. Horváth, E. Gorbunov, I. Nikishina. Low-Resource Machine Translation through the Lens of Personalized Federated Learning (EMNLP Findings 2024)

Irina Nikishina�Postdoc �Universität Hamburg

Chris Biemann�Professor �Universität Hamburg

Nazarii Tupitsa�Research Assistant MBZUAI

Viktor Moskvoretskii�Researcher�Skoltech�(searches for PhD positions!)

61 of 79

61

There are about 7000 Languages

https://phontron.com/class/multiling2022/assets/slides/multiling-03-typology.pdf

…but many of them possess low amount of resources (texts)

62 of 79

62

Training LLMs for Low-Resource Languages

  • Cross-lingual transfer learning (from high-recource to low-recource languages)

Existing approaches:

https://phontron.com/class/multiling2022/assets/slides/multiling-03-typology.pdf

63 of 79

63

Training LLMs for Low-Resource Languages

  • Cross-lingual transfer learning (from high-recource to low-recource languages)

​

  • Zero-shot learning: rain a model in one domain and assume it generalizes more or less out-of-the-box in a low-resource domain

Existing approaches:

https://phontron.com/class/multiling2022/assets/slides/multiling-03-typology.pdf

64 of 79

64

Training LLMs for Low-Resource Languages

  • Cross-lingual transfer learning (from high-recource to low-recource languages)

​

  • Zero-shot learning: rain a model in one domain and assume it generalizes more or less out-of-the-box in a low-resource domain�
  • Few shot learning: train a model in one domain and use only few examples from a low-resource domain to adapt it

Existing approaches:

https://phontron.com/class/multiling2022/assets/slides/multiling-03-typology.pdf

65 of 79

65

Training LLMs for Low-Resource Languages

  • Cross-lingual transfer learning (from high-recource to low-recource languages)

​

  • Zero-shot learning: rain a model in one domain and assume it generalizes more or less out-of-the-box in a low-resource domain�
  • Few shot learning: train a model in one domain and use only few examples from a low-resource domain to adapt it�
  • Joint multilingual learning: train a single model on a mix of datasets in all languages

Existing approaches:

https://phontron.com/class/multiling2022/assets/slides/multiling-03-typology.pdf

66 of 79

66

Training LLMs for Low-Resource Languages

  • Cross-lingual transfer learning (from high-recource to low-recource languages)

​

  • Zero-shot learning: rain a model in one domain and assume it generalizes more or less out-of-the-box in a low-resource domain�
  • Few shot learning: train a model in one domain and use only few examples from a low-resource domain to adapt it�
  • Joint multilingual learning: train a single model on a mix of datasets in all languages

Existing approaches:

https://phontron.com/class/multiling2022/assets/slides/multiling-03-typology.pdf

  • Main difficulty: it is unclear in-advance what languages to use as “helpers” and to what extent

67 of 79

67

Training LLMs for Low-Resource Languages

  • Cross-lingual transfer learning (from high-recource to low-recource languages)

​

  • Zero-shot learning: rain a model in one domain and assume it generalizes more or less out-of-the-box in a low-resource domain�
  • Few shot learning: train a model in one domain and use only few examples from a low-resource domain to adapt it�
  • Joint multilingual learning: train a single model on a mix of datasets in all languages

Existing approaches:

https://phontron.com/class/multiling2022/assets/slides/multiling-03-typology.pdf

  • Main difficulty: it is unclear in-advance what languages to use as “helpers” and to what extent�
  • Wait a second… but this is exactly the same problem as in FL with a target client

68 of 79

68

Languages in NLP As Clients in FL

Federated Learning

Natural Language Processing

69 of 79

69

Languages in NLP As Clients in FL

Federated Learning

Natural Language Processing

Not much data

Harder to train a model

70 of 79

70

Languages in NLP As Clients in FL

Federated Learning

Natural Language Processing

Not much data

Harder to train a model

Let us apply MeritFed!

71 of 79

71

MeritOpt

a.k.a. MeritFed-type wrapper for any stochastic first-order method

For example, OptStep can be a step of SGD, AdaGrad, RMSProp, Adam, etc.

72 of 79

72

Considered Tasks and Setup

  • Multilingual Machine Translation on South East Asian languages
  • Multilingual Machine Translation on he subset of Sami languages (from Finno-Ugric languages)

Tasks:

Model: M2M100

Baselines:

  • Fine-tuning to all languages including the target language (FTall)
  • Fine-tuning to all languages except the target language (FTnot)
  • Fine-tune to the target language only (FTonlyt)
  • Continuous Pretraining to all languages, followed by additional fine-tuning to the target language (CPall)
  • Continuous Pretraining to all languages but the target, followed by additional fine-tuning to the target language (CPnot)

73 of 79

73

South East Asian Languages

74 of 79

74

South East Asian Languages

75 of 79

75

South East Asian Languages

76 of 79

76

Conclusion

77 of 79

77

Main Takeaways

  • Merit(Fed)Opt – new method for (FL with a target client) for learning from the diverse datasets�
  • Personalized Federated Learning can be applied to the training of LLMs�for low-resource languages

78 of 79

78

Limitations

  • Scalability

​

  • We tried the approach for low-resource languages only�
  • Results are provided for M2M100, while numerous LLMs are available�
  • Only one target client�
  • Theory does not show how helpful other clients are if they have different but close distribution

79 of 79

79

Thank you!