Federated Learning: Strategies for Improving Communication Efficiency · HackerPapers discussions

Federated Learning: Strategies for Improving Communication Efficiency — HN submission

Fri, 07 Apr 2017 16:20:22 GMT

Federated Learning: Strategies for Improving Communication Efficiency

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Federated Learning: Strategies for Improving Communication Efficiency — mentioned by whym

Fri, 07 Apr 2017 03:15:27 GMT

Their papers mentioned in the article: Federated Learning: Strategies for Improving Communication Efficiency (2016) https://arxiv.org/abs/1610.05492 Federated Optimization: Distributed Machine Learning for On-Device Intelligence (2016) https://arxiv.org/abs/1610.02527 Communication-Efficient Learning of Deep Networks from Decentralized Data (2017) https://arxiv.org/abs/1602.05629 Practical Secure Aggregation for Privacy Preserving Machine Learning (2017) http://eprint.iacr.org/2017/281

Paper and discussion history. A mention is not an endorsement.

Federated Learning: Strategies for Improving Communication Efficiency — mentioned by wnoise

Fri, 07 Apr 2017 03:10:40 GMT

For the bits described in https://arxiv.org/abs/1610.02527 you're essentially correct. Though it's still stochastic, and you can have mini-batching on each node. The interesting technical bits are in https://arxiv.org/abs/1610.05492 To save on update bandwidth, they either restrict the gradient to a lower dimensional space, or compress by quantizing the full gradient (which should effectively add zero-mean noise) before sending it back. (In theory they could do both of these, but they didn't actually test that.)

Paper and discussion history. A mention is not an endorsement.