Communication-Efficient Learning of Deep Networks from Decentralized Data · HackerPapers discussions

Communication-Efficient Learning of Deep Networks from Decentralized Data — mentioned by techwizrd

Wed, 26 Apr 2023 16:52:22 GMT

It's quite misleading, actually. GBoard uses Federated Learning [0] so that models are trained locally on the device and keywords are not sent to the central server. Both Google and iOS use Federated Learning, Federated Evaluation & Tuning, Differential Privacy, and other privacy-preserving AI techniques and/or on-device ML to avoid sending user data to the server. Both have done so publicly since 2017 and have published many papers and talks on the subject. 0: https://arxiv.org/abs/1602.05629

Paper and discussion history. A mention is not an endorsement.

Communication-Efficient Learning of Deep Networks from Decentralized Data — mentioned by est31

Sun, 27 Dec 2020 14:57:42 GMT

One important corollary of this is that "privacy respecting" federated learning schemes [0] where you train a model locally with your data and only upload the deltas might leak your private data after all. [0]: https://arxiv.org/abs/1602.05629

Paper and discussion history. A mention is not an endorsement.

Communication-Efficient Learning of Deep Networks from Decentralized Data — HN submission

Wed, 24 May 2017 20:30:48 GMT

Communication-Efficient Learning of Deep Networks from Decentralized Data

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Communication-Efficient Learning of Deep Networks from Decentralized Data — mentioned by andreyk

Fri, 07 Apr 2017 06:49:54 GMT

The paper: https://arxiv.org/pdf/1602.05629.pdf The key algorithmic detail: it seems they have each device perform multiple batch updates to the model, and then average all the multi-batch updates. "That is, each client locally takes one step of gradient descent on the current model using its local data, and the server then takes a weighted average of the resulting models. Once the algorithm is written this way, we can add more computation to each client by iterating the local update. " They do some sensible things with model initialization to make sure weight update averaging works, and show…

Paper and discussion history. A mention is not an endorsement.

Communication-Efficient Learning of Deep Networks from Decentralized Data — mentioned by whym

Fri, 07 Apr 2017 03:15:27 GMT

Their papers mentioned in the article: Federated Learning: Strategies for Improving Communication Efficiency (2016) https://arxiv.org/abs/1610.05492 Federated Optimization: Distributed Machine Learning for On-Device Intelligence (2016) https://arxiv.org/abs/1610.02527 Communication-Efficient Learning of Deep Networks from Decentralized Data (2017) https://arxiv.org/abs/1602.05629 Practical Secure Aggregation for Privacy Preserving Machine Learning (2017) http://eprint.iacr.org/2017/281

Paper and discussion history. A mention is not an endorsement.