FedorAS: Federated Architecture Search under system heterogeneity — mentioned by niclane7

Wed, 22 Mar 2023 15:37:09 GMT

Yes, this would be exciting to see. One approach wouldn't require federated learning however. If you had direct access to the data then you could build a conventionally trained large language model (i.e., collect all the data together placed in a data center). However, given the context of this discussion -- you are probably asking about if we could use Flower to train in a federated manner. I believe so. Although again, we'd probably be training a LLM which brings added complications due to its size (and other factors). Internally at Flower we have been testing methods to overcome this and…

Paper and discussion history. A mention is not an endorsement.

FedorAS: Federated Architecture Search under system heterogeneity · HackerPapers discussions

FedorAS: Federated Architecture Search under system heterogeneity — mentioned by niclane7