Energy and Policy Considerations for Deep Learning in NLP · HackerPapers discussions

Energy and Policy Considerations for Deep Learning in NLP — mentioned by logicprog

Thu, 01 Feb 2024 13:32:49 GMT

I'm not sure your premise even makes any sense here, because it doesn't take an artist much more resources to produce art then it took them to just exist for the same amount of time. They're still just eating, sleeping, making basic usage of the computer, using heating and light, and so on either way. Whereas someone using dall-e is doing all of that plus relying on the immense training costs of the artificial intelligence. That basic usage of the computer in order to use the machine learning model might be shorter than the basic use of the computer to use procreate or something, but they'll…

Paper and discussion history. A mention is not an endorsement.

Energy and Policy Considerations for Deep Learning in NLP — mentioned by toomuchtodo

Mon, 20 Nov 2023 03:27:46 GMT

Model training can at least follow the sun or clean energy forecasts. Latency from distance isn’t an issue as it would be for serving users requests. You can stage data for training runs. If you train during periods where the price of power goes negative, you get paid to take the power. These workloads are going to run, so best to run them where the impact is lowest for the effort involved. https://blog.google/inside-google/infrastructure/data-center... https://www.electricitymaps.com/blog/announcing-our-partners... https://www.electricitymaps.com/blog/how-companies-reduce-th...…

Paper and discussion history. A mention is not an endorsement.

Energy and Policy Considerations for Deep Learning in NLP — mentioned by SilverBirch

Mon, 22 May 2023 16:38:40 GMT

I don't know if you just... hope people don't read your citations. But as far as I could see from your citations you're not making sensible claims. The first citation the article talks about her citing someone else's work about carbon footprint. So firstly, they're not her calculations - the paper is by Strubell[1]. Secondly, the paper absolutely does not assume the compute is running from jet fuel! It calculates the total energy to train a model and then uses US EPA data to show how much CO2 that would be in practice and then compares that to how much CO2 emissions a flight puts out.…

Paper and discussion history. A mention is not an endorsement.

Energy and Policy Considerations for Deep Learning in NLP — mentioned by mempko

Mon, 22 May 2023 16:36:14 GMT

The actual paper [0] that Gebru wrote referenced Strubell's climate impact paper [1]. None of them make the assumption that google compute runs from jet fuel. They compare training the models to flying, driving, etc. In the "Gender Shades" paper [2] as you call it, she doesn't use race but dermatologist approved Fitzpatrick Skin Type in their analysis as a proxy which of course covers all people including Asians. The kind of bullshit (a technical term) response you gave is common among reactionaries who don't read, and can't do science. [0] https://dl.acm.org/doi/pdf/10.1145/3442188.3445922…

Paper and discussion history. A mention is not an endorsement.

Energy and Policy Considerations for Deep Learning in NLP — mentioned by soraki_soladead

Thu, 14 Oct 2021 16:31:54 GMT

This is somewhat well covered in prior research, in so far as the information required is available. Here are some recent evaluations of the carbon footprint of modern models: https://arxiv.org/abs/1906.02243 (cited in the above paper) > The U.S. Environmental Protection Agency (EPA) provides average CO2 produced (in pounds per kilowatt-hour) for power consumed in the U.S. (EPA, 2018), which we use to convert power to estimated CO2 emissions: > CO2e = 0.954 pt > This conversion takes into account the relative proportions of different energy sources (primarily natural gas, coal, nuclear and…

Paper and discussion history. A mention is not an endorsement.

Energy and Policy Considerations for Deep Learning in NLP — mentioned by culi

Fri, 24 Sep 2021 22:01:44 GMT

Here's the original study[1] that seems to be the primary source for this article. It's an important study from a respectable journal. To be frank, it's pretty disconcerting that the top comments on this thread are those writing off the topic on the premise alone while those comments actually engaging with the topic seem to be at the bottom [1] https://arxiv.org/pdf/1906.02243.pdf

Paper and discussion history. A mention is not an endorsement.

Energy and Policy Considerations for Deep Learning in NLP — mentioned by css

Fri, 04 Oct 2019 13:21:58 GMT

The citation for the "surprisingly large carbon footprint" [0] is crazy. The paper alleges that a car, including fuel, has a lifetime footprint of 126,000 lbs of CO₂e, and training a big NN transformer consumes 626,155 lbs of CO₂e, almost a 5-fold increase. [0] https://arxiv.org/pdf/1906.02243.pdf

Paper and discussion history. A mention is not an endorsement.

Energy and Policy Considerations for Deep Learning in NLP — HN submission

Fri, 04 Oct 2019 10:49:18 GMT

Energy and Policy Considerations for Deep Learning in NLP

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Energy and Policy Considerations for Deep Learning in NLP — mentioned by briga

Fri, 20 Sep 2019 00:10:06 GMT

On the other hand, as software and frameworks get more complex, they require more compute to run. Software hasn't decoupled economic growth from energy consumption, it has just made the energy consumption harder to see. Edit: since people are asking for examples, I would have to point to the insane cost of training modern ML models. For instance, according to [1], training a single big NLP model can burn up to 620,000 lbs of CO2. I say a lot of energy consumption is hidden because it's generally moved to data centres, which by the way use up astronomical amounts of energy [2] [1]…

Paper and discussion history. A mention is not an endorsement.

Energy and Policy Considerations for Deep Learning in NLP — HN submission

Fri, 26 Jul 2019 13:50:29 GMT

Energy and Policy Considerations for Deep Learning in NLP

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Energy and Policy Considerations for Deep Learning in NLP — HN submission

Wed, 19 Jun 2019 16:18:29 GMT

Energy and Policy Considerations for Deep Learning in NLP [pdf]

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Energy and Policy Considerations for Deep Learning in NLP — HN submission

Tue, 11 Jun 2019 05:32:26 GMT

Energy and Policy Considerations for Deep Learning in NLP [pdf]

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.