Power Hungry Processing: Watts Driving the Cost of AI Deployment? · HackerPapers discussions

Power Hungry Processing: Watts Driving the Cost of AI Deployment? — HN submission

Sun, 14 Apr 2024 07:23:52 GMT

Power Hungry Processing: Watts Driving the Cost of AI Deployment?

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Power Hungry Processing: Watts Driving the Cost of AI Deployment? — mentioned by colesantiago

Wed, 03 Apr 2024 10:49:59 GMT

They are both the same. Both have breathless hype. Both are extremely overvalued and are a solution looking for a problem. Both aren't really useful unless you're (trying to count generating cat memes and broken hallucinating code a use case, it isn't). And last and the most important of all: Both of these waste a ton of energy either by training or running the software and are unlikely to make startups money due to it's rapid commoditization. https://arxiv.org/pdf/2311.16863.pdf https://www.theverge.com/24066646/ai-electricity-energy-watt...

Paper and discussion history. A mention is not an endorsement.

Power Hungry Processing: Watts Driving the Cost of AI Deployment? — mentioned by tomjohnneill

Fri, 23 Feb 2024 13:53:42 GMT

I can definitely imagine they're not covering the amortised cost of the training with the cost per individual inference request. It seems less likely to me that they're making a significant loss on each subsequent request, but again no source from me on that either. Looking a bit more into this, I found this paper: https://arxiv.org/pdf/2311.16863.pdf. It references a table saying that text generation uses 0.047 kWh per 1000 inferences, which is 1-2 orders of magnitude lower than my estimate. Though that is for GPT2, so possibly tracks to something roughly in the ~0.001 kWh per inference for…

Paper and discussion history. A mention is not an endorsement.

Power Hungry Processing: Watts Driving the Cost of AI Deployment? — mentioned by Baldbvrhunter

Thu, 21 Dec 2023 11:06:51 GMT

The actual paper [0] says approx 4860 joules per image, so 70 images is the same as boiling a litre of water. Charging a phone takes 20k joules. [0] https://arxiv.org/pdf/2311.16863.pdf

Paper and discussion history. A mention is not an endorsement.