The Grand Illusion: The Myth of Software Portability and Implications for ML Progress · HackerPapers discussions

The Grand Illusion: The Myth of Software Portability and Implications for ML Progress — mentioned by felarof

Tue, 24 Sep 2024 00:34:37 GMT

Haha, TPUs are not slow :) All of Google's training (including Gemini models) is done on TPUs. There are good 1p [a] and 3p [b] benchmarks comparing TPUs vs NVIDIA GPUs. [a] - https://github.com/GoogleCloudPlatform/vertex-ai-samples/blo... [b] - https://arxiv.org/pdf/2309.07181

Paper and discussion history. A mention is not an endorsement.

The Grand Illusion: The Myth of Software Portability and Implications for ML Progress — mentioned by felarof

Wed, 11 Sep 2024 20:04:58 GMT

Google published this benchmark a year or so ago comparing TPU vs NVIDIA (https://github.com/GoogleCloudPlatform/vertex-ai-samples/blo...) Conclusion is at the bottom, but TLDR was TPUs were 33% cheaper (performance per dollar) and JAX scales very well compared to PyTorch. If you are curious, there was a thorough comparison done by Cohere and they published their paper https://arxiv.org/pdf/2309.07181 -- TPU+JAX turned out to be more performant and more fault tolerant (less weird errors).

Paper and discussion history. A mention is not an endorsement.

The Grand Illusion: The Myth of Software Portability and Implications for ML Progress — HN submission

Fri, 22 Mar 2024 14:38:58 GMT

The Myth of Software Portability and Implications for ML Progress (2023)

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

The Grand Illusion: The Myth of Software Portability and Implications for ML Progress — mentioned by nisten

Fri, 26 Jan 2024 22:35:02 GMT

Meh, the comparison is somewhat pointless when it doesn't account for the slowdown that the vast majority of pytorch codebases experience on TPU's vs using JAX and it's accelerations specific to TPUs, and vice versa. https://arxiv.org/pdf/2309.07181.pdf Meaning the TPU v5p is likely slower than H100 for most ML workloads that depend on pytorch.

Paper and discussion history. A mention is not an endorsement.

The Grand Illusion: The Myth of Software Portability and Implications for ML Progress — mentioned by elwypea

Tue, 17 Oct 2023 08:45:53 GMT

Easier said than done. Even with Google level resources, TPU support for pytorch is patchy (https://arxiv.org/abs/2309.07181). Device abstraction is not great, assumes CUDA in unexpected places.

Paper and discussion history. A mention is not an endorsement.

The Grand Illusion: The Myth of Software Portability and Implications for ML Progress — HN submission

Thu, 21 Sep 2023 17:42:16 GMT

The Myth of Software Portability and Implications for ML Progress

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.