In-Datacenter Performance Analysis of a Tensor Processing Unit · HackerPapers discussions

In-Datacenter Performance Analysis of a Tensor Processing Unit — mentioned by flakiness

Wed, 28 May 2025 03:31:38 GMT

[2017] (https://arxiv.org/abs/1704.04760)

Paper and discussion history. A mention is not an endorsement.

In-Datacenter Performance Analysis of a Tensor Processing Unit — mentioned by dgacmu

Sat, 12 Apr 2025 15:25:01 GMT

The first TPU (Seastar) was designed, tested, and deployed in 15 months: https://arxiv.org/pdf/1704.04760 They started becoming available internally in mid 2015.

Paper and discussion history. A mention is not an endorsement.

In-Datacenter Performance Analysis of a Tensor Processing Unit — mentioned by moandcompany

Tue, 08 Oct 2024 18:39:28 GMT

There's a wide range of inference accelerators in commercial use. For "edge" or embedded applications, an accelerator such as the Google Coral Edge TPU is a useful reference point where it is capable of up to 4 Trillion Operations per Second (4 TOPS), with up to 2 Watts of power consumption (2 TOPS/W), however the accelerator is limited to INT8 operations. It also has around 8 MB of memory for model storage. Meanwhile a general purpose or gaming GPU can support a wider range of instructions, single-precision, double-precision floating point, integer, etc). Geforce GTX 1060 for example: 4.375…

Paper and discussion history. A mention is not an endorsement.

In-Datacenter Performance Analysis of a Tensor Processing Unit — mentioned by vimarsh6739

Mon, 19 Feb 2024 19:07:03 GMT

Might be a bit out of context, but isn't the TPU also optimized for low latency inference? (Judging by reading the original TPU architecture paper here - https://arxiv.org/abs/1704.04760). If so, does Groq actually provide hardware support for LLM inference?

Paper and discussion history. A mention is not an endorsement.

In-Datacenter Performance Analysis of a Tensor Processing Unit — mentioned by emacs28

Mon, 06 Feb 2023 18:49:17 GMT

One good approach could be to base the architecture on the TPU v1 from [1]. There are also open-source accelerators you could get inspiration from, for example [2][3]. If you want to do less work/not hand code the RTL yourself then you could look into methods for automatically mapping OpenCL to an FPGA accelerator architecture (or a service like [3] provides pre-designed architectures for multiple FPGAs). [1] https://arxiv.org/abs/1704.04760 [2] https://github.com/jofrfu/tinyTPU [3] https://github.com/tensil-ai/tensil

Paper and discussion history. A mention is not an endorsement.

In-Datacenter Performance Analysis of a Tensor Processing Unit — mentioned by grandmczeb

Tue, 23 Apr 2019 01:07:11 GMT

In a conv layer, weights are shared across many input features. E.g. assume a 1x1 conv layer with a 28x28x3 input. You only need to load 3 weights even though there are effectively 28x28=748 different dot products. In practice, the input and output activations can be stored on chip as well (except for the first layer) which means the ratio of operations to DRAM accesses can be incredibly high. For some real world examples, take a look at the classic Eyeriss paper[1] which finds a ratio of 345-285 for AlexNet and VGG-16 respectively. You can also check out the TPU paper[2] which places the…

Paper and discussion history. A mention is not an endorsement.

In-Datacenter Performance Analysis of a Tensor Processing Unit — mentioned by elmarhaussmann

Thu, 26 Apr 2018 14:09:50 GMT

So far only very few details are disclosed. Here are two presentations: https://supercomputersfordl2017.github.io/Presentations/Imag... http://learningsys.org/nips17/assets/slides/dean-nips17.pdf For the last version of the TPU, Google provided more detail, e.g., in this paper: https://arxiv.org/pdf/1704.04760.pdf Hopefully, Google will publish something similar for TPUv2, but I have no knowledge whether or when that might happen.

Paper and discussion history. A mention is not an endorsement.

In-Datacenter Performance Analysis of a Tensor Processing Unit — mentioned by indescions_2018

Mon, 12 Feb 2018 14:52:41 GMT

The reserve TPU button has been available on the dashboard for the last few months. But I assume instances have been prioritized for large customers such as Two Sigma. From the paper: "Despite low utilization for some applications, the TPU is on average about 15X - 30X faster than its contemporary GPU or CPU, with TOPS/Watt about 30X - 80X higher. Moreover, using the GPU's GDDR5 memory in the TPU would triple achieved TOPS and raise TOPS/Watt to nearly 70X the GPU and 200X the CPU." In-Datacenter Performance Analysis of a Tensor Processing Unit https://arxiv.org/abs/1704.04760 Price is about…

Paper and discussion history. A mention is not an endorsement.

In-Datacenter Performance Analysis of a Tensor Processing Unit — HN submission

Wed, 06 Dec 2017 13:56:44 GMT

In-Datacenter Performance Analysis of a Tensor Processing Unit

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

In-Datacenter Performance Analysis of a Tensor Processing Unit — mentioned by puzzle

Wed, 23 Aug 2017 19:39:48 GMT

They compared TPU v1 with server hardware available at the time (2015, i.e. K80 and Haswell). Are the cards you mention from 2015? Are they for gaming or servers? Do they use ECC? If not, that rules them out right away. You can find the paper with the methodology, theoretical peaks and latency management at https://arxiv.org/abs/1704.04760

Paper and discussion history. A mention is not an endorsement.

In-Datacenter Performance Analysis of a Tensor Processing Unit — mentioned by dgacmu

Mon, 24 Jul 2017 15:29:37 GMT

One example is: https://arxiv.org/abs/1704.04760 There are many potential designs for these things, but the first gen TPU is one that works, is in production, and has been described in a paper. But you have to differentiate if you mean an inference engine, or something that can also do training. For HoloLens, it's probably going to be an inference unit, which means it'll possibly look something like a TPU, perhaps with more specific hardware support optimized for convolutions (which are very important for visual processing DNNs these days), as the NVidia tensor units are.

Paper and discussion history. A mention is not an endorsement.

In-Datacenter Performance Analysis of a Tensor Processing Unit — mentioned by bazizbaziz

Mon, 12 Jun 2017 18:25:13 GMT

> "The growing body of Big-data, HPC, and especially machine learning applications don’t need Windows and don’t perform on X86. So 2017 is the year Nvidia slips its leash and breaks free to become a genuinely viable competitive alternative to x86 based enterprise computing in valuable new markets that are unsuited to x86 based solutions." Google's TPU paper [0] showed the CPUs were relatively competitive in the machine learning space (within 2x of a K80). It's not true that x86 doesn't perform on these workloads. The existence of the TPU itself threatens Nvidia's dominance in the ML processor…

Paper and discussion history. A mention is not an endorsement.

In-Datacenter Performance Analysis of a Tensor Processing Unit — mentioned by matt_d

Sun, 07 May 2017 15:23:04 GMT

Event description: https://eecs.berkeley.edu/research/colloquium/170315 Slides: https://drive.google.com/file/d/0B5vMMuEXssYnR1oxcXVaaTFTQWM... Paper: https://arxiv.org/abs/1704.04760

Paper and discussion history. A mention is not an endorsement.