Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking · HackerPapers discussions

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking — mentioned by aleinin

Wed, 20 Aug 2025 17:22:40 GMT

Cool project! How do you think about targeting hardware-specific ISAs directly? There’s an interesting paper from Citadel (https://arxiv.org/pdf/1804.06826) that highlights inefficiencies in nvcc for the Volta architecture. Do you see Luminal’s search-based paradigm eventually extending beyond outperforming handwritten kernels, towards actually competing with NVIDIA’s compiler optimizations at the PTX level?

Paper and discussion history. A mention is not an endorsement.

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking — mentioned by GregarianChild

Sat, 07 Jun 2025 09:30:29 GMT

Modern GPU instructions are often VLIW and the compiler has to do a lot to schedule them. For example, Nvidia's Volta (from 2017) uses 128-bit to encode each instruction. According to [1], the 128 bits in a word are used as follows: • at least 91 bits are used to encode the instruction • at least 23 bits are used to encode control information associated to multiple instructions • the remaining 14 bits appeared to be unused AMD GPUs are similar, I believe. VLIW is good for instruction density. VLIW was unsuccessful in CPUs like Itanium because the compiler was expected to handle…

Paper and discussion history. A mention is not an endorsement.

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking — HN submission

Fri, 24 Jan 2025 13:17:42 GMT

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking — mentioned by wcunning

Thu, 31 Oct 2024 16:12:45 GMT

The issue there is that that info is what Nvidia chooses to port out from the on-chip execution. Most of what we can do for observation is in the kernel driver space and not really on-chip or even low level transit to the chip. One of the other commenters pointed out that you can get huge benefits from avoiding busy waiting on the returned data from the chip, which makes total sense, but also increases latency, which didn't work for my near-realtime use case when I was investigating. Other than those types of low hanging fruit where you can accept a little latency for better power state…

Paper and discussion history. A mention is not an endorsement.

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking — HN submission

Sat, 20 Jan 2024 10:58:00 GMT

Dissecting the Nvidia Volta GPU Architecture via Microbenchmarking (2018)

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking — HN submission

Thu, 19 Jan 2023 20:50:18 GMT

Dissecting the Nvidia Volta GPU Architecture via Microbenchmarking (2018) [pdf]

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking — mentioned by dragontamer

Mon, 01 Aug 2022 20:38:23 GMT

There's some very interesting flags at the lowest level of NVidia SASS (machine code) that the PTX compiler adds. https://arxiv.org/pdf/1804.06826.pdf PDF Page 14 (physical page 12) shows that every instruction on NVidia Volta SASS has 4-bits of "Reuse flags", 6-bits of "wait barrier mask", 3-bits of "Read Barrier Index", 3-bits of "Write Barrier Index", 1-bit "Yield Flag" and 4-bits "Stall Cycles". > Wait barrier mask; Read/Write barrier index. While most instructions have fixed latency and can be statically scheduled by the assembler, instructions involving memory and shared resources…

Paper and discussion history. A mention is not an endorsement.

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking — mentioned by dragontamer

Sun, 03 Oct 2021 03:50:26 GMT

> Do you have a hardcore introduction with even more detail / perhaps even with examples of implementations? Hard to beat the actual technical references when you want something hardcore! The actual CUDA documentation is good: https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.... In particular, "Section 5: Performance Guidelines": https://docs.nvidia.com/cuda/cuda-c-programming-guide/index.... gives a lot of micro-architectural details, including those nasty "bank conflicts" that people keep talking about. Honestly, the original documentation says it best with just a few paragraphs…

Paper and discussion history. A mention is not an endorsement.

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking — mentioned by dragontamer

Sat, 30 Jan 2021 14:00:51 GMT

> You assume so, for Volta onwards that's not true. https://arxiv.org/pdf/1804.06826.pdf NVidia's SASS on Volta is pretty clearly SIMD-instructions. The PTX "virtual machine assembly" is fully documented. SASS is not documented, but it ain't a secret either. SASS follows closely with PTX, with exception of some "barrier bits" that seem to track dependencies (probably compiler-managed read/write dependency chains). > Each SIMT thread on NVIDIA GPUs for Volta onwards has its own instruction pointer. And yes, it's scalar instructions at the ISA level. When your "scalar" instruction executes…

Paper and discussion history. A mention is not an endorsement.

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking — mentioned by dragontamer

Mon, 14 Sep 2020 02:35:09 GMT

NVidia Volta: https://arxiv.org/pdf/1804.06826.pdf Each machine instruction on NVidia Volta has the following information: * Reuse Flags * Wait Barrier Mask * Read/Write barrier index (6-bit bitmask) * Read Dependency barriers * Stall Cycles (4-bit) * Yield Flag (1-bit software hint: NVidia CU will select new warp, load-balancing the SMT resources of the compute unit) Itanium's idea of VLIW was commingled with other ideas; in particular, the idea of a compiler static-scheduler to minimize hardware work at runtime. To my eyes: the benefits of Itanium are implemented in NVidia's GPUs. The…

Paper and discussion history. A mention is not an endorsement.

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking — mentioned by dragontamer

Mon, 29 Jul 2019 17:47:24 GMT

> Since you mentioned this in the same breath as GPUs, I feel I have to point out that according to some reverse engineering paper, Nvidia's Turing is a VLIW architecture. (I'm talking about the actual hardware here, not PTX.) NVidia Volta / Turing has been reverse engineered here: https://arxiv.org/abs/1804.06826 . EDIT: I'm talking about the actual hardware, the SASS assembly, not PTX. It doesn't look like a VLIW architecture to me. Page 14 for the specific instruction-set details. I realize this is mostly a matter of opinion, but... those control-codes are very different from the VLIW that…

Paper and discussion history. A mention is not an endorsement.

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking — mentioned by dragontamer

Tue, 26 Mar 2019 22:08:17 GMT

> Incidentally PTX is a fiction - more of an IR than something with true throughput and latency numbers. I don't know if anyone programs GPUs directly but that would be interesting. PTX is a fiction, but its very close to the underlying "SASS" assembly language of NVidia GPUs. There was some research done into this field here: https://arxiv.org/abs/1804.06826 They reverse engineered the SASS assembly language, all the op-codes match up to PTX opcodes. The main difference is that SASS includes control information, such as stall cycles, write or read barrier information. In effect, it seems…

Paper and discussion history. A mention is not an endorsement.

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking — HN submission

Sat, 21 Apr 2018 00:43:16 GMT

Dissecting the NVIDIA Volta GPU Architecture via Microbenchmarking

111 points; 13 discussion replies on HN. Paper and source history. A submission is not an endorsement.