A Survey of Multi-Tenant Deep Learning Inference on GPU · HackerPapers discussionshttps://hackerpaper.kaipfister.de/p/2203.09040/Collected HN comments and submissions linking this paper.enMon, 02 Nov 2026 11:16:22 GMTA Survey of Multi-Tenant Deep Learning Inference on GPU — mentioned by eternalbanhttps://news.ycombinator.com/item?id=35776705urn:hackerpapers:mention:2203.09040:35776705Mon, 01 May 2023 18:03:39 GMT<p>A Survey of Multi-Tenant Deep Learning Inference on GPU (2022) https://arxiv.org/pdf/2203.09040.pdf &quot;As one of the most competitive kernel auto-tuning framework, TVM comes with a built-in assumption of single-tenant execution setting, the tuning configuration of which aims to saturate all SMs and memory bandwidth of the GPU. Such assumption and the single-tenant targeted tuning however becomes unsuitable for multi-kernel concurrent execution with partial resource available for each kernel. According to [7], the maximum throughput gap comparing single-tenant vs. multi-tenant tuned…</p><p><a href="https://hackerpaper.kaipfister.de/p/2203.09040/">Paper and discussion history</a>. A mention is not an endorsement.</p>cs.DC