HackerPapers

arXiv:1906.00091cs.IR

Deep Learning Recommendation Model for Personalization and Recommendation Systems

Maxim Naumov, Dheevatsa Mudigere, Hao-Jun Michael Shi, Jianyu Huang, Narayanan Sundaraman, Jongsoo Park, Xiaodong Wang, Udit Gupta, Carole-Jean Wu, Alisson G. Azzolini, Dmytro Dzhulgakov, Andrey Mallevich, Ilia Cherniavskii, Yinghai Lu, Raghuraman Krishnamoorthi, Ansha Yu, Volodymyr Kondratenko, Stephanie Pereira, Xianjie Chen, Wenlin Chen, Vijay Rao, Bill Jia, Liang Xiong, Misha Smelyanskiy

Abstract

With the advent of deep learning, neural network-based recommendation models have emerged as an important tool for tackling personalization and recommendation tasks. These networks differ significantly from other deep learning networks due to their need to handle categorical features and are not well studied or understood. In this paper, we develop a state-of-the-art deep learning recommendation model (DLRM) and provide its implementation in both PyTorch and Caffe2 frameworks. In addition, we design a specialized parallelization scheme utilizing model parallelism on the embedding tables to mitigate memory constraints while exploiting data parallelism to scale-out compute from the fully-connected layers. We compare DLRM against existing recommendation models and characterize its performance on the Big Basin AI platform, demonstrating its usefulness as a benchmark for future algorithmic experimentation and system co-design.

arXiv comments
10 pages, 6 figures
Journal reference
DOI
Subjects
cs.IRInformation Retrieval cs.LGMachine Learning

On Hacker News

Linked by 3 identifiable commenters in 4 comments across 3 threads, from Mar 2020 to Mar 2025.

Comments linking this paper, per year.

Discussed in comments3 threads

  1. Strobelight: A profiling service built on open source technology 1 comment

    • > You’ll note that nobody is open sourcing their ad recommender Actually... (2019) https://ai.meta.com/blog/dlrm-an-advanced-open-source-deep-l... Source code: https://github.com/facebookresearch/dlrm Paper: https://arxiv.org/abs/1906.00091 Updated 2023 blog post, but solely for…

      ipsum2View on HNParent context

  2. Building Meta's GenAI infrastructure 1 comment

    • I think it’s always useful to pay attention to the history on stuff like this and it’s a rare pleasure to be able to give some pointers in the literature along with some color to those interested from first-hand experience. I’d point the interested at the DLRM paper [1]: that…

      benreesmanView on HNParent context

  3. Smart Algorithms beat Hardware Acceleration for Large-Scale Deep Learning 2 comments

    • The reverse is true, embeddings are both the performance and memory-footprint bottleneck of modern NN models. Check figure 6. of : https://arxiv.org/pdf/1906.00091.pdf Embeddings are used to lookup sparse features, so you have those pesky data-dependent lookups.

      yvdriessView on HNParent context

    • So you agree that the problem is fundamentally sparse? Embeddings are used to make sparse (e.g. categorical) data possible on GPUs, and real-world models are limited by how large they can make the embeddings to fit in GPU memory. Embedding lookups is also a compute bottleneck…

      yvdriessView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.