HackerPapers

arXiv:2106.03253cs.LG

Tabular Data: Deep Learning is Not All You Need

Ravid Shwartz-Ziv, Amitai Armon

Abstract

A key element in solving real-life data science problems is selecting the types of models to use. Tree ensemble models (such as XGBoost) are usually recommended for classification and regression problems with tabular data. However, several deep learning models for tabular data have recently been proposed, claiming to outperform XGBoost for some use cases. This paper explores whether these deep models should be a recommended option for tabular data by rigorously comparing the new deep models to XGBoost on various datasets. In addition to systematically comparing their performance, we consider the tuning and computation they require. Our study shows that XGBoost outperforms these deep models across the datasets, including the datasets used in the papers that proposed the deep models. We also demonstrate that XGBoost requires much less tuning. On the positive side, we show that an ensemble of deep models and XGBoost performs better on these datasets than XGBoost alone.

arXiv comments
Journal reference
DOI
Subjects
cs.LGMachine Learning

On Hacker News

Linked by 4 identifiable commenters in 4 comments across 4 threads, from Jun 2021 to Jul 2024. Found in 1 HN submission from Jun 2021 to Jun 2021. Open submission discussions.

Comments linking this paper, per year.

Discussed in comments4 threads

  1. Applied Machine Learning for Tabular Data 1 comment

    • Like others say, usually just go for XGBoost, which is increasingly even shown (proven?) in literature, eg: https://arxiv.org/abs/2106.03253 You can also start with scikit-learn's `LogisticRegressionCV`, if a linear model is more palatable to your audience. The bigger challenge…

      antipaulView on HNParent context

  2. My Python code is a neural network 1 comment

    • This area is covered by non-parametric statistics more generally. There are many other methods to non-parametrically estimate functions (that satisfy some regularity conditions). Tree-based methods are one family of such methods, and the consensus still seems to be that they…

      gnyekiView on HNParent context

  3. Act-1: Transformer for Actions 1 comment

    • Wow! Love it, this is the most exciting thing I've seen in a while. I'm working on something similar, and it's so great to see others who seem to get-it and are chasing generalization in AI systems! A few questions: 1. I'm curious if you're representing the task-operations using…

      bluecoconutView on HNParent context

  4. Regularization is all you need: simple neural nets can excel on tabular data 1 comment

    • This paper compares xgboost vs nueural nets and an ensemble [Tabular Data: Deep Learning is Not All You Need] https://arxiv.org/abs/2106.03253

      audi0slaveView on HNParent context

Submitted to Hacker News

Connected through shared HN threads, not inferred similarity or a recommendation.