Bag of Tricks for Efficient Text Classification · HackerPapers discussions

Bag of Tricks for Efficient Text Classification — mentioned by braza

Wed, 26 Jun 2024 13:34:02 GMT

A lot of great takes in this thread, but let me give more perspective being a MLE/MLOps Engineer myself around the math. For me mathematics and statistics represent (a) unemployment insurance since it give me a lot of transitivity around roles in the space,(b) give me a good toolbox to talk as an equal with DS, and (c) for all implementation made by the Research Engineers/Data Scientists I can chime in and give insights and avoid waste of time and resources. One example: When BERT was released (ca. 2018) I was working in a place where several Research Engineers and DSs wanted to use it in…

Paper and discussion history. A mention is not an endorsement.

Bag of Tricks for Efficient Text Classification — mentioned by wenc

Fri, 02 Mar 2018 14:50:40 GMT

The cognoscenti already know this, but word vectors are a game changer in the multi-class text classification game. Finding the correct representation of text makes the classification task so much easier. For my problem (1000+ total classes, 1 class per input), I experimented with Naive Bayes + TFIDF (~50% accuracy, < 1 sec training), then Word2vec + CNN model on GPU (~70% accuracy, 6 hrs training), and finally FastText (99% accuracy, 10 minutes training). FastText [0] in particular is quite impressive. It is essentially a variant of Word2Vec that also supports n-grams, and there is a default…

Paper and discussion history. A mention is not an endorsement.

Bag of Tricks for Efficient Text Classification — mentioned by pasbesoin

Sun, 21 Aug 2016 23:03:31 GMT

That FB hand-off page has an annoying auto refresh with NoScript enabled. So, here's the PDF it points to. I guess I'll change it from linking the PDF document to a link to its abstract page, while I'm at it. http://arxiv.org/abs/1607.01759v3

Paper and discussion history. A mention is not an endorsement.

Bag of Tricks for Efficient Text Classification — mentioned by kough

Thu, 04 Aug 2016 20:29:27 GMT

Links to the relevant papers: Bag of Tricks for Efficient Text Classification: https://arxiv.org/abs/1607.01759v2 Enriching Word Vectors with Subword Information: https://arxiv.org/abs/1607.04606 Both fantastic papers. For those who aren't aware, Mikolov also helped create word2vec. One curious thing: this seems to use heirarchal softmax instead of the "negative sampling" described in their earlier paper http://arxiv.org/abs/1310.4546, despite that paper reporting that "negative sampling" is more computationally efficient and of similar quality. Anyone know why that might be?

Paper and discussion history. A mention is not an endorsement.

Bag of Tricks for Efficient Text Classification — mentioned by Smerity

Thu, 04 Aug 2016 20:04:20 GMT

Just to mirror what was said on the thread a month ago when the paper came out[1], if you're interested in FastText I'd strongly recommend checking out Vowpal Wabbit[2] and BIDMach[3]. My main issue is that the FastText paper [7] only compares to other intensive deep methods and not to comparable performance focused systems like Vowpal Wabbit or BIDMach. Many of the features implemented in FastText have been existing in Vowpal Wabbit (VW) for many years. Vowpal Wabbit also serves as a test bed for many other interesting, but all highly performant, ideas, and has reasonable strong…

Paper and discussion history. A mention is not an endorsement.

Bag of Tricks for Efficient Text Classification — mentioned by smhx

Thu, 04 Aug 2016 18:57:26 GMT

Link to the paper: https://arxiv.org/abs/1607.01759 Quotes from the paper: Both char-CNN and VDCNN are trained on a NVIDIA Tesla K40 GPU, while our models are trained on a CPU using 20 threads. Table2 shows that methods using convolutions are several orders of magnitude slower than fastText. Our speed-up compared to CNN based methods increases with the size of the dataset, going up to atleast a 15, 000× speed-up. Table 2 shows the speedups of: ConvNets: 2 to 5 days on GPUs FastText: 52 seconds on CPU

Paper and discussion history. A mention is not an endorsement.

Bag of Tricks for Efficient Text Classification — HN submission

Sat, 09 Jul 2016 12:38:49 GMT

Bag of Tricks for Efficient Text Classification

169 points; 15 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Bag of Tricks for Efficient Text Classification — HN submission

Thu, 07 Jul 2016 15:50:41 GMT

FastText: Training text classifiers on 1b words in less than 10 minutes

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.