Cramming: Training a Language Model on a Single GPU in One Day · HackerPapers discussions

Cramming: Training a Language Model on a Single GPU in One Day — mentioned by lsb

Thu, 14 Aug 2025 19:54:07 GMT

This is evocative of “cramming”, a paper from a few years ago, where the author tried to find the best model they could train for a day on a modern laptop: https://arxiv.org/abs/2212.14034

Paper and discussion history. A mention is not an endorsement.

Cramming: Training a Language Model on a Single GPU in One Day — HN submission

Thu, 21 Nov 2024 23:01:03 GMT

Cramming: Training a Language Model on a Single GPU in One Day

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Cramming: Training a Language Model on a Single GPU in One Day — HN submission

Mon, 08 Apr 2024 10:09:08 GMT

Cramming: Training a Language Model on a Single GPU in One Day (2022)

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Cramming: Training a Language Model on a Single GPU in One Day — HN submission

Sun, 29 Jan 2023 17:40:13 GMT

Training a Language Model on a Single GPU in One Day

2 points; 1 discussion reply on HN. Paper and source history. A submission is not an endorsement.

Cramming: Training a Language Model on a Single GPU in One Day — mentioned by onnodigcomplex

Thu, 19 Jan 2023 09:01:45 GMT

I'm not on my desktop, but I used nanoGPT extensively on my RTX-4090. I trained a GPT2-small but with a small context window (125 > 90M params) at batch sizes of 1536 using gradient accumulation (3*512). This runs at just a smidge over 1 it/s. Some notes - Gradient checkpointing and 2048 batch size in a single go allows ~10% performance improvement on a per sample basis. - torch.compile doesn't work for me yet (lowest cuda version I got my 4090 to run on was 11.8 but highest cuda version on which I got the model to compile is 11.7). - I did the optimalisations in…

Paper and discussion history. A mention is not an endorsement.

Cramming: Training a Language Model on a Single GPU in One Day — mentioned by jph00

Wed, 11 Jan 2023 21:03:35 GMT

A couple of weeks ago a new paper came out that shows how to train a high quality language model on a single GPU in one day. https://arxiv.org/abs/2212.14034

Paper and discussion history. A mention is not an endorsement.

Cramming: Training a Language Model on a Single GPU in One Day — mentioned by homarp

Wed, 11 Jan 2023 13:50:38 GMT

see also 'Cramming: Training a Language Model on a Single GPU in One Day' https://arxiv.org/abs/2212.14034 and https://github.com/JonasGeiping/cramming

Paper and discussion history. A mention is not an endorsement.

Cramming: Training a Language Model on a Single GPU in One Day — HN submission

Wed, 11 Jan 2023 13:47:58 GMT

Cramming: Training a Language Model on a Single GPU in One Day

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Cramming: Training a Language Model on a Single GPU in One Day — HN submission

Tue, 03 Jan 2023 14:57:13 GMT

Cramming: Training a Language Model on a Single GPU in One Day

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Cramming: Training a Language Model on a Single GPU in One Day — HN submission

Thu, 29 Dec 2022 21:44:44 GMT

Cramming: Training a Language Model on a Single GPU in One Day

6 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.