MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases · HackerPapers discussions

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases — HN submission

Tue, 08 Oct 2024 12:13:28 GMT

MobileLLM: Optimizing Subbillion Parameter Language Models for OnDevice UseCases

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases — mentioned by imurray

Tue, 09 Jul 2024 15:24:09 GMT

The paper says they tried that: https://arxiv.org/abs/2402.14905 Deep link to the relevant snippet in html version: https://ar5iv.labs.arxiv.org/html/2402.14905#S3.SS5 "So far, we trained compact models from scratch using next tokens as hard labels. We explored Knowledge Distillation (KD)... Unfortunately KD increases training time (slowdown of 2.6−3.2×) and exhibits comparable or inferior accuracy to label-based training (details in appendix)."

Paper and discussion history. A mention is not an endorsement.

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases — HN submission

Sun, 07 Jul 2024 18:24:59 GMT

Optimizing Sub-Billion Parameter Language Models for On-Device Use Cases

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases — HN submission

Sun, 07 Jul 2024 07:53:32 GMT

MobileLLM: Optimizing Sub-Billion Parameter Language Models for On-Device Use

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

MobileLLM: Optimizing Sub-billion Parameter Language Models for On-Device Use Cases — HN submission

Fri, 01 Mar 2024 00:07:48 GMT

Optimizing Sub-Billion Parameter Language Models for On-Device Use Cases

2 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.