ReLoRA: High-Rank Training Through Low-Rank Updates · HackerPapers discussions

ReLoRA: High-Rank Training Through Low-Rank Updates — mentioned by mountainriver

Sat, 04 Oct 2025 15:36:09 GMT

Don’t forget ReLoRA! https://arxiv.org/abs/2307.05695

Paper and discussion history. A mention is not an endorsement.

ReLoRA: High-Rank Training Through Low-Rank Updates — mentioned by cabidaher

Fri, 17 May 2024 14:56:54 GMT

This paper [1] does atempt that and reports similar performance compared to conventional pre-training. However, they do start off by doing a normal full-rank training and claim that it is needed to 'warm start' the training process. [1] https://arxiv.org/abs/2307.05695

Paper and discussion history. A mention is not an endorsement.

ReLoRA: High-Rank Training Through Low-Rank Updates — HN submission

Thu, 21 Dec 2023 14:09:38 GMT

ReLoRA: High-Rank Training Through Low-Rank Updates

3 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.