PiCa: Parameter-Efficient Fine-Tuning with Column Space Projection — mentioned by cheald

Sun, 05 Oct 2025 07:39:18 GMT

The one I was referring to was from this paper, first published in May: https://arxiv.org/abs/2505.20211v1 I don't recall how I found out about it, but it was either paperswithcode or an LLM research session working through the intruder dimensions problem. In my Stable Diffusion tests, it substantially improves LoRA training speed and fidelity, though I've got some experiments that seem to even further substantially improve on it by adding learnable rotations of the singular vectors.

Paper and discussion history. A mention is not an endorsement.

PiCa: Parameter-Efficient Fine-Tuning with Column Space Projection · HackerPapers discussions

PiCa: Parameter-Efficient Fine-Tuning with Column Space Projection — mentioned by cheald