Matryoshka Representation Learning · HackerPapers discussions

Matryoshka Representation Learning — mentioned by jsrozner

Sun, 09 Aug 2026 17:44:20 GMT

The original MRL paper (https://arxiv.org/pdf/2205.13147) reported an SVD baseline, which showed comparable performance (Table 1, top-1 accuracy) at d>=256, but much degraded performance at lower dims (d \in {8,32,64}). (Though Table 2, nearest-neighbor accuracy, doesn't show degradation until d <= 16.) In your conclusion, you report that PCA won on most dimensions. Did you investigate why you found that PCA outperforms MRL when the original paper found that their SVD baseline did not?

Paper and discussion history. A mention is not an endorsement.

Matryoshka Representation Learning — HN submission

Thu, 31 Jul 2025 06:50:35 GMT

Matryoshka Representation Learning

6 points; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Matryoshka Representation Learning — mentioned by idontknowmuch

Tue, 06 Aug 2024 23:29:38 GMT

Ahh that's an interesting paper I must of missed that one - thanks for the link. I think another paper that recently got a lot of hype has been the Matroyshka representation learning paper -- essentially training models with different parameters and output embedding sizes at the same time, basically distillation during training rather than post-training (https://arxiv.org/abs/2205.13147).

Paper and discussion history. A mention is not an endorsement.

Matryoshka Representation Learning — HN submission

Thu, 01 Feb 2024 05:12:32 GMT

Matryoshka Representation Learning

83 points; 11 discussion replies on HN. Paper and source history. A submission is not an endorsement.