Robust fine-tuning of zero-shot models — mentioned by liuliu

Thu, 24 Nov 2022 02:54:17 GMT

Seems the structure of UNet hasn't changed other than the text encoder input (768 to 1024). The biggest change is on the text encoder, switched from ViT-L14 to ViT-H14 and fine-tuned based on https://arxiv.org/pdf/2109.01903.pdf. Seems the 768-v model, if used properly, can substantially speed-up the generation, but not exactly sure yet. Seems straightforward to switch to 512-base model for my app next week.

Paper and discussion history. A mention is not an endorsement.

Robust fine-tuning of zero-shot models · HackerPapers discussions

Robust fine-tuning of zero-shot models — mentioned by liuliu