Synthetic Data from Diffusion Models Improves ImageNet Classification · HackerPapers discussions

Synthetic Data from Diffusion Models Improves ImageNet Classification — mentioned by airgapstopgap

Sun, 23 Apr 2023 00:20:09 GMT

We won't hit the wall. Somewhat counterintuitively, scaling datasets is the lazy and economical approach. If you have the compute already, might as well dig an OOM more text tokens. But there are other sources of data, and slightly different ways to utilize it. Multimodality, in very large training runs, will almost inevitably increase sample efficiency (for obvious reasons of context richness), synthetic data is already very effective [1], and there are and will be discovered other ways to do more in the condition of diminishing raw text resources. But a thorough abandonment of the scaling…

Paper and discussion history. A mention is not an endorsement.

Synthetic Data from Diffusion Models Improves ImageNet Classification — mentioned by blackbear_

Sat, 22 Apr 2023 22:04:10 GMT

> Because so far we've never seen an AI improve based on its own output. Maybe it's because AI is such an overloaded term, but this is pretty commonplace for (semi-)supervised learning algorithms. Pseudo-labeling [1,2] is an example of this that has been around for decades. When done properly it does improve the performance of the original model, up to a certain limit (far from the singularity). Moreover, it is apparently possible to improve a model's performance by augmenting it's training set with synthetic examples generated by a second model [3]. Finally, boosting [4] can also be seen as…

Paper and discussion history. A mention is not an endorsement.

Synthetic Data from Diffusion Models Improves ImageNet Classification — HN submission

Tue, 18 Apr 2023 14:15:21 GMT

Synthetic Data from Diffusion Models Improves ImageNet Classification

78 points; 60 discussion replies on HN. Paper and source history. A submission is not an endorsement.

Synthetic Data from Diffusion Models Improves ImageNet Classification — HN submission

Tue, 18 Apr 2023 03:45:59 GMT

Synthetic Data from Diffusion Models Improves ImageNet Classification

1 point; 0 discussion replies on HN. Paper and source history. A submission is not an endorsement.