<story><title>Stable Diffusion 2.0</title><url>https://stability.ai/blog/stable-diffusion-v2-release</url></story><parent_chain><item><author>kmeisthax</author><text>Is there a good explanation of how to train this from scratch with a custom dataset[0]?&lt;p&gt;I&amp;#x27;ve been looking around the documentation on Huggingface, but all I could find was either how to train unconditional U-Nets[1], or how to use the pretrained Stable Diffusion model to process image prompts (which I already know how to do). Writing a training loop for CLIP manually wound up with me banging against all sorts of strange roadblocks and missing bits of documentation, and I still don&amp;#x27;t have it working. I&amp;#x27;m pretty sure I also need some other trainables at some point, too.&lt;p&gt;[0] Specifically, Wikimedia Commons images in the PD-Art-100 category, because the images will be public domain in the US and the labels CC-BY-SA. This would rule out a lot of the complaints people have about living artists&amp;#x27; work getting scraped into the machine; and probably satisfy Debian&amp;#x27;s ML guidelines.&lt;p&gt;[1] Which actually &lt;i&gt;does work&lt;/i&gt;</text></item></parent_chain><comment><author>TaylorAlexander</author><text>Ah I am glad to see someone else talking about using public domain images!&lt;p&gt;Honestly it baffles me that in all this discussion, I rarely see people discussing how to do this with appropriately licensed images. There are some pretty large datasets out there of public images, and doing so might even help encourage more people to contribute to open datasets.&lt;p&gt;Also if the big ML companies HAD to use open images, they would be forced to figure out sample efficiency for these models. Which is good for the ML community! They would also be motivated to encourage the creation of larger openly licensed datasets, which would be great. I still think if we got twitter and other social media sites to add image license options, then people who want to contribute to open datasets could do so in an easy and socially contagious way. Maybe this would be a good project for mastodon contributors, since that is something we actually have control over. I&amp;#x27;d be happy to license my photography with an open license!&lt;p&gt;It is really a wonderful idea to try to do this with open data. Maybe it won&amp;#x27;t work very well with current techniques, but that just becomes an engineering problem worth looking at (sample efficiency).</text></comment>
<story><title>Stable Diffusion 2.0</title><url>https://stability.ai/blog/stable-diffusion-v2-release</url></story><parent_chain><item><author>kmeisthax</author><text>Is there a good explanation of how to train this from scratch with a custom dataset[0]?&lt;p&gt;I&amp;#x27;ve been looking around the documentation on Huggingface, but all I could find was either how to train unconditional U-Nets[1], or how to use the pretrained Stable Diffusion model to process image prompts (which I already know how to do). Writing a training loop for CLIP manually wound up with me banging against all sorts of strange roadblocks and missing bits of documentation, and I still don&amp;#x27;t have it working. I&amp;#x27;m pretty sure I also need some other trainables at some point, too.&lt;p&gt;[0] Specifically, Wikimedia Commons images in the PD-Art-100 category, because the images will be public domain in the US and the labels CC-BY-SA. This would rule out a lot of the complaints people have about living artists&amp;#x27; work getting scraped into the machine; and probably satisfy Debian&amp;#x27;s ML guidelines.&lt;p&gt;[1] Which actually &lt;i&gt;does work&lt;/i&gt;</text></item></parent_chain><comment><author>echelon</author><text>&amp;gt; train this from scratch&lt;p&gt;If you&amp;#x27;re talking about training from scratch and not fine tuning, that won&amp;#x27;t be cheap or easy to do. You need thousands upon thousands of dollars of GPU compute [1] and a gigantic data set.&lt;p&gt;I trained something nowhere near the scale of Stable Diffusion on Lambda Labs, and my bill was $14,000.&lt;p&gt;[1] Assuming you rent GPUs hourly, because buying the hardware outright will be prohibitively expensive.</text></comment>