Best ways for upscaling the results?
I wonder how one can get bigger results without having the computing power for large resolutions. Can you recommend other networks for upscaling?
And some brainstorming ideas:
Do you think, if a pipeline using inpainting/img2img would be interesting?
I tried to scale a result with gimp and then use img2img with low strength to see if I could improve it patch-by-patch thinking about if the differences at the boundaries could be fixed using the inpainting pipeline. The problem here is, that with low strength the input is only slightly changed and the quality does not improve at all and with larger strength the style changes.
Maybe one could upscale the initialization and use patches from the upscaled noise as initialization in the img2img process?
Or to avoid losing details one could try to generate noise, e.g., in Full-HD resolution and then downscale it for the initial image, e.g., in 640x384 resolution. Afterward one can upscale the image and use patches from the upscaled image with patches from the original noise together with the img2img pipeline.
The advantage would be, that it would not only upscale, but could also add finer details to the patches.
Hey, I stumbled across this reddit post, might be interesting to you.
This should also help to significantly reduce memory requirements allowing for larger images:
https://mobile.twitter.com/PatrickPlaten/status/1567185296453779456
- I tried bigger rendering with the attention slicing and it kind of works, but produces no good results. The network seems not to be trained for larger sizes and you get often the things from your prompt several times in your image.
- I also tried inpainting for extrapolating. I generated a 512x512 image, than I increased the canvas size and made a mask covering the empty area and some overlap. The results depend very much (even with low/high strength) on the initialization of the empty area.
- Real-EsrGAN provides nice upscaling. I still have to experiment with using img2img to improve on EsrGAN results.
- The method in the Reddit post looks way too complicated, especially when it is necessary to change the prompt for different patches.
-It may be worth to try (for some prompts) to generate a large image with duplicated objects due to a too large size and then use inpainting to remove the unwanted objects, possibly with another prompt, which does not contain the object descriptions. (e.g. first "A forest with a bird" and then removing all but one bird and inpainting "A forest")