@dvruette , It's me again. A space crashed a week ago. I don't know how to get your attention. Can you please look it up? Many thanks in advance.

Hi Arnon, thanks for the reminder. I'll take a look!

You are back! Have you been on vacation? You haven't replied for a long time. No offence meant, just curious.

Indeed I have! :)

@dvruette , I actually figured this out based on your github activity. You have a month of inactivity about every 14 months or so. I wanted to ask you something, but I forgot the question. Where is the best place to write to you when I'll remember my question, so you can notice? And thanks for keeping space running.

I get notified if you open an issue on any of my projects, so probably best to just pick the one that's most relevant. Here also works of course if it's diffusion/FABRIC related.

@dvruette , I see. Well, good luck... Right, now I remembered. I wanted to ask about the way fabric works. Not the technical details themselves, of course, just a question. I don't really understand how the like/dislike pictures work. Sometimes it feels like they add to the generated image, other times they give the result as some sort of average between themselves and the normal generated image. My theory with negatives seemed to work, and I was about to make another expeiment, but the space have crashed. So which way it actually is? I'm leaning onto additive principle, since it looks like colors, at least, slowly build up, but other details seem to suggest an average principle. I'd like to know, since it would make easier to get actual results out of it.

Arnon2 changed pull request status to closed

Yes, so while the exact answer is rather technical, I think the way to think about it intuitively is that the model tries to generate images "like" all of the positive feedback and "unlike" all of the negative feedback simultaneously. What components are/aren't incorporated can depend a lot on the prompt, the feedback image, the seed, etc. so it can be hard to predict the effect of a given feedback image. From experience, a certain feedback image will generally have a certain effect, and combining multiple ones will generally combine those effects, but it's sometimes hard to predict whether/why a certain image does/doesn't have a certain effect.

@dvruette , I don't really do things intuitively, having a good mathematical base and all, but I've noticed some general consistencies. With dislike images, it's actually easier to figure out, as it's pretty clear that if, for example, you have image of something up close, result will be futher out, and vice versa. However, there seem to be some sort of "overflow" going, when images have a radically reversed effect. I guess working with pretty extreme values does make some things more pronounced in that regard. I think I've figured some possible method that would give consistently good results, but it still needs testing. Thinking about your answer, I've realised, that indeed, the reason I would rely on disliked images is that "liked" images tend to drive result to itself, meaning, that if I want to get better result, I need to dislodge it with appropriate "dislike". Although it doesn't answer, why it wrecks coloration so badly. I guess, this part either works on differnt principle, or uses multiple "palettes", meaning that pictures from different "palettes" combined would give result that look completely white, black, or something else entirely. I wonder what you thinks on that?

Arnon2 changed pull request status to open
Ready to merge
This branch is ready to get merged automatically.

Sign up or log in to comment