diffusion / usage.md
adamelliotfields's picture
Add IP-Adapter
61ad3d2 verified
|
raw
history blame
5.14 kB

Usage

Enter a prompt and click Generate.

Prompting

Positive and negative prompts are embedded by Compel for weighting. You can use a float or +/-. For example:

  • man, portrait, blue+ eyes, close-up
  • man, portrait, (blue)1.1 eyes, close-up
  • man, portrait, (blue eyes)-, close-up
  • man, portrait, (blue eyes)0.9, close-up

Note that ++ is 1.1^2 (and so on). See syntax features to learn more and read Civitai's guide on prompting for best practices.

You can also press the 🎲 button to generate a random prompt.

Arrays

Arrays allow you to generate different images from a single prompt. For example, [[cat,corgi]] will expand into 2 separate prompts. Make sure Images is set accordingly (e.g., 2). Only works for the positive prompt. Inspired by Fooocus.

Embeddings

Select multiple negative textual inversion embeddings. Fast Negative and Bad Dream can be used standalone or together; Unrealistic Dream should be combined with one of the others:

Styles

Styles are prompt templates from twri's sdxl_prompt_styler Comfy node. Start with a subject like "cat", pick a style, and iterate from there.

Scale

Rescale up to 4x using Real-ESRGAN (Wang et al. 2021).

Models

Each model checkpoint has a different aesthetic:

Schedulers

The default is DEIS 2M with Karras enabled. The other multistep scheduler, DPM++ 2M, is also good. For realism, DDIM is recommended. Euler a is worth trying for a different look.

Image-to-Image

The 🖼️ Image tab enables the image-to-image and IP-Adapter pipelines. Either use the image input or select a generation from the gallery. To disable, simply clear the image input (the x overlay button).

Denoising strength is essentially how much the generation will differ from the input image. A value of 0 will be identical to the original, while 1 will be a completely new image. You may want to also increase the number of inference steps. Only applies to the image-to-image input.

IP-Adapter

In an image-to-image pipeline, the input image is used as the initial latent. With IP-Adapter (Ye et al. 2023), the input image is processed by a separate image encoder and the encoded features are used as conditioning along with the text prompt.

For capturing faces, enable IP-Adapter Face to use the full-face model. You should use an input image that is mostly a face along with the Realistic Vision model. The input image should also be the same aspect ratio as the output to avoid distortion.

Advanced

DeepCache

DeepCache (Ma et al. 2023) caches lower UNet layers and reuses them every Interval steps:

  • 1: no caching
  • 2: more quality (default)
  • 3: balanced
  • 4: more speed

FreeU

FreeU (Si et al. 2023) re-weights the contributions sourced from the UNet’s skip connections and backbone feature maps to potentially improve image quality.

Clip Skip

When enabled, the last CLIP layer is skipped. This can sometimes improve image quality with anime models.

Tiny VAE

Enable madebyollin/taesd for near-instant latent decoding with a minor loss in detail. Useful for development.

Prompt Truncation

When enabled, prompts will be truncated to CLIP's limit of 77 tokens. By default this is disabled, so Compel will chunk prompts into segments rather than cutting them off.