How Negative Prompts Work (with examples)

A still life painting, 716765; Left - no negative prompt, Right - negative prompt: apple

    The aiTransformer Super Stylizer is based on the Stable Diffusion text-to-image model, which can sometimes generate images with unwanted features like extra limbs, fingers, or certain objects. Negative prompts are introduced to help to correct this.To effectively use negative prompts, it is necessary to understand what they are and how they work.

     What is a negative prompt and how it works?

    A negative prompt tells AI what you don't want in images, it serves as a guide for AI in determining what should not be present in the generated images. 

     Why can't we simply use some negative words such as "without" or "no" in prompts to specify what should not be included in the generated images?  To better understand, let's take "a still life painting" as an example before exploring further.

a still life painting

      Using the prompt "a still life painting", I get 10 still life paintings with various fruits, but as an apple hater, I don't want any apples in my pictures. So I input a new prompt "a still life painting without apple" to try to exclude apples from my pictures.

                              

a still life painting without apple

     Despite requesting "a still life painting without apples", I got 10 images all with apples. The word "without" doesn't work here. As a result, I decide to use the word "no" in the prompt instead, trying the phrase "a still life painting, no apples".

.

a still life painting, no apple

      Similar results, 10 more images all containing apples. The word "no" can't take out apples neither.

      Why does this occur? When negative words like "without" or "no" are included in prompts, AI might have difficulty interpreting the meaning of such prompts or may misinterpret them. It may not understand the connections between words in prompts like "a still life painting without apple" or "a still life, no apple." Instead of omitting apples, it concentrates on "a still life painting" and "apple" and includes apples in all images.

     Negative prompts provide a more direct and precise way for the AI to understand what should not appear in the generated images. Input "apple" in the negative prompt text box, see if it works.

a still life painting, negative prompt: apple

    

     Viola! No apples. I'm finally able to get apple free images. So negative prompts can be a useful tool in directing the AI towards generating images that meet specific requirements and exclude undesirable elements.

     However, it is important to note that negative prompts are not a guarantee to completely eliminate certain elements from the generated images. Rather, they serve as a guide to make AI less likely to include those undesired elements. Despite using negative prompts, unwanted elements may still occur in generated images, but less frequently. Negative prompts should be considered as a tool to steer the model's results, but not as a foolproof method to completely erase certain elements. To illustrate this, let's take another example: "an oil painting of a horse".

    

an oil painting of a horse

      I don't like the frames in two images, so I used "frame" as a negative prompt to get rid of them.

an oil painting of a horse; with negative prompt: frame

     I still get a picture with frame. Why don't negative prompts work consistently? Because the language model is trained on both positive and negative examples, allowing it to recognize and respond to negative prompts. It may not have been trained sufficiently on certain negative prompts, leading to inaccuracies in its responses.