How to paint with stable diffusion

Stable Diffusion is one of the leading AI text-to-image generation tools available online. Developed by Stability.ai, the software is a revolutionary image translator. It uses a latent diffusion model to render some of the fastest images based on text cues.

Among its many functions, exterior paint is one of the most interesting to use. Helps expand an existing image in one or more directions based on text descriptions, sampling methods, and resolution settings. And unlike purely AI-generated images, the exterior painting feature gives artists and casual Stable Diffusion users more control over the final product.

Creating your image using prompts

Before tackling painting in Stable Diffusion, you need to generate an image. This is how to use the Automatic1111 GUI interface to create a new image.

1. Go to the txt2img tab.
   
2. Type the words that describe your image in the pop-up text box.
   
3. Type words that describe graphic elements that you do not want to see in the negative text box.
   
4. Set the sampling steps between 50 and 55, depending on the capabilities of your graphics card.
   
5. Use the sampling method you want and set the width and height of the image.
   
6. Check the “Restore Faces” option if you are generating people in the image.
   
7. Adjust the “Batch Count” slider if you want to render more than one image.
   
8. Set the seed to “-1” unless you have a specific image seed in mind that you want to recreate.
   
9. Click the “Generate” button.
   

Please note that the words you type in the message text box are very important. If you want specific strokes, styles, colors, and other effects, you should mention them in that section.

The default Stable Diffusion imager will generate something photographic-feeling unless otherwise noted.

Using the Outpaint function in stable diffusion

Once you render an image, you can start expanding or painting it. Select the image you like best from your batch and click the “Send to img2img” button.

Follow the steps below to configure Stable diffusion to paint.

1. Go to the img2img tab in the Automatic1111 GUI interface.
   
2. Scroll down until you see the “Script” drop-down menu and select “Outpainting mk2.”
   
3. Choose the painting direction by checking the “Left”, “Right”, “Up” or “Down” boxes.
   
4. Describe the visual elements of the exterior paint in the pop-up text box at the top of the user interface.
   
5. Check the “Just resize” option.
   
6. Set the sampling steps between 80 and 100 using the slider.
   
7. Adjust the number of pixels, which you can often maximize for maximum image quality.
   
8. Click the “Generate” button.
   

The best way to set up the painting process

One of the biggest problems with exterior painting in Stable Diffusion and other AI-enabled text-to-image programs is AI overhead. For example, expanding an image in more than one direction can produce mixed results.

Also, if only one side looks good, you won’t be able to keep it and repaint in the other directions. Instead, you must start from scratch. That means you may not be able to recreate that side that seemed perfect.

Here’s what you should do to get the best exterior painting results in Stable diffusion.

Expand left

1. Upload your image to the img2img tab.
   
2. Describe the visual elements you want in the exterior paint.
   
3. Add negative messages if necessary.
   
4. Choose the “Left” exterior paint direction and uncheck the other three options.
   
5. Click “Generate” until you are satisfied with the result.
   
6. Drag the image from the right panel of the user interface to the img2img tab.

Expand to the right

1. Describe the visual elements you want and don’t want in the request fields.
   
2. Add negative messages if you want the AI ​​to skip specific details.
   
3. Choose the “Right” exterior painting direction and uncheck the previous one.
   
4. Hit “Generate” until you get something great.
   
5. Drag the image to the img2img tab or press the “Send to img2img” button.
   

Expand up

1. Add new visual elements to the image.
   
2. Add negative messages if necessary.
   
3. Select the exterior painting direction “Up” and uncheck the last one.
   
4. Press “Generate” until you like the result.
   
5. Drag and drop the final image back into the img2img tab.

Expand down

1. Write your final description before generating a top-down expansion.
   
2. Use some negative cues, preferably to avoid overlapping with the original image.
   
3. Select “Down” as the exterior paint direction and uncheck everything else.
   
4. Click the “Generate” button until you are satisfied with the results.
   

Another great benefit of expanding your image in one direction at a time is that you can describe the images better.

For example, let’s say your initial image had the following message: “children playing in the garden.” You could add “dog playing in the garden” on one side and “cat playing in the garden” on the other. The AI ​​will have a much better chance of getting things right and representing realistic animals in the final image.

In the same example, you could write “blue sky and birds” in the message and have the AI ​​fill out the original image instead of getting chunks of sky in all directions.

What you need to master when painting in stable diffusion

Stable diffusion is not cheap. But since it’s Open AI software, you can run it on your own machine to avoid paying a per-generation fee. Keep in mind that you’ll probably need a decent graphics card with around 8GB of VRAM and a good CPU to avoid super slow render times.

Now it’s a matter of installing the Automatic1111 GUI web user interface.

How to Install Stable Diffusion Automatic1111 Web UI on Windows

1. First, install Python 3.11.1since Stable Diffusion relies heavily on Python coding.
   
2. Download and install the appropriate 32 or 64 bit Standalone Git Installer depending on your CPU.
   
3. Run “git clone” https://github.com/AUTOMATIC1111/stable-diffusion-webui.git” to get the correct repository.
   
4. Download the Stable Diffusion model checkpoint named “model.ckpt” from here.
   
5. Add “model.ckpt” in the “models” directory.
   
6. Double click on the “webui-user.bat” executable file.
   
7. Build an image and experiment with exterior paint, interior paint, and other features.

How to Install Stable Diffusion Automatic1111 Web UI on Linux

Installing the Automatic1111 web UI on a Linux machine is slightly different but quite simple. These are the steps you must follow.

Red Hat Linux

1. Guy “sudo dnf install wget git python3“In the terminal.
   
2. Install the UI by running “bash <(wget –q0-https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui/master/webui.sh).”
   

Linux

1. Guy “sudo apt install wget git python3 python3-venv”And run it in the terminal.
   
2. Install the UI with the following command “bash <(wget –q0-https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui/master/webui.sh).”
   

Linux Arch

1. Guy “sudo P –S wget git python3“In the terminal.
   
2. Run “bash <(wget –q0- https://raw.githubusercontent.com/AUTOMATIC1111/stable-diffusion-webui/master/webui.sh)“In the terminal.
   

How to Install Stable Diffusion Automatic1111 Web UI on Apple Silicon

The commands to install the Automatic1111 web UI on an Apple Silicon machine are different from other operating systems.

1. Launch the Terminal app.
   
2. Execute “$ cd ~/Documents/“
   
3. Run “$ curl https://raw.githubusercontent.com/dylancl/stable-diffusion-webui-mps/master/setup_mac.sh -0 setup_mac.sh“
   
4. Execute “$ chmod +x setup_mac.sh“
   
5. Run “./setup_mac.sh“
   
6. Follow the following instructions presented on the terminal interface.

You can run the Automatic1111 UI by running “./run webui_mac.sh“In the terminal. Every time you run it, it will start with the latest updates made to the repository.

What to do after painting a picture

As good as Stable Diffusion AI is, it is far from perfect. Overloading it with extensive information and descriptions can lead to some unexciting images. Likewise, vague descriptions can expand the image without adding enough relevant visual elements.

Once you’re done painting, you have two options. You can save the image and edit it further in your favorite image editor. But another way to take advantage of more Stable Diffusion is to switch to inpainting.

Select your image in the right panel and click the “Send to Inpaint” button. The user interface will change to a new type of editing function. It is a great method for filling in blank spaces in certain parts of the image.

1. Select a brush size and mask a part of the image where you want to add something.
   
2. Describe the elements included in the painting.
   
3. Deselect the “Outpainting mk2” script.
   
4. Click “Generate” until you are satisfied with the result.
   

A much needed feature is finally here

Although Stable Diffusion launched in August 2022, the exterior painting feature didn’t really exist at first. At least its first version wasn’t spectacular. But in a short period of time, Stable Diffusion’s exterior paint has surpassed the capabilities of its interior paint function.

The software can generate stunning images. And with 75-word descriptive texts and a separate field for negative messages, the AI’s accuracy is astonishing.

Now we want to hear from you. What is your experience with Stable Diffusion or similar software so far? Can you usually render satisfactory images or does it take too long? Tell us about your experiences with Stable Diffusion and your painting tips in the comments section below.