Skip to content
Starfield Labs

Tutorials

Stable Diffusion prompt tutorial: Basics of prompt engineering for Stable Diffusion

AI hackathon guide to Stable Diffusion prompts: raw prompts, styles, artist names and finishing touches for better images.

By Starfield LabsPublished Intermediate55 min

Cover card for the tutorial "Stable Diffusion prompt tutorial: Basics of prompt engineering for Stable Diffusion"

What is Prompt Engineering?

Prompt engineering is a concept in artificial intelligence, particularly natural language processing (NLP). In prompt engineering, the description of the task is embedded in the input. The output of prompts might differ, regarding tool and model you choose. For example, different results you will get from different Stable Diffusion models.

Raw prompts?

A raw prompt is a very basic description of what you want to generate. Most people start by using only raw prompts. It is a common mistake because these images tend to get random and chaotic. The results are OK but but we can do better :) As an example I generated a tiger, a skyscraper and a watch with Stable Diffusion. With raw prompts the images are unremarkable, much like a stock image search result :)

Adding style to our prompt

The style of the prompt is very important. If the AI is missing a specific style, it will usually choose the one it has seen the most in similar images. Having a well-chosen style and raw prompt is sometimes enough, as the style influences the image the most right after the raw prompt.

The most commonly used styles include:

  1. Realistic
  2. Oil painting
  3. Pencil drawing
  4. Concept art
  5. digital art
  6. Cartoon
  7. Anime
  8. Manga
  9. Sketch
  10. Watercolor
  11. Abstract
  12. Surrealism
  13. Pop art
  14. Futurism

The result tends to improve when you add a style; a tiger as a pencil drawing is a good example.

In the case of a realistic image, there are various ways of making it look realistic. Here are some commonly used techniques of making the image realistic:

  1. photo of + raw prompt
  2. photograph of + raw prompt
  3. raw prompt, hyperrealistic
  4. raw prompt, realistic

If you want your prompt to generate an oil painting, you can add "an oil painting of" to your prompt. This may sometimes result in the image showing an oil painting in a frame. If this happens, you can just re-run the prompt or use raw prompt + "oil painting."

To create a pencil drawing, add "a pencil drawing of" to your raw prompt, or make your prompt raw prompt + "pencil drawing."

Adding an artist to your prompt

To make your style more specific, you can use artists’ names in your prompt. For instance, if you want a very abstract image, you can add “made by Pablo Picasso” or just simply, “Picasso”. Below are lists of artists in different styles that you can use, but I always encourage you to search for different artists as it is a cool way of discovering new art.

Portrait

John Singer Sargent, Edgar Degas, Paul Cézanne, Jan van Eyck

Oil painting

Leonardo DaVinci, Vincent Van Gogh, Johannes Vermeer, Rembrandt

Pencil/Pen drawing

Albrecht Dürer, Leonardo da Vinci, Michelangelo, Jean-Auguste-Dominique Ingres

Landscape art

Thomas Moran, Claude Monet, Alfred Bierstadt, Frederic Edwin Church

Finishing touches

Some final things you can add to your prompt to make it look more like what you want include adding "trending on artstation" for a more artistic image, or "Unreal Engine" for more realistic lighting. You can add anything you want, but these are some examples: Highly detailed, surrealism, trending on art station, triadic color scheme, smooth, sharp focus, matte, elegant, the most beautiful image ever seen, illustration, digital paint, dark, gloomy, octane render, 8k, 4k, washed colors, sharp, dramatic lighting, beautiful, post processing, picture of the day, ambient lighting, epic composition, bokeh, 15mm wide lens, 35mm wide lens, 85mm wide lens.

Another example prompt: "tiger toy made of yarn"

More resources

For creating samples I used Stable Diffusion model on https://beta.dreamstudio.ai to search art with prompts https://lexica.art/.

Final words

Hope you enjoyed this simple guide!

Build it at an AI hackathon

Image generation is a popular choice for an AI hackathon project, and a well-built prompt is an inexpensive way to improve the result. The steps here, a raw prompt, a style, an artist and finishing touches, are quick to try and quick to compare. To find an event to build in, see the list of upcoming AI hackathons.

Frequently asked questions

What is a raw prompt?

A raw prompt is a very basic description of what you want to generate. Images made from raw prompts alone tend to get random and chaotic.

How do I make an image look realistic?

Commonly used techniques are "photo of" or "photograph of" plus the raw prompt, or the raw prompt followed by "hyperrealistic" or "realistic".

How do I make a style more specific?

Add an artist's name to the prompt, for example "made by Pablo Picasso" or just "Picasso" for a very abstract image.

How this tutorial was tested

This content was carried over from an earlier tutorial and updated for Starfield Labs. It was not re-run in this build. None of the steps, code or commands was executed, and no output was checked. Running it needs: access to a Stable Diffusion tool such as DreamStudio, which may need an account and paid credits. None of these was used here, so versions, prices, model names and links may have changed. Check each one before you start.