0
0
stable diffusion clip aesthetic – Aesthetic gradients are proposed as a means to personalize a CLIP-conditioned diffusion model by steering the generating process towards specific aesthetics defined by the user from a group of photos.
Aesthetic gradients effectively interpose the aesthetic embedding provided by the user (i.e., by their contributed photos and single-word definition) in the conventional prompt>CLIP>noise>image process.
The donated pictures are ‘averaged out’ in the pipeline before being normalized to the typical Stable Diffusion text2img process’s ‘unitary norm’ – supplementing rather than replacing it.
stable diffusion clip aesthetic-1
prompt: intensive girl, Redhead curly hair, detailed intricate environment, winter dawn, detailed, intricate, [by loish and rossdraws and artgerm and Kamal Rao:0.1]
Negative prompt: 3d, (disfigured), (bad art), (deformed), (poorly drawn), (extra limbs), strange colours, blurry, boring, sketch, lacklustre, repetitive, cropped, hands,text,watermark,signature,tiled
Steps: 20, Sampler: DPM++ 2S a, CFG scale: 7, Seed: 1290542316, Size: 768×832, Model hash: 2c02b20a, Model: sd-v2.0-768-v-ema, Batch size: 4, Batch pos: 0
stable diffusion clip aesthetic-2
- prompt: “Infinity pool with a tropical forest in the background, high resolution, detail, 8 k, DSLR, good lighting, ray tracing, realistic”
- size: 512×512
- guidance scale: 12
- steps: 50
- sampler: DDIM
stable diffusion clip aesthetic-3
We notice some intriguing effects when we add the same negative prompt as in the previous case. The negative cue, in particular, appears to be detrimental to SD 1 but uniformly beneficial to SD 2.
Each image from SD 2 improves with negative prompting, however, the caption alignment for SD 1 appears to decrease generally. Adding the negative prompt appears to push the generated images closer to photorealism.
- prompt: “Roman city on top of a ridge, sci-fi illustration by Greg Rutkowski #sci-fi detailed vivid colors gothic concept illustration by James Gurney and Zdzislaw Beksiński vivid vivid colorsg concept illustration colorful interior”
- size: 512×512
- guidance scale: 12
- steps: 50
- sampler: DDIM
stable diffusion clip aesthetic-4
- prompt: “Roman city on top of a ridge, sci-fi illustration by Greg Rutkowski #sci-fi detailed vivid colors gothic concept illustration by James Gurney and Zdzislaw Beksiński vivid vivid colorsg concept illustration colorful interior”
- size: 512×512
- guidance scale: 12
- steps: 50
- sampler: DDIM
stable diffusion clip aesthetic-5
- prompt: “a gothic cathedral in a stunning landscape by Jean-Honoré Fragonard”
- size: 512×512
- guidance scale: 12
- steps: 50
- sampler: DDIM
stable diffusion clip aesthetic-6
- prompt: “Cyberpunk ikea, close up shot from the top, anime art, greg rutkowski, studio ghibli, dramatic lighting”
- size: 512×512
- guidance scale: 12
- steps: 50
- sampler: DDIM
stable diffusion clip aesthetic-7
- prompt: “A studio photograph of Robert Downey Jr., cinematic lighting, hyperdetailed, 8 k realistic, global illumination, radiant light, frostbite 3 engine, CryEngine, trending on artstation, digital art”
- size: 512×512
- guidance scale: 7
- steps: 50
- seed: 119-121
stable diffusion clip aesthetic-8
- prompt: “A monster fighting a hero by greg rutkowski, romanticism, cinematic lighting, hyperdetailed, 8 k realistic, global illumination, radiant light, trending on artstation, digital art”
- size: 512×512
- guidance scale: 9
- steps: 50
- seed: 119-122
stable diffusion clip aesthetic-9
PROMPT: digital painting of a (young) (woman) (medieval knight) portrait, beautiful eyes, forest in the background, intricate details, concept art, [art by ross tran and artgerm:0.1] [and leyendecker:0.2]
