metadata

base_model:
  - stabilityai/stable-diffusion-xl-base-1.0
pipeline_tag: text-to-image
tags:
  - art

The Superposition of Diffusion Models Using the Itô Density Estimator: Pipeline

This pipeline shows how to superimpose different text prompts from Stable Diffusion-XL 1.0 based the paper The Superposition of Diffusion Models Using the Itô Density Estimator. The authors would like to thank Viktor Ohanesian for developing the SD-XL pipeline.

drawing

Requirements

This pipeline can be run with the following packages & versions:

PyTorch 2.5.1
Diffusers 0.32.1
Accelerate 1.2.1
Transformers 4.47.1

You can install these with:

pip install torch
pip install diffusers accelerate transformers

Example usage

from PIL import Image
from diffusers import DiffusionPipeline

pipeline = DiffusionPipeline.from_pretrained("stabilityai/stable-diffusion-xl-base-1.0", custom_pipeline="superdiff/superdiff-sdxl-v1-0")
output = pipeline("a flamingo", "a candy cane", seed=1, num_inference_steps=200, batch_size=1)

image = Image.fromarray(output[0])
image.save("superdiff_output.png")

Arguments that can be set by user in pipeline():

prompt_1 [required]: text prompt describing first concept to superimpose (e.g. "a flamingo")
prompt_2[required]: text prompt describing second concept to superimpose (e.g. "a candy cane")
seed[optional: default=None]: seed for random noise generator for reproducibility; for non-deterministic outputs, set to None
num_inference_steps[optional: default=200]: number of denoising steps
batch_size [optional: default=1]: batch size
guidance_scale [optional: default=7.5]: scale for classifier-free guidance
height, width [optional: default=512]: height and width of generated images

Citation

BibTeX:

@article{skreta2024superposition,
  title={The Superposition of Diffusion Models Using the It$\backslash$\^{} o Density Estimator},
  author={Skreta, Marta and Atanackovic, Lazar and Bose, Avishek Joey and Tong, Alexander and Neklyudov, Kirill},
  journal={arXiv preprint arXiv:2412.17762},
  year={2024}
}