SDXL LoRA DreamBooth - DKTech/dreambooth-test-1

Prompt

A photo of sks dog in a bucket

Prompt

A photo of sks dog in a bucket

Prompt

A photo of sks dog in a bucket

Prompt

A photo of sks dog in a bucket

Model description

These are DKTech/dreambooth-test-1 LoRA adaption weights for stabilityai/stable-diffusion-xl-base-1.0.

The weights were trained using DreamBooth.

LoRA for the text encoder was enabled: False.

Special VAE used for training: madebyollin/sdxl-vae-fp16-fix.

Trigger words

You should use a photo of sks dog to trigger the image generation.

Download model

Weights for this model are available in Safetensors format.

Download them in the Files & versions tab.

Intended uses & limitations

How to use

Set up the environment on command-line / terminal.

# Create and activate conda environment
conda create –name dreambooth python=3.10
conda activate dreambooth

# Install ipykernel (needed only if you want to run the inference inside a jupyter-notebook)
conda install -c anaconda ipykernel
python -m ipykernel install --user --name=dreambooth

# Clone and install diffusers package
git clone https://github.com/huggingface/diffusers
cd diffusers
pip install -e .

# Browse to examples/dreambooth directory in the diffusers installation directory
cd examples/dreambooth

# Install dreambooth sdxl training dependencies
pip install -r requirements_sdxl.txt

Run the inference in Python.

from huggingface_hub.repocard import RepoCard
from diffusers import DiffusionPipeline
import torch

lora_model_id = "DKTech/dreambooth-test-1"
card = RepoCard.load(lora_model_id)
base_model_id = card.data.to_dict()["base_model"]

pipe = DiffusionPipeline.from_pretrained(base_model_id, torch_dtype=torch.float16)
pipe = pipe.to("cuda")
pipe.load_lora_weights(lora_model_id)
image = pipe("A picture of an elephant that looks like a dog.", num_inference_steps=25).images[0]
image.save("my_image.png")

Fine tuning the original model

This model was created by fine tuning the original stable diffusion model based on the instructions here- https://github.com/huggingface/diffusers/blob/main/examples/dreambooth/README_sdxl.md Various other base models (other than stable diffusion) can also be fine tuned using DreamBooth. For example, some discussion on fine tuning Playground 2.5 model can be found here- https://github.com/huggingface/diffusers/pull/7126

Limitations and bias

[TODO: provide examples of latent issues and potential remediations]

Training details

[TODO: describe the data used to train the model]