sdxl-zelda64 LoRA by Julian BILCKE (HF: jbilcke-hf, Replicate: jbilcke)
A SDXL LoRA inspired by Zelda games on Nintendo 64
Inference with Replicate API
Grab your replicate token here
pip install replicate
export REPLICATE_API_TOKEN=r8_*************************************
import replicate
output = replicate.run(
"sdxl-zelda64@sha256:435913219645a80ee6743ca500940ab8708889172ca5c4c71bbb701309bb4a60",
input={"prompt": "Link working as a pizza delivery driver, on a scooter, in new york, in the style of TOK"}
)
print(output)
You may also do inference via the API with Node.js or curl, and locally with COG and Docker, check out the Replicate API page for this model
Inference with 𧨠diffusers
Replicate SDXL LoRAs are trained with Pivotal Tuning, which combines training a concept via Dreambooth LoRA with training a new token with Textual Inversion.
As diffusers
doesn't yet support textual inversion for SDXL, we will use cog-sdxl TokenEmbeddingsHandler
class.
The trigger tokens for your prompt will be <s0><s1>
pip install diffusers transformers accelerate safetensors huggingface_hub
git clone https://github.com/replicate/cog-sdxl cog_sdxl
import torch
from huggingface_hub import hf_hub_download
from diffusers import DiffusionPipeline
from cog_sdxl.dataset_and_utils import TokenEmbeddingsHandler
from diffusers.models import AutoencoderKL
pipe = DiffusionPipeline.from_pretrained(
"stabilityai/stable-diffusion-xl-base-1.0",
torch_dtype=torch.float16,
variant="fp16",
).to("cuda")
load_lora_weights("jbilcke-hf/sdxl-zelda64", weight_name="lora.safetensors")
text_encoders = [pipe.text_encoder, pipe.text_encoder_2]
tokenizers = [pipe.tokenizer, pipe.tokenizer_2]
embedding_path = hf_hub_download(repo_id="jbilcke-hf/sdxl-zelda64", filename="embeddings.pti", repo_type="model")
embhandler = TokenEmbeddingsHandler(text_encoders, tokenizers)
embhandler.load_embeddings(embedding_path)
prompt="Link working as a pizza delivery driver, on a scooter, in new york, in the style of <s0><s1>"
images = pipe(
prompt,
cross_attention_kwargs={"scale": 0.8},
).images
#your output image
images[0]
- Downloads last month
- 839
Model tree for jbilcke-hf/sdxl-zelda64
Base model
stabilityai/stable-diffusion-xl-base-1.0
Adapter
this model