File size: 1,632 Bytes
8162c44 ffe9610 d262df1 |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 |
---
license: openrail
---
# ControlNet-XS model for StableDiffusion 2.1 and depth input
🔬 Original paper and models by https://github.com/vislearn/ControlNet-XS
👷🏽♂️ Translated into diffusers architecture by https://twitter.com/UmerHAdil
This model is trained for use with [StableDiffusion 2.1](https://huggingface.co/stabilityai/stable-diffusion-2-1)
---
ControlNet-XS was introduced in [ControlNet-XS](https://vislearn.github.io/ControlNet-XS/) by Denis Zavadski and Carsten Rother. It is based on the observation that the control model in the [original ControlNet](https://huggingface.co/papers/2302.05543) can be made much smaller and still produces good results.
As with the original ControlNet model, you can provide an additional control image to condition and control Stable Diffusion generation. For example, if you provide a depth map, the ControlNet model generates an image that'll preserve the spatial information from the depth map. It is a more flexible and accurate way to control the image generation process.
Using ControlNet-XS instead of regular ControlNet will produce images of roughly the same quality, but 20-25% faster ([see benchmark](https://github.com/UmerHA/controlnet-xs-benchmark/blob/main/Speed%20Benchmark.ipynb)) and with ~45% less memory usage.
---
Other ControlNet-XS models:
- [StableDiffusion-XL and canny edges input](https://huggingface.co/UmerHA/ConrolNetXS-SDXL-canny)
- [StableDiffusion-XL and depth input](https://huggingface.co/UmerHA/ConrolNetXS-SDXL-depth)
- [StableDiffusion 2.1 and canny edges input](https://huggingface.co/UmerHA/ConrolNetXS-SD2.1-canny) |