File size: 2,469 Bytes
8311637 d19b7e5 5c3d006 8311637 96c7f05 8311637 689d647 ac0c36e 689d647 431cdce 689d647 70f8801 431cdce 689d647 d6fecf3 689d647 cec6b93 689d647 ac0c36e 689d647 5c3d006 689d647 92ca95d adc1f0e 92ca95d 3b084d2 689d647 ee620e4 8244a2e |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 |
---
base_model:
- black-forest-labs/FLUX.1-dev
library_name: diffusers
license: cc-by-nc-4.0
license_name: flux-1-dev-non-commercial-license
license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md
pipeline_tag: image-to-image
tags:
- ControlNet
---
# ⚡ Flux.1-dev: Depth ControlNet ⚡
This is [Flux.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) ControlNet for Depth map developped by Jasper research team.
<p align="center">
<img style="width:700px;" src="examples/showcase.jpg">
</p>
# How to use
This model can be used directly with the `diffusers` library
```python
import torch
from diffusers.utils import load_image
from diffusers import FluxControlNetModel
from diffusers.pipelines import FluxControlNetPipeline
# Load pipeline
controlnet = FluxControlNetModel.from_pretrained(
"jasperai/Flux.1-dev-Controlnet-Depth",
torch_dtype=torch.bfloat16
)
pipe = FluxControlNetPipeline.from_pretrained(
"black-forest-labs/FLUX.1-dev",
controlnet=controlnet,
torch_dtype=torch.bfloat16
)
pipe.to("cuda")
# Load a control image
control_image = load_image(
"https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Depth/resolve/main/examples/depth.jpg"
)
prompt = "a statue of a gnome in a field of purple tulips"
image = pipe(
prompt,
control_image=control_image,
controlnet_conditioning_scale=0.6,
num_inference_steps=28,
guidance_scale=3.5,
height=control_image.size[1],
width=control_image.size[0]
).images[0]
image
```
<p align="center">
<img style="width:500px;" src="examples/output.jpg">
</p>
💡 Note: You can compute the conditioning map using for instance the `MidasDetector` from the `controlnet_aux` library
```python
from controlnet_aux import MidasDetector
from diffusers.utils import load_image
midas = MidasDetector.from_pretrained("lllyasviel/Annotators")
midas.to("cuda")
# Load an image
im = load_image(
"https://huggingface.co/jasperai/jasperai/Flux.1-dev-Controlnet-Depth/resolve/main/examples/output.jpg"
)
surface = midas(im)
```
# Training
This model was trained with depth maps computed with [Clipdrop's depth estimator model](https://clipdrop.co/apis/docs/portrait-depth-estimation) as well as open-souce depth estimation models such as Midas or Leres.
# Licence
This model is released under the the Creative Commons BY-NC license.
Note that if used with Flux.1-dev, the license under the Flux.1-dev model also applies to this model. |