File size: 2,402 Bytes
8311637 c577f80 5c3d006 8311637 96c7f05 8311637 689d647 291bbd7 689d647 431cdce 689d647 70f8801 431cdce 689d647 d6fecf3 689d647 cec6b93 689d647 ac0c36e 689d647 5c3d006 689d647 92ca95d adc1f0e 92ca95d 2e4874a 92ca95d d7b1dc7 92ca95d 3b084d2 689d647 46745ac |
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 |
---
base_model:
- black-forest-labs/FLUX.1-dev
library_name: diffusers
license: other
license_name: flux-1-dev-non-commercial-license
license_link: https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md
pipeline_tag: image-to-image
tags:
- ControlNet
---
# ⚡ Flux.1-dev: Depth ControlNet ⚡
This is [Flux.1-dev](https://huggingface.co/black-forest-labs/FLUX.1-dev) ControlNet for Depth map developed by Jasper research team.
<p align="center">
<img style="width:700px;" src="examples/showcase.jpg">
</p>
# How to use
This model can be used directly with the `diffusers` library
```python
import torch
from diffusers.utils import load_image
from diffusers import FluxControlNetModel
from diffusers.pipelines import FluxControlNetPipeline
# Load pipeline
controlnet = FluxControlNetModel.from_pretrained(
"jasperai/Flux.1-dev-Controlnet-Depth",
torch_dtype=torch.bfloat16
)
pipe = FluxControlNetPipeline.from_pretrained(
"black-forest-labs/FLUX.1-dev",
controlnet=controlnet,
torch_dtype=torch.bfloat16
)
pipe.to("cuda")
# Load a control image
control_image = load_image(
"https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Depth/resolve/main/examples/depth.jpg"
)
prompt = "a statue of a gnome in a field of purple tulips"
image = pipe(
prompt,
control_image=control_image,
controlnet_conditioning_scale=0.6,
num_inference_steps=28,
guidance_scale=3.5,
height=control_image.size[1],
width=control_image.size[0]
).images[0]
image
```
<p align="center">
<img style="width:500px;" src="examples/output.jpg">
</p>
💡 Note: You can compute the conditioning map using for instance the `MidasDetector` from the `controlnet_aux` library
```python
from controlnet_aux import MidasDetector
from diffusers.utils import load_image
midas = MidasDetector.from_pretrained("lllyasviel/Annotators")
midas.to("cuda")
# Load an image
im = load_image(
"https://huggingface.co/jasperai/Flux.1-dev-Controlnet-Depth/resolve/main/examples/output.jpg"
)
depth = midas(im)
```
# Training
This model was trained with depth maps computed with [Clipdrop's depth estimator model](https://clipdrop.co/apis/docs/portrait-depth-estimation) as well as open-souce depth estimation models such as Midas or Leres.
# Licence
This model falls under the [Flux.1-dev licence](https://huggingface.co/black-forest-labs/FLUX.1-dev/blob/main/LICENSE.md). |