Update README.md

README.md

@@ -18,14 +18,15 @@ tags:
 base_model: black-forest-labs/FLUX.1-dev
 ---
 
-# FLUX.1-dev-ControlNet-Depth
+# FLUX.1-dev-ControlNet-Union-Pro
 
-This repository contains an unified ControlNet for FLUX.1-dev model jointly trained by researchers from [InstantX Team](https://huggingface.co/InstantX) and [Shakker Labs](https://huggingface.co/Shakker-Labs).
+This repository contains an unified ControlNet for FLUX.1-dev model jointly released by researchers from [InstantX Team](https://huggingface.co/InstantX) and [Shakker Labs](https://huggingface.co/Shakker-Labs).
 
 # Model Cards
 - This checkpoint is a Pro version of [FLUX.1-dev-Controlnet-Union](https://huggingface.co/InstantX/FLUX.1-dev-Controlnet-Union).
 - This model support 7 control modes, including canny (0), tile (1), depth (2), blur (3), pose (4), gray (5), low quality (6).
 - The recommended controlnet_conditioning_scale is 0.3-0.7.
+- This model can be jointly used with other ControlNets.
 
 
 # Showcases
@@ -36,9 +37,56 @@ This repository contains an unified ControlNet for FLUX.1-dev model jointly trai
 
 
 # Inference
+Please install `diffusers` from the source, as the PR has not been included in currently released version yet.
+
+# Multi-Controls Inference
+```python
+import torch
+from diffusers.utils import load_image
+
+from diffusers import FluxControlNetPipeline, FluxControlNetModel, FluxMultiControlNetModel
+
+base_model = 'black-forest-labs/FLUX.1-dev'
+controlnet_model_union = './Shakker-Labs/FLUX.1-dev-Controlnet-Union'
+
+controlnet_union = FluxControlNetModel.from_pretrained(controlnet_model_union, torch_dtype=torch.bfloat16)
+controlnet = FluxMultiControlNetModel([controlnet_union]) # we always recommend loading via FluxMultiControlNetModel
+
+pipe = FluxControlNetPipeline.from_pretrained(base_model, controlnet=controlnet, torch_dtype=torch.bfloat16)
+pipe.to("cuda")
+
+prompt = 'a young girl'
+control_image_depth = load_image("https://huggingface.co/InstantX/FLUX.1-dev-Controlnet-Union/resolve/main/images/depth.jpg")
+control_mode_depth = 2
+
+control_image_canny = load_image("https://huggingface.co/InstantX/FLUX.1-dev-Controlnet-Union/resolve/main/images/canny.jpg")
+control_mode_canny = 0
+
+width, height = control_image.size
+
+image = pipe(
+    prompt, 
+    control_image=[control_image_depth, control_image_canny],
+    control_mode=[control_mode_depth, control_mode_canny],
+    width=width,
+    height=height,
+    controlnet_conditioning_scale=[0.5, 0.5],
+    num_inference_steps=24, 
+    guidance_scale=3.5,
+    generator=torch.manual_seed(42),
+).images[0]
+```
+
+We also support loading multiple ControlNets as before, you can load as
 ```python
-# TODO
+controlnet_model_union = './Shakker-Labs/FLUX.1-dev-Controlnet-Union'
+controlnet_union = FluxControlNetModel.from_pretrained(controlnet_model_union, torch_dtype=torch.bfloat16)
+
+controlnet_model_depth = './Shakker-Labs/FLUX.1-dev-Controlnet-Depth'
+controlnet_depth = FluxControlNetModel.from_pretrained(controlnet_model_depth, torch_dtype=torch.bfloat16)
+
+controlnet = FluxMultiControlNetModel([controlnet_union, controlnet_depth])
 ```
 
 # Acknowledgements
-This project is sponsored by [Shakker AI](https://www.shakker.ai/). All copyright reserved.
+This project is trained by [InstantX Team](https://huggingface.co/InstantX) and sponsored by [Shakker AI](https://www.shakker.ai/). All copyright reserved.