How to use multiple controlnets for Flux in comfyui?
How to use multiple controlnets for Flux in comfyui? Loading a controlnet model consumes a lot of video memory, which makes it impossible to use multiple controlnets. Is there any good solution?
Good question, I was trying to do the same
Flux has many built-in styles, and if you don't add a few more controlnets you really can't control it. For example, with the same set of prompt words and different seeds, the generated characters have completely different styles. It is difficult for me to control the head-to-body ratio of a character just by using pose. DualCFGGuider is also needed, because it can add negative prompt words to Flux.
If the strength is set too high, the picture quality will be poor, and if it is set too low, the guiding effect will be weak, so multiple controlnets are needed to control together. In addition, other functions in the controlNet applicator are also very useful for professional designers and we hope they can be added. Flux has a lot of potential, but it's hard for us to control it right now.
From what I read, the creators of the controlnet nodes for Flux (Kosinkadink and EeroHeikkinen) have not tuned them for the Pro version of the Union model yet. They work properly only with the alpha version of Union. This is why we get poor results with higher controlnet strengths. If you have any problems, use the alpha version of the Union model. Hopefully they can implement support for Union Pro soon.
Absolutely agree, the lineart models allow for much better sketch painting, relative to the regular canny one
Any news on this?
I am trying to combine both Canny and depth to restor an old photograph.