Z ImageでControlNetが利用できるようになったと聞きました。どうやって利用するのでしょうか?
Z-Image-Turbo-Fun-Controlnet-Union-2.0 をComfyUIで利用する手順を紹介ます。
HuggingFaceのZ-Image-Turbo-Fun-Controlnet-Union-2.0 のHubからモデルをダウンロードします。
以下のURLにアクセスして、Z-Image-Turbo-Fun-Controlnet-Union-2.0.safetensors ファイルをダウンロードします。
ダウンロードしたZ-Image-Turbo-Fun-Controlnet-Union-2.0.safetensors ファイルを次のディレクトリに配置します。
(ComfyUIの配置ディレクトリ)\ComfyUI\models\model_patches
または、(モデルをたくさん扱う場合でフォルダ分けしておいたほうが良い場合)
(ComfyUIの配置ディレクトリ)\ComfyUI\models\model_patches\Z-Image
ComfyUIを最新バージョンに更新します。ComfyUIはポータブル版を利用します。
アップデートの手順はこちらの記事を参照してください。
ControlNet Unionを利用するワークフローを作成します。ComfyUIにテンプレートが用意されていますが、
入れ子構造になっているテンプレートのため、今回はZ Imageのテンプレートから変更してワークフローを作成します。
こちらが元のZ ImageのTxt2Imgのワークフローです。
以下のワークフローを作成します。
作成手順は、Z ImageのTxt2Imgのワークフローに以下のノードを追加します。
Z Imageの拡散モデルとモデルパッチローダーから読み込んだZ-Image-Turbo-Fun-Controlnet-Union-2.0.safetensorsを[QwenImageDiffsynthControlnet]のノードにに入力します。
[画像を読み込む]ノードで読み込んだControlNetの入力画像も、[QwenImageDiffsynthControlnet]に入力します。
出力画像はControlNetの入力画像と同じサイズにしますので、[画像を読み込む]ノードで読み込んだControlNetの入力画像を[画像サイズ取得]で画像のサイズを取得し、
その値を[からのSD3潜在画像]の値に入力します。
[QwenImageDiffsynthControlnet]のモデル出力を[モデルサンプリングオーラフロー]の入力に設定します。
新しく配置するノードは左側の[ノード]ボタンをクリックするとパネルが表示されます。上部のテキストボックスで配置したいノードを検索で探せます。
ノードの項目を右側のワークフローのエリアにドラッグ＆ドロップするとノードを配置できます。
[画像を読み込む]ノードにControlNetの入力画像を設定します。[実行する]ボタンをクリックすると画像生成が開始されます。
右側の[画像を保存]のノードに画像生成結果が表示されます。
下図は実行結果です。ControlNetの入力画像の輪郭、構図で画像が生成できており、ControlNetを利用した画像生成ができていることが確認できます。
{
"id": "92112d97-bb64-4b44-86f2-ea5691ef8f6e",
"revision": 0,
"last_node_id": 31,
"last_link_id": 59,
"nodes": [
{
"id": 8,
"type": "VAEDecode",
"pos": [
1476.6373680509794,
185.0992738536158
],
"size": [
252,
102
],
"flags": {},
"order": 13,
"mode": 0,
"inputs": [
{
"name": "samples",
"type": "LATENT",
"link": 51
},
{
"name": "vae",
"type": "VAE",
"link": 45
}
],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"slot_index": 0,
"links": [
16
]
}
],
"properties": {
"Node name for S&R": "VAEDecode"
},
"widgets_values": []
},
{
"id": 9,
"type": "SaveImage",
"pos": [
1756.3297226240475,
183.04712727255492
],
"size": [
743.734375,
698.09375
],
"flags": {},
"order": 14,
"mode": 0,
"inputs": [
{
"name": "images",
"type": "IMAGE",
"link": 16
}
],
"outputs": [],
"properties": {},
"widgets_values": [
"ComfyUI"
]
},
{
"id": 15,
"type": "Note",
"pos": [
99.31122673040068,
213.67411688457722
],
"size": [
339.828125,
242.140625
],
"flags": {
"collapsed": false
},
"order": 0,
"mode": 0,
"inputs": [],
"outputs": [],
"properties": {},
"widgets_values": [
"The \"You are an assistant... <Prompt Start> \" text before the actual prompt is the one used in the official example.\n\nThe reason it is exposed to the user like this is because the model still works if you modify or remove it."
],
"color": "#432",
"bgcolor": "#653"
},
{
"id": 17,
"type": "VAELoader",
"pos": [
-33.67763808670634,
714.1307735371555
],
"size": [
447.09375,
112
],
"flags": {},
"order": 1,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "VAE",
"type": "VAE",
"links": [
45,
59
]
}
],
"properties": {
"Node name for S&R": "VAELoader"
},
"widgets_values": [
"ae.safetensors"
]
},
{
"id": 13,
"type": "EmptySD3LatentImage",
"pos": [
1168.8618604394235,
978.8059782076324
],
"size": [
378,
180
],
"flags": {},
"order": 10,
"mode": 0,
"inputs": [
{
"name": "width",
"type": "INT",
"widget": {
"name": "width"
},
"link": 56
},
{
"name": "height",
"type": "INT",
"widget": {
"name": "height"
},
"link": 57
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"slot_index": 0,
"links": [
17
]
}
],
"properties": {
"Node name for S&R": "EmptySD3LatentImage"
},
"widgets_values": [
1440,
1024,
1
]
},
{
"id": 7,
"type": "CLIPTextEncode",
"pos": [
506.0003310340792,
446.00025041612514
],
"size": [
510.328125,
222.71875
],
"flags": {},
"order": 7,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 44
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
6
]
}
],
"title": "CLIP Text Encode (Negative Prompt)",
"properties": {
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
""
],
"color": "#322",
"bgcolor": "#533"
},
{
"id": 18,
"type": "CLIPLoader",
"pos": [
-38.343784951076486,
514.067014509375
],
"size": [
452.46875,
180
],
"flags": {},
"order": 2,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "CLIP",
"type": "CLIP",
"links": [
43,
44
]
}
],
"properties": {
"Node name for S&R": "CLIPLoader"
},
"widgets_values": [
"Z-Image\\qwen_3_4b.safetensors",
"lumina2",
"default"
]
},
{
"id": 16,
"type": "UNETLoader",
"pos": [
-56.199706949792926,
-51.41007845473868
],
"size": [
474.75,
146
],
"flags": {},
"order": 3,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
"links": [
55
]
}
],
"properties": {
"Node name for S&R": "UNETLoader"
},
"widgets_values": [
"Z-Image\\z_image_turbo_bf16.safetensors",
"default"
]
},
{
"id": 11,
"type": "ModelSamplingAuraFlow",
"pos": [
470.0001900648797,
38.00008355841325
],
"size": [
378,
112
],
"flags": {},
"order": 11,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 54
}
],
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
"slot_index": 0,
"links": [
47
]
}
],
"properties": {
"Node name for S&R": "ModelSamplingAuraFlow"
},
"widgets_values": [
3
]
},
{
"id": 3,
"type": "KSampler",
"pos": [
1066.8534108494694,
187.03315597819946
],
"size": [
378,
354
],
"flags": {},
"order": 12,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 47
},
{
"name": "positive",
"type": "CONDITIONING",
"link": 4
},
{
"name": "negative",
"type": "CONDITIONING",
"link": 6
},
{
"name": "latent_image",
"type": "LATENT",
"link": 17
}
],
"outputs": [
{
"name": "LATENT",
"type": "LATENT",
"slot_index": 0,
"links": [
51
]
}
],
"properties": {
"Node name for S&R": "KSampler"
},
"widgets_values": [
357953819609516,
"randomize",
9,
1,
"euler",
"simple",
1
]
},
{
"id": 6,
"type": "CLIPTextEncode",
"pos": [
502.712085206889,
178.71216576298568
],
"size": [
508.59375,
218.53125
],
"flags": {},
"order": 6,
"mode": 0,
"inputs": [
{
"name": "clip",
"type": "CLIP",
"link": 43
}
],
"outputs": [
{
"name": "CONDITIONING",
"type": "CONDITIONING",
"slot_index": 0,
"links": [
4
]
}
],
"title": "CLIP Text Encode (Positive Prompt)",
"properties": {
"Node name for S&R": "CLIPTextEncode"
},
"widgets_values": [
"anime style image, Freight train yard, railroad tank car, low angle, train railway, wide view"
],
"color": "#232",
"bgcolor": "#353"
},
{
"id": 28,
"type": "ModelPatchLoader",
"pos": [
-27.249634466867747,
875.7637726567428
],
"size": [
444.296875,
115.046875
],
"flags": {},
"order": 4,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "MODEL_PATCH",
"type": "MODEL_PATCH",
"links": [
58
]
}
],
"properties": {
"Node name for S&R": "ModelPatchLoader"
},
"widgets_values": [
"Z-Image\\Z-Image-Turbo-Fun-Controlnet-Union-2.0.safetensors"
]
},
{
"id": 29,
"type": "LoadImage",
"pos": [
497.78198177745037,
961.6704931184462
],
"size": [
344,
136
],
"flags": {},
"order": 5,
"mode": 0,
"inputs": [],
"outputs": [
{
"name": "IMAGE",
"type": "IMAGE",
"links": [
52,
53
]
},
{
"name": "MASK",
"type": "MASK",
"links": null
}
],
"properties": {
"Node name for S&R": "LoadImage"
},
"widgets_values": [
"ComfyUI_temp_xmxoe_00001_.png",
"image"
]
},
{
"id": 30,
"type": "GetImageSize",
"pos": [
899.4770190799748,
979.8063385114975
],
"size": [
225,
190
],
"flags": {},
"order": 8,
"mode": 0,
"inputs": [
{
"name": "image",
"type": "IMAGE",
"link": 52
}
],
"outputs": [
{
"name": "width",
"type": "INT",
"links": [
56
]
},
{
"name": "height",
"type": "INT",
"links": [
57
]
},
{
"name": "batch_size",
"type": "INT",
"links": null
}
],
"properties": {
"Node name for S&R": "GetImageSize"
},
"widgets_values": [
"width: 3168, height: 1344\n batch size: 1"
]
},
{
"id": 31,
"type": "QwenImageDiffsynthControlnet",
"pos": [
880.9822400482728,
711.8111904410184
],
"size": [
307.5625,
208
],
"flags": {},
"order": 9,
"mode": 0,
"inputs": [
{
"name": "model",
"type": "MODEL",
"link": 55
},
{
"name": "model_patch",
"type": "MODEL_PATCH",
"link": 58
},
{
"name": "vae",
"type": "VAE",
"link": 59
},
{
"name": "image",
"type": "IMAGE",
"link": 53
},
{
"name": "mask",
"shape": 7,
"type": "MASK",
"link": null
}
],
"outputs": [
{
"name": "MODEL",
"type": "MODEL",
"links": [
54
]
}
],
"properties": {
"Node name for S&R": "QwenImageDiffsynthControlnet"
},
"widgets_values": [
1
]
}
],
"links": [
[
4,
6,
0,
3,
1,
"CONDITIONING"
],
[
6,
7,
0,
3,
2,
"CONDITIONING"
],
[
16,
8,
0,
9,
0,
"IMAGE"
],
[
17,
13,
0,
3,
3,
"LATENT"
],
[
43,
18,
0,
6,
0,
"CLIP"
],
[
44,
18,
0,
7,
0,
"CLIP"
],
[
45,
17,
0,
8,
1,
"VAE"
],
[
47,
11,
0,
3,
0,
"MODEL"
],
[
51,
3,
0,
8,
0,
"LATENT"
],
[
52,
29,
0,
30,
0,
"IMAGE"
],
[
53,
29,
0,
31,
3,
"IMAGE"
],
[
54,
31,
0,
11,
0,
"MODEL"
],
[
55,
16,
0,
31,
0,
"MODEL"
],
[
56,
30,
0,
13,
0,
"INT"
],
[
57,
30,
1,
13,
1,
"INT"
],
[
58,
28,
0,
31,
1,
"MODEL_PATCH"
],
[
59,
17,
0,
31,
2,
"VAE"
]
],
"groups": [],
"config": {},
"extra": {
"ds": {
"scale": 0.6934334949441587,
"offset": [
186.0083895123587,
223.63875109053
]
},
"frontendVersion": "1.34.8",
"workflowRendererVersion": "Vue"
},
"version": 0.4
}
