8 things I measured building a 2h manhwa recap in ComfyUI (SDXL/Illustrious) — plus 3 problems I still can't solve

[zaozbr]

I'm building a two-hour manhwa-style recap locally: RTX 4080 16GB, ComfyUI, an
Illustrious-XL checkpoint plus a character LoRA I trained. Hundreds of panels,
one character who has to stay the same person across all of them.

Most of what I learned cost me GPU hours, so here it is. Then three things I'm
still stuck on — if you know any of them, that's what I'm really after.

What I measured

1. Expression belongs in the base pass, not FaceDetailer. FaceDetailer runs
~0.45 denoise on a small crop. It adjusts a face; it will not open a mouth
the base pass drew closed. I burned 4 attempts on a "shouting" face before
moving the clause to the base prompt, where it worked first try.

2. Low denoise is cumulative, and this one hurt. I had a composited scene I
refined in four successive passes at 0.30–0.32. Each pass looks safe. Stacked,
they add up to one high-denoise pass — and the face goes first, because it's
small and high-frequency. Across those four passes the skin went from pale to
tanned, the eyebrows dissolved into floating smudges, and one eye lost its iris
entirely and became a blank white oval. A sibling image generated the same way
but never fused has a perfect face. If you refine iteratively, count your
total denoise, not the per-pass number.

3. IPAdapterAdvanced transfers content, not just style. At weight 0.35 it
pulled two background extras from my style reference into a frame that had
solo:1.4 in the prompt. Switching to IPAdapterPreciseStyleTransfer at weight
0.60 with style_boost 2.0 and end_at 1.0 leaked nothing and matched the
reference better. style_boost 3.5 at the same weight was worse — the ink
outline came back and an extra figure appeared. High boost pushes the whole
embedding, content included.

4. Hypothesis I had that turned out wrong: low end_at for style. I assumed
palette and texture are decided in early steps and composition late, so cutting
IPAdapter early would lock the finish and give the scene back to the prompt.
Tested end_at 0.35 at weights 0.45 and 0.65 — both reverted to clean-line
flat-colour anime, exactly what I was trying to escape. The finish isn't only
set early; it's lost if IPAdapter exits before the end.

5. Negations in the positive prompt inject the thing. "and nothing else
blue" is one more mention of blue. Obvious in hindsight; it silently corrupted
10 scenes. Prohibitions go in the negative, always.

6. An attribute that must show up goes in the first ~90 tokens. Same words,
same weight, moved from the tail of the prompt to the critical block at the
front — and the drawing changed. CLIP reads the tail weakly.

7. BiRefNet masks are soft. Over a light background the halo brings wall and
floor along with the cutout. Threshold the mask hard, and always inspect the
cutout over magenta — over white, a white halo is invisible.

8. Effects on finished art are compositing, not repainting. I tried to add
an impact dust burst to a finished punch. Global denoise 0.32: nothing appeared
(low denoise preserves, it does not add). Local inpaint radius 150: destroyed
the opponent's head 115px away. Radius 82 on the fist: destroyed the fist. What
worked: generate the dust as its own asset on black, cut it out, desaturate,
drop opacity, composite. Zero pixels of the original touched.

Also small but large in wall-clock: {"prompt": g, "front": True} on
POST /prompt puts a job at the front of the queue. A 1.4s cutout was waiting
139s behind a batch; a 29s fusion waited 361s. About 80% of my per-scene elapsed
time was queue wait, not compute.

What I still can't solve

A) SetLatentNoiseMask does not seem to preserve at low denoise. I want to
fuse a composite's seams without touching the faces. I build a mask (black =
preserve) from a YOLO face detector, feed it through SetLatentNoiseMask, and
sample at denoise 0.32. Measured mean absolute pixel change: 16.4 inside the
"protected" face vs 13.7 in the free area — the protected region changed
more. With DifferentialDiffusion in the model path: 16.4. Without it: 15.4.
Neither preserves. Solid mask core is 17,712px, so it isn't a blur problem
anymore (that was my first bug — a fixed 28px blur over a 64px face erased the
mask entirely; minimum value never went below 0.047).

Is SetLatentNoiseMask simply not meant for denoise < 1.0? Is
DifferentialDiffusion only correct at denoise 1.0? Is there a node that means
"resample everything except here" at partial denoise?

B) OpenPose/DWPose: the head doesn't follow the body. In dynamic action the
body takes the skeleton correctly but the head keeps drifting to a
three-quarter front view regardless of the head keypoints. Anyone found a
reliable fix — separate face ControlNet, higher weight only on head joints,
something else?

C) Character LoRA gives a face that's "too anime" when I want semi-real.
Lowering LoRA weight loses identity before it loses the anime read. I've seen
LoraLoaderBlockWeight (Inspire Pack) suggested for applying the LoRA only to
some UNet blocks — does that actually separate "identity" from "style" in
practice, or is that wishful thinking?

Happy to share exact graphs/numbers for any of the eight above if useful.