The endpoint returns 503 when:

• Any backend's HTTP `/system_stats` doesn't return 200.
• Any backend's WebSocket connect fails.
• Any backend has latched a non-OOM CUDA fault (illegal memory

access, device-side assert, misaligned address, etc.) — these

don't survive ComfyUI's /free, so the wrapper marks the

backend permanently unhealthy and the orchestrator should

replace the pod.

OOM is not a health failure — the wrapper calls the affected

backend's /free to unload models and continues. The job that

hit OOM still fails (we don't retry inside the wrapper) but

subsequent jobs work.

Reliability

A short list of failure modes the wrapper handles for you, in

case you've hit them in plain ComfyUI:

• **WebSocket goes silent mid-job.** The wrapper cross-checks

/queue + `/history` and reconnects up to

WEBSOCKET_MAX_RECONNECTS times if the job is still alive on

ComfyUI's side. Without this, a transient WS close fails the

job even when ComfyUI is still happily running it.

• **ComfyUI prompt-cache returns empty outputs.** ComfyUI

assigns a new prompt_id on cache hit but doesn't copy the

original outputs into the new history entry. The wrapper

detects this (empty outputs map on a "completed" job),

clears the stale entry from /history, and re-submits to

force a fresh execution.

• **OOM mid-generation.** The wrapper detects "out of memory"

in the error path, posts to ComfyUI's /free to unload all

models, and keeps the pod alive. The OOM job itself is failed.

• **CUDA-unrecoverable faults.** Distinct from OOM: illegal

memory access, device-side assert, etc. The wrapper latches

this on the affected backend and /health flips to 503. Your

orchestrator should treat that as "replace the pod".

• **Workflow `output[*]` references a stale path.** Postprocess

prefers ComfyUI's files array (this run's actual outputs)

over images (which can carry stale references from a cached

prior run) when both are present.

• **Wrapper starts before ComfyUI is listening.** First-job

startup probes ComfyUI's HTTP and WebSocket endpoints before

the first POST.

• **Inbound saturation.** When the preprocess queue is full,

submits return 503 + Retry-After: 5 instead of stacking

forever. Lets the caller back off.

Workflow modifiers

Two ways to specify a workflow:

Raw workflow

Send the API-format workflow directly:

{
  "input": {
    "workflow_json": {
      "10": {
        "inputs": { "image": "https://example.com/input.jpg", "upload": "image" },
        "class_type": "LoadImage"
      }
      // … rest of workflow
    }
  }
}

# modifiers/image2image.py
from modifiers.basemodifier import BaseModifier
import random


class Image2Image(BaseModifier):
    WORKFLOW_JSON = "workflows/image2image.json"

    async def apply_modifications(self):
        self.workflow["3"]["inputs"]["seed"]  = await self.modify_workflow_value(
            "seed", random.randint(0, 2**32))
        self.workflow["6"]["inputs"]["text"]  = await self.modify_workflow_value(
            "prompt", "")
        self.workflow["10"]["inputs"]["image"] = await self.modify_workflow_value(
            "input_image", "https://example.com/default.jpg")
        await super().apply_modifications()      # runs the URL-download pass

{
  "input": {
    "modifier": "Image2Image",
    "modifications": {
      "prompt": "a beautiful sunset",
      "seed": 12345,
      "input_image": "https://example.com/photo.jpg"
    }
  }
}

"3": {
  "inputs": { "seed": "__RANDOM_INT__", "steps": 20, ... },
  "class_type": "KSampler"
}

                    ┌─ preprocess_worker × N (URL fetch, modifier apply)
   /generate{,/sync,/stream}
   ──▶ Pydantic validate ──▶ preprocess_queue
                                 │
                                 ▼
                              generation_worker × len(COMFYUI_BACKENDS)
                              (POST /prompt → WS wait → /history)
                                 │
                                 ▼
                              postprocess_worker × N
                              (move outputs, S3 upload, webhook)
                                 │
                                 ▼
                              response_store ──▶ /result, /generate/sync resolve, SSE final_result

pip install -r requirements-dev.txt
pytest                # unit tests for the bits that don't need ComfyUI
uvicorn main:app --reload --port 8000