development/overview

[giscus[bot]]

development/overview

Using ComfyUI as a Developer

https://docs.comfy.org/development/overview

[sigm3099-blip]

e

[meimeicrepe]

e

[meimeicrepe]

e

[horsten]

ComfyUI Internals: The Missing Manual

"The missing blueprint for the ComfyUI Engine."

https://github.com/horsten/comfyui-internals-doc/blob/main/README.md

If you want to write nodes or add/change features that go a bit deeper than the APIs described here, and you need to interact with the "core engine" of ComfyUI, I've often found it tricky to navigate the both brilliantly efficient but also very undocumented internals of ComfyUI when writing custom nodes, patching models, or generally hacking around. There are a lot of implicit "API contracts" and core concepts that are non-obvious because they're not documented anywhere, so it often involves trying to find a "close enough example" of what you want to do if you're not a core developer and trying to understand the system.

I've tried to document the key concepts that one needs to understand when adding features, models, or experiments. It started mostly for my own benefit, but I hope others might find it useful - perhaps especially those who create "wrapper nodes" for external models and similar, often resulting in something very fragile that doesn't coexist happily with anything else Comfy.

As of now, you'll find information about the following:

📚 The Architecture

1. The Object Lifecycle Understanding ModelPatcher, model_management, and why you must always .clone() before modifying.
2. The Execution Engine How the graph is sorted, validated, and executed. Explains Caching, Lazy Evaluation, and Subgraphs.
3. The Node Interface How INPUT_TYPES, RETURN_TYPES, and the V1 vs V3 initialization protocols work.
4. The Sampling Process A deep dive into comfy.samplers. How Guiders, Schedulers, and Conditionings interact to generate an image.
5. Model Loading & Detection Analysis of comfy.sd. How the system "guesses" which model architecture a SafeTensors file contains.
6. Guide: Adding New Architectures A step-by-step guide to implementing support for new Diffusion Models (e.g., Flux, Cosmos, OmniGen).
7. Patching Guide How to inject code into the model (LoRAs, ControlNets, and Attention Hooks) without breaking the engine.

⚡ Quick Reference

• API Contracts & Pitfalls The "Rules of the Road". Do's and Don'ts to prevent memory leaks and execution bugs.
• Tensor Management Where to create tensors (CPU vs GPU), correct data types, and how to handle intermediate_device.
• Type Definitions What exactly is inside a MODEL, LATENT, or MASK object?