data-ai MCP Server
AI co-pilot for ComfyUI. Talk to your workflow — load, patch, run, learn. 113 MCP tools, four LLM providers (Claude, GPT-4o, Gemini, Ollama), full undo, native sidebar.
Discovered via github-topic:mcp-server and last synced 3mo ago.
Install instructions not detected yet
Check the source repository for the latest setup steps.
**Python 3.11+**
Resolution
Go[Execute] OK -->
canvas has a node the server doesn't
**Local** (your edit)
7-12
FAIL(["Failed: infra down"]) CB -->
touched references a node not on canvas (and not in workflow)
**Inherits** (experience)
TOOLS list → `_HANDLERS`
`4000`
~25
TOOLS list → `_HANDLERS`
Count
*(auto per provider)*
Agent Agent -->
D(["agent run"]) classDef orange fill:#d99458,color:#1a1a1a,stroke:#1a1a1a classDef yellow fill:#d9c958,color:#1a1a1a,stroke:#1a1a1a class D orange class A,B,C yellow ``` **Three prerequisites, four copy-paste steps. Under 2 minutes start to finish.**
**References**
**VariantSets**
A["Claude<br/>Streaming + Cache"] LLM -->
C["Gemini<br/>Function Decl."] LLM -->
~1.0 (guidance)
5-7
1-3.5
Agent classDef orange fill:#d99458,color:#1a1a1a,stroke:#1a1a1a classDef yellow fill:#d9c958,color:#1a1a1a,stroke:#1a1a1a class You,Verify orange class Understand,Discover,Pilot,Agent yellow ``` **Four phases, always in order:** 1. **UNDERSTAND** -- Reads your workflow, scans your models, checks what's installed 2. **DISCOVER** -- Searches CivitAI, HuggingFace, ComfyUI Manager (31k+ nodes) 3. **PILOT** -- Makes changes through safe, reversible delta layers (never edits your original) 4. **VERIFY** -- Runs the workflow, checks the output, records what worked When validation finds errors, the agent **auto-repairs**. One continuous flow, no stopping to ask: ```mermaid flowchart TD Run(["You: 'run this'"]) --> Validate["validate_before_execute"] Validate --> Check{"Errors?"} Check -->
WF{"Workflow\nloaded?"} Type -->
Parse["ExecutionEvent<br/>.from_ws_message()"] Parse --> Dispatch["TriggerRegistry<br/>.dispatch()"] Dispatch --> CB1["on_complete<br/>→ auto-evaluate"] Dispatch --> CB2["on_error<br/>→ log to session"] Dispatch --> CB3["on_progress<br/>→ custom callback"] Dispatch --> WH["Webhook<br/>→ POST JSON to URL"] classDef orange fill:#d99458,color:#1a1a1a,stroke:#1a1a1a classDef yellow fill:#d9c958,color:#1a1a1a,stroke:#1a1a1a class WS,CB1,CB3,WH orange class Parse,Dispatch,CB2,from_ws_message,dispatch yellow ``` ```python from cognitive.transport.triggers import on_execution_complete, register_webhook # Python callback on_execution_complete(lambda event: print(f"Done in {event.elapsed:.1f}s")) # External webhook (POSTs JSON on every execution_complete + execution_error) register_webhook("https://your-server.com/hook", ["execution_complete", "execution_error"]) ``` --- ## Autonomous Mode Write a creative intent. Hit go. No workflow file needed, no parameters to tune -- the agent composes a workflow, runs it on ComfyUI, scores the result, and learns from it automatically. ```mermaid flowchart TD You(["Creative Intent<br/>'cinematic portrait, golden hour'"]) --> INTENT["INTENT<br/>Parse + validate"] INTENT --> COMPOSE["COMPOSE<br/>Load template<br/>Blend with experience"] COMPOSE --> PROVISION{"PROVISION CHECK<br/>Models on disk?"} PROVISION -->
Forge Forge["Forge<br/>validated<br/>RFC6902 patches"] -->
Vision Vision["Vision<br/>judge quality<br/>(analyze_image)"] -->
Class["self_healing_ladder<br/>classify(error)"] Class -->
W["widget.value = new_value"] Kind -->
Trigger
server workflow has a node the canvas doesn't
unparseable node id, unknown kind, or non-conforming shape
Layer
Architect Architect["Architect<br/>plan: params,<br/>graph structure"] -->
node found, but input / widget name doesn't match
BrainAgent SDK → `_BRAIN_TOOL_NAMES`
`2000`
Done(["Stage flushed"]) classDef orange fill:#d99458,color:#1a1a1a,stroke:#1a1a1a classDef yellow fill:#d9c958,color:#1a1a1a,stroke:#1a1a1a class Intent,Scout,Architect,Provisioner,Forge,Crucible,Vision,Scribe,Done yellow ``` **Self-healing ladder** -- Article III mandates that every error gets classified once by `self_healing_ladder()` and routed to one of three policies. **TERMINAL is the only path that halts**; everything else burns iteration budget and continues. This is what makes a 24-hour autonomous run survivable: ComfyUI hiccups, missing assets, and rate-limit blips never stop the loop. ```mermaid flowchart TD Try["execute_fn(change_context)"] -->
G["✓ thinking / text branches<br/>mutually exclusive (if/elif)<br/>✓ ThinkingBlock skipped<br/>(was sending repr as text)"] Provider -->
Default
`8188`
LiteGraph's `connect` / `disconnectInput` returned `false`
A["✓ thinking blocks preserved<br/>in _to_response (with signature)<br/>✓ empty deltas filtered<br/>✓ signature-bearing ThinkingBlock<br/>replayed verbatim in convert_messages<br/>(c61c65f — supersedes cycle 20)"] Provider -->
Patch classDef orange fill:#d99458,color:#1a1a1a,stroke:#1a1a1a classDef yellow fill:#d9c958,color:#1a1a1a,stroke:#1a1a1a class Gate,Run,Check orange class Load,Repair,Fix,Analyze,Learn,Validate,Errors,Patch yellow ``` ### Project Structure ``` agent/ llm/ Multi-provider LLM abstraction (Anthropic, OpenAI, Gemini, Ollama) engine/ Execution-engine abstraction (IAIEngine + ComfyUIAdapter) Wraps POST /prompt, POST /interrupt, GET /history, WS /ws so the agent's execution path is backend-pluggable embedder.py MiniLM (all-MiniLM-L6-v2) -- 384-dim L2-normalized vectors Lazy-loaded, thread-safe, opt-in via requirements.txt tools/ 76 tools -- workflow ops, model search, provisioning, auto-wire, graph surgery, canvas bridge, UI->API parser, execution profiling workflow_patch.py wraps the cognitive engine for non-destructive PILOT comfy_execute.py routes execution traffic through agent/engine/ brain/ 27 tools -- vision, planning, memory, optimization adapters/ Pure-function translators between brain modules stage/ 23 tools -- USD state, prediction, composition (USD optional via [stage]) dag/ Workflow intelligence (6 computation nodes) gate/ Pre-dispatch safety (5-check pipeline) metrics.py Observability (Counter, Histogram, Gauge -- pure stdlib, thread-safe) degradation.py Fault isolation manager config.py Environment + 4 kill switches + LLM provider selection mcp_server.py MCP server (primary interface) cognitive/ LIVRPS state engine -- installed as top-level package (Phase 0.5) core/ CognitiveGraphEngine, DeltaLayer, WorkflowGraph (link-preserving) experience/ ExperienceChunk, GenerationContextSignature, Accumulator prediction/ CognitiveWorldModel, SimulationArbiter, CounterfactualGenerator transport/ SchemaCache, ExecutionEvent, interrupt, system_stats, TriggerRegistry pipeline/ Autonomous end-to-end orchestration tools/ Phase 3 macro-tools (analyze, mutate, query, compose, ...) ui/ __init__.py WEB_DIRECTORY + route registration web/js/sidebar.js Native left sidebar -- chat, quick actions, progress web/css/ Design system v3 -- ComfyUI-native CSS variables, theme-reactive server/routes.py WebSocket + REST endpoints for sidebar chat panel/ __init__.py WEB_DIRECTORY + route registration + sys.path injection server/routes.py 51 REST routes -- full tool surface (+ write-back v1 endpoints) server/touched.py Per-session "last pushed" snapshot + compute_touched (F-1) server/chat.py WebSocket chat handler -- clears touched session on disconnect web/js/ Bidirectional canvas bridge (no visible UI -- sidebar is primary) _deltaFailures.js L-7 surface-report accumulator _pushApplyTouched.js L-3/L-4/L-5/L-8 apply pipeline (widget + link + surface) _pushControl.js L-6 debounce + withObserverPause (module-level refcount) _pushOrchestrator.js Composed push: clear → fetch → pause → apply → ack superduperPanel.js Headless canvas↔agent bridge entry point agentClient.js HTTP client incl. getWorkflowApiWithTouched / ackPush graphMode.js GRAPH-mode panel + delta-failure status bar + modal tests/ 4,400+ pytest + 87 Vitest, all mocked, ~60s + ~250ms panel/ Vitest suite for write-back v1 (sample, deltaFailures, pushApplyTouched, pushControl, pushOrchestrator, integration, stress + LiteGraph stubs) integration/ Skips cleanly when ComfyUI not running test_bridge_routes_integration.py -- live agent<->node-pack seam ``` ### Production Hardening
`127.0.0.1`
Adapter["ComfyUIAdapter<br/>httpx + websockets<br/>circuit breaker"] Adapter -->
*(required for Anthropic)*
`~/ComfyUI`
I["/interrupt"] Adapter -->
`anthropic`
W["/ws"] classDef orange fill:#d99458,color:#1a1a1a,stroke:#1a1a1a classDef yellow fill:#d9c958,color:#1a1a1a,stroke:#1a1a1a class Adapter,P,I,H,W orange class Tool,Get,IFace yellow ``` The split is deliberate: **execution operations** live behind `IAIEngine` because they're the path that a future backend (a remote queue, a hosted ComfyUI fleet, a mock for tests) would re-implement. **Introspection endpoints** (`/object_info`, `/system_stats`, `/queue` status, `/userdata`) stay as direct `httpx` calls in their existing tool modules — they're discovery-only and not part of the execution surface. Engine errors form a hierarchy that parallels the LLM error hierarchy: `EngineError` base + `EngineConnectionError`, `EngineTimeoutError`, `EngineValidationError` (carries `node_errors`), `EngineServerError` (carries `status_code`), `EngineUnavailableError` (circuit-breaker open). The `subscribe_ws` context manager yields `EngineEvent(type, data, raw)` objects plus a `__timeout__` sentinel event that lets the caller re-check its deadline without losing the connection. ### Graceful Degradation Every subsystem has an independent kill switch. Set any of these to `0` in your `.env` to disable: `BRAIN_ENABLED` `DAG_ENABLED` `GATE_ENABLED` `OBSERVATION_ENABLED` All default to ON. The agent works fine with any combination disabled -- features gracefully disappear. ### Experience Loop Every generation is an experiment. The agent tracks what worked: - **Sessions 1-30**: Uses built-in knowledge only - **Sessions 30-100**: Blends knowledge with what it's learned from your renders - **Sessions 100+**: Primarily driven by your personal history ### Semantic Knowledge Retrieval The agent ships with 12 knowledge files (1,300+ lines) covering ControlNet preprocessor selection and strength scheduling (174 lines), Flux guidance and T5 encoder tuning (172 lines), multi-pass compositing for Nuke/AE/Fusion (119 lines), video workflows, 3D pipelines, and more. Retrieval is hybrid: keyword triggers fire first (fast path), then TF-IDF semantic search fills gaps when keywords miss. Pure Python, zero external dependencies -- no vector DB required. ```mermaid flowchart LR Context["Workflow context<br/>+ session notes"] --> KW{"Keyword<br/>triggers"} KW -->
*(auto per provider)*
Analyze[DAG<br/>Analysis] Repair --> Validate Fix --> Validate Analyze --> Gate[Safety<br/>Gate] Gate --> Patch[Patch via<br/>Delta Layer] Patch --> Run[Run on<br/>ComfyUI] Run --> Check[Check<br/>Output] Check --> Learn[Record<br/>Experience] Patch -->
`http://localhost:11434/v1`
*(same as `AGENT_MODEL`)*