Files
lobehub/docs/development/basic/comfyui-development.mdx
René Wang 3dfc86fd0f feat: Update user guide & changelog (#11518)
* feat: Redesign doc

* chore: uopdate site

* chore: uopdate site

* chore: uopdate site

* chore: uopdate site

* chore: uopdate site

* feat: Uopdate content

* chore: New doc

* chore: Update content

* chore: Update content

* chore: add images

* chore: add images

* chore: add images

* chore: add images

* feat: Add more images

* feat: Add more images

* fix: Cannot reach end

* chore: Update content

* chore: Update content

* chore: Update content

* chore: Update content

* chore: Update content

* Revise README content and structure

Updated README to reflect changes in project description and removed outdated notes.

* Revise 'Getting Started' and TOC in README

Updated the 'Getting Started' section and modified the table of contents.

* chore: Update content

* Revise README structure and content

Updated the Getting Started section and removed the Table of Contents. Adjusted the Local Development instructions.

* Remove custom themes section from README

Removed section about custom themes from README.

* Update README.md

* Refine introduction and highlight cloud version

Updated wording for clarity and added recommendation for cloud version.

* chore: Update content

* chore: Update content

* chore: Update content

* chore: Update content

* chore: Update content

* chore: Update content

* chore: Update content

* fix: add missing translation

* 🔀 chore: Move README changes to feat/readme branch

Co-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>

* fix: add missing translation

* chore: update cdn

* docs: add migration guide from v1.x local database to v2.x and update help sections

Signed-off-by: Innei <tukon479@gmail.com>

* fix: add missing translation

* fix: add missing images

* fix: add missing changelogs

* fix: add missing changelogs

* fix: add missing changelogs

* fix: add missing changelogs

* fix: add missing changelogs

* style: update cdn

---------

Signed-off-by: Innei <tukon479@gmail.com>
Co-authored-by: Claude Opus 4.5 <noreply@anthropic.com>
Co-authored-by: canisminor1990 <i@canisminor.cc>
Co-authored-by: Innei <tukon479@gmail.com>
2026-01-26 15:28:33 +08:00

1012 lines
30 KiB
Plaintext

---
title: ComfyUI Extension Development Guide
description: >-
Learn how to add new models, workflows, and features to LobeHub's ComfyUI
integration
tags:
- ComfyUI
- Development Guide
- Model Extension
- Workflow Development
---
# ComfyUI Extension Development Guide
This guide is based on actual code implementation and helps developers extend LobeHub's ComfyUI integration functionality.
## Architecture Overview
LobeHub ComfyUI integration uses a four-layer service architecture built around the main `LobeComfyUI` class:
```plaintext
packages/model-runtime/src/providers/comfyui/
├── index.ts # LobeComfyUI main class entry
├── services/ # Four core services
│ ├── comfyuiClient.ts # ComfyUIClientService - client and auth
│ ├── modelResolver.ts # ModelResolverService - model resolution
│ ├── workflowBuilder.ts # WorkflowBuilderService - workflow building
│ └── imageService.ts # ImageService - image generation
├── config/ # Configuration system
│ ├── modelRegistry.ts # Main model registry (222 models)
│ ├── fluxModelRegistry.ts # 130 FLUX model configurations
│ ├── sdModelRegistry.ts # 92 SD series model configurations
│ ├── systemComponents.ts # VAE/CLIP/T5/LoRA/ControlNet components
│ └── workflowRegistry.ts # Workflow routing configurations
├── workflows/ # Workflow implementations
│ ├── flux-dev.ts # FLUX Dev 20-step workflow
│ ├── flux-schnell.ts # FLUX Schnell 4-step fast workflow
│ ├── flux-kontext.ts # FLUX Kontext fill workflow
│ ├── sd35.ts # SD3.5 external encoder workflow
│ ├── simple-sd.ts # Generic SD workflow
│ └── index.ts # Workflow exports
├── utils/ # Utility layer
│ ├── staticModelLookup.ts # Model lookup functions
│ ├── workflowDetector.ts # Model architecture detection
│ ├── promptSplitter.ts # FLUX dual prompt splitting
│ ├── seedGenerator.ts # Random seed generation
│ ├── cacheManager.ts # TTL cache management
│ └── workflowUtils.ts # Workflow utility functions
└── errors/ # Error handling
├── base.ts # Base error classes
├── modelResolverError.ts # Model resolution errors
├── workflowError.ts # Workflow errors
└── servicesError.ts # Service errors
src/server/services/comfyui/ # Server-side implementation
├── core/ # Core server services
│ ├── comfyUIAuthService.ts # Authentication service
│ ├── comfyUIClientService.ts # Client service
│ ├── comfyUIConnectionService.ts # Connection service
│ ├── errorHandlerService.ts # Error handling service
│ ├── imageService.ts # Image generation service
│ ├── modelResolverService.ts # Model resolution service
│ └── workflowBuilderService.ts # Workflow builder service
├── config/ # Server-side configurations
│ ├── constants.ts # Constants and defaults
│ ├── modelRegistry.ts # Model registry
│ ├── fluxModelRegistry.ts # FLUX models
│ ├── sdModelRegistry.ts # SD models
│ ├── systemComponents.ts # System components
│ └── workflowRegistry.ts # Workflow registry
├── workflows/ # Server-side workflow implementations
│ ├── flux-dev.ts # FLUX Dev workflow
│ ├── flux-schnell.ts # FLUX Schnell workflow
│ ├── flux-kontext.ts # FLUX Kontext workflow
│ ├── sd35.ts # SD3.5 workflow
│ └── simple-sd.ts # Simple SD workflow
├── utils/ # Server utilities
│ ├── cacheManager.ts # Cache management
│ ├── componentInfo.ts # Component information
│ ├── imageResizer.ts # Image resizing
│ ├── promptSplitter.ts # Prompt splitting
│ ├── staticModelLookup.ts # Model lookup
│ ├── weightDType.ts # Weight dtype utilities
│ ├── workflowDetector.ts # Workflow detection
│ └── workflowUtils.ts # Workflow utilities
└── errors/ # Server error handling
├── base.ts # Base error classes
├── configError.ts # Configuration errors
├── modelResolverError.ts # Model resolver errors
├── servicesError.ts # Service errors
├── utilsError.ts # Utility errors
└── workflowError.ts # Workflow errors
packages/model-runtime/src/utils/ # Shared utilities
└── comfyuiErrorParser.ts # Unified error parser for client/server
```
### Core Service Architecture
The main `LobeComfyUI` class initializes four core services:
```typescript
// packages/model-runtime/src/providers/comfyui/index.ts
export class LobeComfyUI implements LobeRuntimeAI, AuthenticatedImageRuntime {
constructor(options: ComfyUIKeyVault = {}) {
// 1. Client Service - handles auth and API calls
this.clientService = new ComfyUIClientService(options);
// 2. Model Resolver Service - model lookup and component selection
const modelResolverService = new ModelResolverService(this.clientService);
// 3. Workflow Builder Service - routes and builds workflows
const workflowBuilderService = new WorkflowBuilderService({
clientService: this.clientService,
modelResolverService: modelResolverService,
});
// 4. Image Service - unified image generation entry point
this.imageService = new ImageService(
this.clientService,
modelResolverService,
workflowBuilderService,
);
}
}
```
## Authentication System
ComfyUI integration supports four authentication methods, handled by `AuthManager` within `ComfyUIClientService`:
### Supported Authentication Types
```typescript
interface ComfyUIKeyVault {
baseURL: string;
authType?: 'none' | 'basic' | 'bearer' | 'custom';
// Basic Auth
username?: string;
password?: string;
// Bearer Token
apiKey?: string;
// Custom Headers
customHeaders?: Record<string, string>;
}
```
### Authentication Configuration Examples
```typescript
// No authentication
const comfyUI = new LobeComfyUI({
baseURL: 'http://localhost:8000',
authType: 'none'
});
// Basic authentication
const comfyUI = new LobeComfyUI({
baseURL: 'https://your-comfyui-server.com',
authType: 'basic',
username: 'your-username',
password: 'your-password'
});
// Bearer Token
const comfyUI = new LobeComfyUI({
baseURL: 'https://your-comfyui-server.com',
authType: 'bearer',
apiKey: 'your-api-key'
});
// Custom headers
const comfyUI = new LobeComfyUI({
baseURL: 'https://your-comfyui-server.com',
authType: 'custom',
customHeaders: {
'X-API-Key': 'your-custom-key',
'Authorization': 'Custom your-token'
}
});
```
## WebAPI Routes
ComfyUI provides a REST WebAPI route for image generation, supporting both regular authentication and internal service authentication:
### Route Details
```typescript
// src/app/(backend)/webapi/create-image/comfyui/route.ts
export const runtime = 'nodejs';
export const maxDuration = 300; // 5 minutes max
// POST /api/create-image/comfyui
{
model: string; // Model identifier
params: { // Generation parameters
prompt: string;
width?: number;
height?: number;
// ... other parameters
};
options?: { // Optional generation options
// ... additional options
};
}
```
### Authentication Middleware
The WebAPI route uses the `checkAuth` middleware for authentication:
```typescript
import { checkAuth } from '@/app/(backend)/middleware/auth';
// The route automatically validates JWT tokens
// and passes authentication context to the tRPC caller
```
### Error Handling
The WebAPI route provides structured error responses:
```typescript
// AgentRuntimeError is extracted from TRPCError's cause
if (agentError && 'errorType' in agentError) {
// Convert errorType to appropriate HTTP status
// 401 for InvalidProviderAPIKey
// 403 for PermissionDenied
// 404 for NotFound
// 500+ for server errors
}
```
## Adding New Models
### 1. Understanding Model Registry Structure
Model configurations are stored in configuration files:
```typescript
// packages/model-runtime/src/providers/comfyui/config/modelRegistry.ts
export interface ModelConfig {
modelFamily: 'FLUX' | 'SD1' | 'SDXL' | 'SD3';
priority: number; // 1=Official, 2=Enterprise, 3=Community
recommendedDtype?: 'default' | 'fp8_e4m3fn' | 'fp8_e5m2';
variant: string; // Model variant identifier
}
```
### 2. Adding FLUX Models
Add new models in `fluxModelRegistry.ts`:
```typescript
// packages/model-runtime/src/providers/comfyui/config/fluxModelRegistry.ts
export const FLUX_MODEL_REGISTRY: Record<string, ModelConfig> = {
// Existing models...
// Add new FLUX Dev model
'your-custom-flux-dev.safetensors': {
modelFamily: 'FLUX',
priority: 2, // Enterprise-level model
variant: 'dev',
recommendedDtype: 'default',
},
// Add quantized version
'your-custom-flux-dev-fp8.safetensors': {
modelFamily: 'FLUX',
priority: 2,
variant: 'dev',
recommendedDtype: 'fp8_e4m3fn',
},
};
```
### 3. Adding SD Series Models
Add in `sdModelRegistry.ts`:
```typescript
// packages/model-runtime/src/providers/comfyui/config/sdModelRegistry.ts
export const SD_MODEL_REGISTRY: Record<string, ModelConfig> = {
// Existing models...
// Add new SD3.5 model
'your-custom-sd35.safetensors': {
modelFamily: 'SD3',
priority: 2,
variant: 'sd35',
recommendedDtype: 'default',
},
};
```
### 4. Update Model ID Mapping (Optional)
If you need friendly model IDs for frontend, add mapping in `modelRegistry.ts`:
```typescript
// packages/model-runtime/src/providers/comfyui/config/modelRegistry.ts
export const MODEL_ID_VARIANT_MAP: Record<string, string> = {
// Existing mappings...
// Add new model friendly IDs
'my-custom-flux': 'dev', // Maps to dev variant
'my-custom-sd35': 'sd35', // Maps to sd35 variant
};
```
## Creating New Workflows
### Workflow Creation Principles
**Important: Workflow node structures come from native ComfyUI exports**
1. Design workflow in ComfyUI interface
2. Export as JSON using "Export (API Format)"
3. Copy JSON structure to TypeScript file
4. Wrap and parameterize using `PromptBuilder`
### 1. Export Workflow from ComfyUI
In the ComfyUI interface:
1. Drag nodes to build desired workflow
2. Connect node inputs and outputs
3. Right-click empty area → "Export (API Format)"
4. Copy the generated JSON structure
### 2. Workflow File Template
Create new file `workflows/your-workflow.ts`:
```typescript
import { PromptBuilder } from '@saintno/comfyui-sdk';
import type { WorkflowContext } from '../services/workflowBuilder';
import { generateUniqueSeeds } from '../utils/seedGenerator';
import { getWorkflowFilenamePrefix } from '../utils/workflowUtils';
/**
* Build custom workflow
* @param modelFileName - Model file name
* @param params - Generation parameters
* @param context - Workflow context
*/
export async function buildYourCustomWorkflow(
modelFileName: string,
params: Record<string, any>,
context: WorkflowContext,
): Promise<PromptBuilder<any, any, any>> {
// JSON structure from ComfyUI "Export (API Format)"
const workflow = {
'1': {
_meta: { title: 'Load Checkpoint' },
class_type: 'CheckpointLoaderSimple',
inputs: {
ckpt_name: modelFileName,
},
},
'2': {
_meta: { title: 'CLIP Text Encode' },
class_type: 'CLIPTextEncode',
inputs: {
clip: ['1', 1], // Connect to node 1 CLIP output
text: params.prompt,
},
},
'3': {
_meta: { title: 'Empty Latent' },
class_type: 'EmptyLatentImage',
inputs: {
width: params.width,
height: params.height,
batch_size: 1,
},
},
'4': {
_meta: { title: 'KSampler' },
class_type: 'KSampler',
inputs: {
model: ['1', 0], // Connect to node 1 MODEL output
positive: ['2', 0], // Connect to node 2 CONDITIONING output
negative: ['2', 0], // Can configure negative prompts
latent_image: ['3', 0],
seed: params.seed ?? generateUniqueSeeds(1)[0],
steps: params.steps,
cfg: params.cfg,
sampler_name: 'euler',
scheduler: 'normal',
denoise: 1.0,
},
},
'5': {
_meta: { title: 'VAE Decode' },
class_type: 'VAEDecode',
inputs: {
samples: ['4', 0],
vae: ['1', 2], // Connect to node 1 VAE output
},
},
'6': {
_meta: { title: 'Save Image' },
class_type: 'SaveImage',
inputs: {
filename_prefix: getWorkflowFilenamePrefix('buildYourCustomWorkflow', context.variant),
images: ['5', 0],
},
},
};
// Wrap static JSON with PromptBuilder
const builder = new PromptBuilder(
workflow,
['width', 'height', 'steps', 'cfg', 'seed'], // Input parameters
['images'], // Output parameters
);
// Set output nodes
builder.setOutputNode('images', '6');
// Set input node paths
builder.setInputNode('width', '3.inputs.width');
builder.setInputNode('height', '3.inputs.height');
builder.setInputNode('steps', '4.inputs.steps');
builder.setInputNode('cfg', '4.inputs.cfg');
builder.setInputNode('seed', '4.inputs.seed');
// Set parameter values
builder
.input('width', params.width)
.input('height', params.height)
.input('steps', params.steps)
.input('cfg', params.cfg)
.input('seed', params.seed ?? generateUniqueSeeds(1)[0]);
return builder;
}
```
### 3. Register New Workflow
Add workflow mapping in `workflowRegistry.ts`:
```typescript
// packages/model-runtime/src/providers/comfyui/config/workflowRegistry.ts
import { buildYourCustomWorkflow } from '../workflows/your-workflow';
export const VARIANT_WORKFLOW_MAP: Record<string, WorkflowBuilder> = {
// Existing mappings...
// Add new workflow
'your-variant': buildYourCustomWorkflow,
};
```
### 4. Actual Workflow Example
Reference the real implementation in `flux-dev.ts`:
```typescript
// packages/model-runtime/src/providers/comfyui/workflows/flux-dev.ts (simplified)
export async function buildFluxDevWorkflow(
modelFileName: string,
params: Record<string, any>,
context: WorkflowContext,
): Promise<PromptBuilder<any, any, any>> {
// Get required components
const selectedT5Model = await context.modelResolverService.getOptimalComponent('t5', 'FLUX');
const selectedVAE = await context.modelResolverService.getOptimalComponent('vae', 'FLUX');
const selectedCLIP = await context.modelResolverService.getOptimalComponent('clip', 'FLUX');
// Handle dual prompt splitting
const { t5xxlPrompt, clipLPrompt } = splitPromptForDualCLIP(params.prompt);
// Static workflow definition (from ComfyUI export)
const workflow = {
'1': {
class_type: 'DualCLIPLoader',
inputs: {
clip_name1: selectedT5Model,
clip_name2: selectedCLIP,
type: 'flux',
},
},
// ... more nodes
};
// Parameter injection (must be done within workflow file)
workflow['5'].inputs.clip_l = clipLPrompt;
workflow['5'].inputs.t5xxl = t5xxlPrompt;
workflow['4'].inputs.width = params.width;
workflow['4'].inputs.height = params.height;
// Create and configure PromptBuilder
const builder = new PromptBuilder(workflow, inputs, outputs);
// Configure input/output mappings...
return builder;
}
```
## System Component Management
### Component Configuration Structure
All system components (VAE, CLIP, T5, LoRA, ControlNet) are unified in `systemComponents.ts`:
```typescript
// packages/model-runtime/src/providers/comfyui/config/systemComponents.ts
export interface ComponentConfig {
modelFamily: string; // Model family
priority: number; // 1=Required, 2=Standard, 3=Optional
type: string; // Component type
compatibleVariants?: string[]; // Compatible variants (LoRA/ControlNet)
controlnetType?: string; // ControlNet type
}
export const SYSTEM_COMPONENTS: Record<string, ComponentConfig> = {
// VAE components
'ae.safetensors': {
modelFamily: 'FLUX',
priority: 1,
type: 'vae',
},
// CLIP components
'clip_l.safetensors': {
modelFamily: 'FLUX',
priority: 1,
type: 'clip',
},
// T5 encoders
't5xxl_fp16.safetensors': {
modelFamily: 'FLUX',
priority: 1,
type: 't5',
},
// LoRA adapters
'realism_lora.safetensors': {
compatibleVariants: ['dev'],
modelFamily: 'FLUX',
priority: 1,
type: 'lora',
},
// ControlNet models
'flux-controlnet-canny-v3.safetensors': {
compatibleVariants: ['dev'],
controlnetType: 'canny',
modelFamily: 'FLUX',
priority: 1,
type: 'controlnet',
},
};
```
### Adding New Components
```typescript
// Add new LoRA
'your-custom-lora.safetensors': {
compatibleVariants: ['dev', 'schnell'],
modelFamily: 'FLUX',
priority: 2,
type: 'lora',
},
// Add new ControlNet
'your-controlnet-pose.safetensors': {
compatibleVariants: ['dev'],
controlnetType: 'pose',
modelFamily: 'FLUX',
priority: 2,
type: 'controlnet',
},
```
### Component Query API
```typescript
import { getAllComponentsWithNames, getOptimalComponent } from '../config/systemComponents';
// Get optimal component
const bestVAE = getOptimalComponent('vae', 'FLUX');
const bestT5 = getOptimalComponent('t5', 'FLUX');
// Query specific type components
const availableLoras = getAllComponentsWithNames({
type: 'lora',
modelFamily: 'FLUX',
compatibleVariant: 'dev'
});
// Query ControlNet
const cannyControlNets = getAllComponentsWithNames({
type: 'controlnet',
controlnetType: 'canny',
modelFamily: 'FLUX'
});
```
## Model Resolution and Lookup
### ModelResolverService Working Principles
```typescript
// packages/model-runtime/src/providers/comfyui/services/modelResolver.ts
export class ModelResolverService {
async resolveModelFileName(modelId: string): Promise<string | undefined> {
// 1. Clean model ID
const cleanId = modelId.replace(/^comfyui\//, '');
// 2. Check model ID mapping
const mappedVariant = MODEL_ID_VARIANT_MAP[cleanId];
if (mappedVariant) {
const prioritizedModels = getModelsByVariant(mappedVariant);
const serverModels = await this.getAvailableModelFiles();
// Find first available model by priority
for (const filename of prioritizedModels) {
if (serverModels.includes(filename)) {
return filename;
}
}
}
// 3. Direct registry lookup
if (MODEL_REGISTRY[cleanId]) {
return cleanId;
}
// 4. Check server file existence
if (isModelFile(cleanId)) {
const serverModels = await this.getAvailableModelFiles();
if (serverModels.includes(cleanId)) {
return cleanId;
}
}
return undefined;
}
}
```
### Model Lookup Examples
```typescript
// Actual usage examples
const resolver = new ModelResolverService(clientService);
// Friendly ID lookup
const fluxDevFile = await resolver.resolveModelFileName('flux-dev');
// Returns: 'flux1-dev.safetensors' (if exists)
// Direct filename lookup
const directFile = await resolver.resolveModelFileName('my-custom-model.safetensors');
// Returns: 'my-custom-model.safetensors' (if exists)
// Variant lookup
const devModels = getModelsByVariant('dev');
console.log(devModels.slice(0, 3));
// Output: ['flux1-dev.safetensors', 'flux1-dev-fp8.safetensors', ...]
```
## Error Handling
### Error Type Hierarchy
```plaintext
// packages/model-runtime/src/providers/comfyui/errors/
ComfyUIInternalError // Base error
├── ModelResolverError // Model resolution errors
├── WorkflowError // Workflow errors
├── ServicesError // Service errors
└── UtilsError // Utility errors
```
### Error Handling Examples
```typescript
import { ModelResolverError, WorkflowError } from '../errors';
try {
const result = await comfyUI.createImage({
model: 'nonexistent-model',
params: { prompt: 'test' }
});
} catch (error) {
if (error instanceof ModelResolverError) {
console.log('Model resolution failed:', error.message);
console.log('Error reason:', error.reason);
console.log('Error details:', error.details);
} else if (error instanceof WorkflowError) {
console.log('Workflow error:', error.message);
}
}
```
### Unified Error Parser
A shared error parser is available for both client and server-side error handling:
```typescript
// packages/model-runtime/src/utils/comfyuiErrorParser.ts
import { parseComfyUIErrorMessage, cleanComfyUIErrorMessage } from '../utils/comfyuiErrorParser';
// Parse error messages and determine error types
const { error, errorType } = parseComfyUIErrorMessage(rawError);
// Clean error messages from ComfyUI formatting
const cleanMessage = cleanComfyUIErrorMessage(errorMessage);
```
The error parser handles:
- HTTP status code mapping to error types
- Server-side error enhancement
- Model file missing detection
- Network error identification
- Workflow validation errors
## Testing Architecture and Development
### Testing Architecture Overview
The ComfyUI integration uses a unified testing architecture that ensures maintainability and customization-friendly tests. This architecture includes:
- **Unified Mock System**: Centralized management of all external dependency mocks
- **Parameterized Testing**: Automatically adapts to new models without modifying existing tests
- **Fixture System**: Retrieves test data from configuration files to ensure accuracy
- **Coverage Goals**: ComfyUI module maintains 97%+ coverage
### Test File Structure
```plaintext
packages/model-runtime/src/providers/comfyui/__tests__/
├── setup/
│ └── unifiedMocks.ts # Unified Mock configuration
├── fixtures/
│ ├── parameters.fixture.ts # Parameter test fixtures
│ └── workflow.fixture.ts # Workflow test fixtures
├── integration/
│ ├── parameterMapping.test.ts # Parameter mapping integration tests
│ └── workflowBuilder.test.ts # Workflow builder tests
├── services/ # Service unit tests
└── workflows/ # Workflow unit tests
```
### Adding Tests for New Models
When adding new models, tests will automatically recognize and run appropriate parameter mapping tests. You only need to:
#### 1. Add Parameter Schema in Model Configuration
```typescript
// packages/model-bank/src/aiModels/comfyui.ts
export const myNewModelParamsSchema = {
prompt: { type: 'string', required: true },
steps: { type: 'number', default: 20, min: 1, max: 150 },
cfg: { type: 'number', default: 7.0, min: 1.0, max: 30.0 }
};
```
#### 2. Create Workflow Builder
```typescript
// packages/model-runtime/src/providers/comfyui/workflows/myNewModel.ts
export async function buildMyNewModelWorkflow(
modelName: string,
params: MyNewModelParams,
context: ComfyUIContext
) {
const workflow = { /* workflow definition */ };
// Parameter injection
workflow['1'].inputs.prompt = params.prompt;
workflow['2'].inputs.steps = params.steps;
return workflow;
}
```
#### 3. Register Model in Fixtures
```typescript
// packages/model-runtime/src/providers/comfyui/__tests__/fixtures/parameters.fixture.ts
import {
myNewModelParamsSchema,
// ... other schemas
} from '../../../../../model-bank/src/aiModels/comfyui';
export const parametersFixture = {
models: {
'my-new-model': {
schema: myNewModelParamsSchema,
defaults: {
steps: myNewModelParamsSchema.steps.default,
cfg: myNewModelParamsSchema.cfg.default,
},
boundaries: {
min: { steps: myNewModelParamsSchema.steps.min },
max: { steps: myNewModelParamsSchema.steps.max }
}
}
}
};
```
### Testing Best Practices
#### Use Unified Mock System
```typescript
import { setupAllMocks } from '../setup/unifiedMocks';
describe('MyTest', () => {
const mocks = setupAllMocks();
beforeEach(() => {
vi.clearAllMocks();
});
});
```
#### Write Parameter Mapping Tests
Parameter mapping tests run automatically, verifying that frontend parameters are correctly injected into workflows:
```typescript
// Tests automatically include newly registered models
describe.each(
Object.entries(models).filter(([name]) => workflowBuilders[name])
)(
'%s parameter mapping',
(modelName, modelConfig) => {
it('should map schema parameters to workflow', async () => {
const params = {
prompt: 'test prompt',
...modelConfig.defaults,
};
const workflow = await builder(`${modelName}.safetensors`, params, mockContext);
expect(workflow).toBeDefined();
});
}
);
```
#### Customization-Friendly Testing Principles
- **Don't test workflow structure**: Workflows are ComfyUI's official format; only test parameter mapping
- **Use configuration-driven data**: Test data comes from model configuration files to ensure consistency
- **Avoid brittle assertions**: Don't check specific node IDs or internal structures
- **Support extension**: New models should only affect coverage, not break existing tests
### Running Tests
```bash
# Run ComfyUI related tests
cd packages/model-runtime
bunx vitest run --silent='passed-only' 'src/comfyui'
# View coverage
bunx vitest run --coverage 'src/comfyui'
# Run specific test files
bunx vitest run 'src/comfyui/__tests__/integration/parameterMapping.test.ts'
```
### Coverage Targets
- **Overall coverage**: ComfyUI module maintains 97%+ coverage
- **Core functionality**: 100% branch coverage
- **New features**: Maintain or improve existing coverage levels
## Development and Testing
### 1. Local Development Setup
```bash
# Start ComfyUI debug mode
DEBUG=lobe-image:* pnpm dev
```
### 2. Testing New Features
```typescript
// Create test file
import { buildYourCustomWorkflow } from './your-workflow';
describe('Custom Workflow', () => {
test('should build workflow correctly', async () => {
const mockContext = {
clientService: mockClientService,
modelResolverService: mockModelResolver,
};
const workflow = await buildYourCustomWorkflow(
'test-model.safetensors',
{ prompt: 'test', width: 512, height: 512 },
mockContext
);
expect(workflow).toBeDefined();
// Verify workflow structure...
});
});
```
### 3. Model Configuration Testing
```typescript
import { getModelConfig, getAllModelNames } from '../config/modelRegistry';
describe('Model Registry', () => {
test('should find new model', () => {
const config = getModelConfig('your-new-model.safetensors');
expect(config).toBeDefined();
expect(config?.variant).toBe('dev');
expect(config?.modelFamily).toBe('FLUX');
});
});
```
## Complete Usage Examples
### Basic Image Generation
```typescript
import { LobeComfyUI } from '@/libs/model-runtime/comfyui';
const comfyUI = new LobeComfyUI({
baseURL: 'http://localhost:8000',
authType: 'none'
});
// FLUX Dev model generation
const result = await comfyUI.createImage({
model: 'flux-dev',
params: {
prompt: 'Beautiful landscape painting, high quality, detailed',
width: 1024,
height: 1024,
steps: 20,
cfg: 3.5,
seed: -1
}
});
console.log('Generated image URL:', result.imageUrl);
```
### SD3.5 Model Usage
```typescript
// SD3.5 automatically detects available encoders
const sd35Result = await comfyUI.createImage({
model: 'stable-diffusion-35',
params: {
prompt: 'Futuristic cityscape',
width: 1344,
height: 768,
steps: 28,
cfg: 4.5
}
});
```
### Enterprise Optimized Models
```typescript
// System automatically selects best available variant (e.g., FP8 quantized)
const optimizedResult = await comfyUI.createImage({
model: 'flux-dev',
params: {
prompt: 'Professional business portrait',
width: 768,
height: 1024,
steps: 15 // FP8 models can use fewer steps
}
});
```
## Important Notes
- Ensure ComfyUI service is running and accessible
- Check that all required model files are properly installed
- Pay attention to model file naming conventions and path configurations
- Regularly check and update workflow configurations to support new features
- Be aware of parameter differences and compatibility across model families
- When adding new models, follow the testing architecture guidelines to ensure test completeness
- Always run relevant tests before committing code to ensure coverage targets are met
## Summary
This documentation is based on actual code implementation and includes:
1. **Real Architecture Description**: Four-layer service architecture with clear responsibility separation
2. **Accurate API Calls**: Using `PromptBuilder` instead of fictional classes
3. **Correct Workflow Creation**: Real process of exporting JSON from ComfyUI
4. **Actual Configuration Structure**: Based on real registry files
5. **Working Code Examples**: All examples can be run directly
6. **Comprehensive Testing Guide**: Unified testing architecture with customization-friendly approach
Developers can use this accurate documentation to effectively extend ComfyUI integration functionality while maintaining high code quality and test coverage.