deer-flow/backend/CLAUDE.md

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Project Overview

DeerFlow is a LangGraph-based AI super agent system with a full-stack architecture. The backend provides a "super agent" with sandbox execution, persistent memory, subagent delegation, and extensible tool integration - all operating in per-thread isolated environments.

Architecture:

• LangGraph Server (port 2024): Agent runtime and workflow execution
• Gateway API (port 8001): REST API for models, MCP, skills, memory, artifacts, and uploads
• Frontend (port 3000): Next.js web interface
• Nginx (port 2026): Unified reverse proxy entry point

Project Structure:

deer-flow/
├── Makefile                    # Root commands (check, install, dev, stop)
├── config.yaml                 # Main application configuration
├── extensions_config.json      # MCP servers and skills configuration
├── backend/                    # Backend application (this directory)
│   ├── Makefile               # Backend-only commands (dev, gateway, lint)
│   ├── langgraph.json         # LangGraph server configuration
│   ├── src/
│   │   ├── agents/            # LangGraph agent system
│   │   │   ├── lead_agent/    # Main agent (factory + system prompt)
│   │   │   ├── middlewares/   # 9 middleware components
│   │   │   ├── memory/        # Memory extraction, queue, prompts
│   │   │   └── thread_state.py # ThreadState schema
│   │   ├── gateway/           # FastAPI Gateway API
│   │   │   ├── app.py         # FastAPI application
│   │   │   └── routers/       # 6 route modules
│   │   ├── sandbox/           # Sandbox execution system
│   │   │   ├── local/         # Local filesystem provider
│   │   │   ├── sandbox.py     # Abstract Sandbox interface
│   │   │   ├── tools.py       # bash, ls, read/write/str_replace
│   │   │   └── middleware.py  # Sandbox lifecycle management
│   │   ├── subagents/         # Subagent delegation system
│   │   │   ├── builtins/      # general-purpose, bash agents
│   │   │   ├── executor.py    # Background execution engine
│   │   │   └── registry.py    # Agent registry
│   │   ├── tools/builtins/    # Built-in tools (present_files, ask_clarification, view_image)
│   │   ├── mcp/               # MCP integration (tools, cache, client)
│   │   ├── models/            # Model factory with thinking/vision support
│   │   ├── skills/            # Skills discovery, loading, parsing
│   │   ├── config/            # Configuration system (app, model, sandbox, tool, etc.)
│   │   ├── community/         # Community tools (tavily, jina_ai, firecrawl, image_search, aio_sandbox)
│   │   ├── reflection/        # Dynamic module loading (resolve_variable, resolve_class)
│   │   └── utils/             # Utilities (network, readability)
│   ├── tests/                 # Test suite
│   └── docs/                  # Documentation
├── frontend/                   # Next.js frontend application
└── skills/                     # Agent skills directory
    ├── public/                # Public skills (committed)
    └── custom/                # Custom skills (gitignored)

Important Development Guidelines

Documentation Update Policy

CRITICAL: Always update README.md and CLAUDE.md after every code change

When making code changes, you MUST update the relevant documentation:

• Update README.md for user-facing changes (features, setup, usage instructions)
• Update CLAUDE.md for development changes (architecture, commands, workflows, internal systems)
• Keep documentation synchronized with the codebase at all times
• Ensure accuracy and timeliness of all documentation

Commands

Root directory (for full application):

make check      # Check system requirements
make install    # Install all dependencies (frontend + backend)
make dev        # Start all services (LangGraph + Gateway + Frontend + Nginx)
make stop       # Stop all services

Backend directory (for backend development only):

make install    # Install backend dependencies
make dev        # Run LangGraph server only (port 2024)
make gateway    # Run Gateway API only (port 8001)
make lint       # Lint with ruff
make format     # Format code with ruff

Architecture

Agent System

Lead Agent (src/agents/lead_agent/agent.py):

• Entry point: make_lead_agent(config: RunnableConfig) registered in langgraph.json
• Dynamic model selection via create_chat_model() with thinking/vision support
• Tools loaded via get_available_tools() - combines sandbox, built-in, MCP, community, and subagent tools
• System prompt generated by apply_prompt_template() with skills, memory, and subagent instructions

ThreadState (src/agents/thread_state.py):

• Extends AgentState with: sandbox, thread_data, title, artifacts, todos, uploaded_files, viewed_images
• Uses custom reducers: merge_artifacts (deduplicate), merge_viewed_images (merge/clear)

Runtime Configuration (via config.configurable):

• thinking_enabled - Enable model's extended thinking
• model_name - Select specific LLM model
• is_plan_mode - Enable TodoList middleware
• subagent_enabled - Enable task delegation tool

Middleware Chain

Middlewares execute in strict order in src/agents/lead_agent/agent.py:

1. ThreadDataMiddleware - Creates per-thread directories (backend/.deer-flow/threads/{thread_id}/user-data/{workspace,uploads,outputs})
2. UploadsMiddleware - Tracks and injects newly uploaded files into conversation
3. SandboxMiddleware - Acquires sandbox, stores sandbox_id in state
4. SummarizationMiddleware - Context reduction when approaching token limits (optional, if enabled)
5. TodoListMiddleware - Task tracking with write_todos tool (optional, if plan_mode)
6. TitleMiddleware - Auto-generates thread title after first complete exchange
7. MemoryMiddleware - Queues conversations for async memory update (filters to user + final AI responses)
8. ViewImageMiddleware - Injects base64 image data before LLM call (conditional on vision support)
9. ClarificationMiddleware - Intercepts ask_clarification tool calls, interrupts via Command(goto=END) (must be last)

Configuration System

Main Configuration (config.yaml):

Setup: Copy config.example.yaml to config.yaml in the project root directory.

Configuration priority:

1. Explicit config_path argument
2. DEER_FLOW_CONFIG_PATH environment variable
3. config.yaml in current directory (backend/)
4. config.yaml in parent directory (project root - recommended location)

Config values starting with $ are resolved as environment variables (e.g., $OPENAI_API_KEY).

Extensions Configuration (extensions_config.json):

MCP servers and skills are configured together in extensions_config.json in project root:

Configuration priority:

1. Explicit config_path argument
2. DEER_FLOW_EXTENSIONS_CONFIG_PATH environment variable
3. extensions_config.json in current directory (backend/)
4. extensions_config.json in parent directory (project root - recommended location)

Gateway API (src/gateway/)

FastAPI application on port 8001 with health check at GET /health.

Routers:

Router	Endpoints
Models (/api/models)	GET / - list models; GET /{name} - model details
MCP (/api/mcp)	GET /config - get config; PUT /config - update config (saves to extensions_config.json)
Skills (/api/skills)	GET / - list skills; GET /{name} - details; PUT /{name} - update enabled; POST /install - install from .skill archive
Memory (/api/memory)	GET / - memory data; POST /reload - force reload; GET /config - config; GET /status - config + data
Uploads (/api/threads/{id}/uploads)	POST / - upload files (auto-converts PDF/PPT/Excel/Word); GET /list - list; DELETE /{filename} - delete
Artifacts (/api/threads/{id}/artifacts)	GET /{path} - serve artifacts; ?download=true for download with citation removal

Proxied through nginx: /api/langgraph/* → LangGraph, all other /api/* → Gateway.

Sandbox System (src/sandbox/)

Interface: Abstract Sandbox with execute_command, read_file, write_file, list_dir Provider Pattern: SandboxProvider with acquire, get, release lifecycle Implementations:

• LocalSandboxProvider - Singleton local filesystem execution with path mappings
• AioSandboxProvider (src/community/) - Docker-based isolation

Virtual Path System:

• Agent sees: /mnt/user-data/{workspace,uploads,outputs}, /mnt/skills
• Physical: backend/.deer-flow/threads/{thread_id}/user-data/..., deer-flow/skills/
• Translation: replace_virtual_path() / replace_virtual_paths_in_command()
• Detection: is_local_sandbox() checks sandbox_id == "local"

Sandbox Tools (in src/sandbox/tools.py):

• bash - Execute commands with path translation and error handling
• ls - Directory listing (tree format, max 2 levels)
• read_file - Read file contents with optional line range
• write_file - Write/append to files, creates directories
• str_replace - Substring replacement (single or all occurrences)

Subagent System (src/subagents/)

Built-in Agents: general-purpose (all tools except task) and bash (command specialist) Execution: Dual thread pool - _scheduler_pool (3 workers) + _execution_pool (3 workers) Concurrency: MAX_CONCURRENT_SUBAGENTS = 3 per trace, 15-minute timeout Flow: task() tool → SubagentExecutor → background thread → poll 5s → SSE events → result Events: task_started, task_running, task_completed/task_failed/task_timed_out

Tool System (src/tools/)

get_available_tools(groups, include_mcp, model_name, subagent_enabled) assembles:

1. Config-defined tools - Resolved from config.yaml via resolve_variable()
2. MCP tools - From enabled MCP servers (lazy initialized, cached with mtime invalidation)
3. Built-in tools:
   • present_files - Make output files visible to user (only /mnt/user-data/outputs)
   • ask_clarification - Request clarification (intercepted by ClarificationMiddleware → interrupts)
   • view_image - Read image as base64 (added only if model supports vision)
4. Subagent tool (if enabled):
   • task - Delegate to subagent (description, prompt, subagent_type, max_turns)

Community tools (src/community/):

• tavily/ - Web search (5 results default) and web fetch (4KB limit)
• jina_ai/ - Web fetch via Jina reader API with readability extraction
• firecrawl/ - Web scraping via Firecrawl API
• image_search/ - Image search via DuckDuckGo

MCP System (src/mcp/)

• Uses langchain-mcp-adapters MultiServerMCPClient for multi-server management
• Lazy initialization: Tools loaded on first use via get_cached_mcp_tools()
• Cache invalidation: Detects config file changes via mtime comparison
• Transports: stdio (command-based), SSE, HTTP
• Runtime updates: Gateway API saves to extensions_config.json; LangGraph detects via mtime

Skills System (src/skills/)

• Location: deer-flow/skills/{public,custom}/
• Format: Directory with SKILL.md (YAML frontmatter: name, description, license, allowed-tools)
• Loading: load_skills() scans directories, parses SKILL.md, reads enabled state from extensions_config.json
• Injection: Enabled skills listed in agent system prompt with container paths
• Installation: POST /api/skills/install extracts .skill ZIP archive to custom/ directory

Model Factory (src/models/factory.py)

• create_chat_model(name, thinking_enabled) instantiates LLM from config via reflection
• Supports thinking_enabled flag with per-model when_thinking_enabled overrides
• Supports supports_vision flag for image understanding models
• Config values starting with $ resolved as environment variables

Memory System (src/agents/memory/)

Components:

• updater.py - LLM-based memory updates with fact extraction and atomic file I/O
• queue.py - Debounced update queue (per-thread deduplication, configurable wait time)
• prompt.py - Prompt templates for memory updates

Data Structure (stored in backend/.deer-flow/memory.json):

• User Context: workContext, personalContext, topOfMind (1-3 sentence summaries)
• History: recentMonths, earlierContext, longTermBackground
• Facts: Discrete facts with id, content, category (preference/knowledge/context/behavior/goal), confidence (0-1), createdAt, source

Workflow:

1. MemoryMiddleware filters messages (user inputs + final AI responses) and queues conversation
2. Queue debounces (30s default), batches updates, deduplicates per-thread
3. Background thread invokes LLM to extract context updates and facts
4. Applies updates atomically (temp file + rename) with cache invalidation
5. Next interaction injects top 15 facts + context into <memory> tags in system prompt

Configuration (config.yaml → memory):

• enabled / injection_enabled - Master switches
• storage_path - Path to memory.json
• debounce_seconds - Wait time before processing (default: 30)
• model_name - LLM for updates (null = default model)
• max_facts / fact_confidence_threshold - Fact storage limits (100 / 0.7)
• max_injection_tokens - Token limit for prompt injection (2000)

Reflection System (src/reflection/)

• resolve_variable(path) - Import module and return variable (e.g., module.path:variable_name)
• resolve_class(path, base_class) - Import and validate class against base class

Config Schema

config.yaml key sections:

• models[] - LLM configs with use class path, supports_thinking, supports_vision, provider-specific fields
• tools[] - Tool configs with use variable path and group
• tool_groups[] - Logical groupings for tools
• sandbox.use - Sandbox provider class path
• skills.path / skills.container_path - Host and container paths to skills directory
• title - Auto-title generation (enabled, max_words, max_chars, prompt_template)
• summarization - Context summarization (enabled, trigger conditions, keep policy)
• subagents.enabled - Master switch for subagent delegation
• memory - Memory system (enabled, storage_path, debounce_seconds, model_name, max_facts, fact_confidence_threshold, injection_enabled, max_injection_tokens)

extensions_config.json:

• mcpServers - Map of server name → config (enabled, type, command, args, env, url, headers, description)
• skills - Map of skill name → state (enabled)

Both can be modified at runtime via Gateway API endpoints.

Development Workflow

Running the Full Application

From the project root directory:

make dev

This starts all services and makes the application available at http://localhost:2026.

Nginx routing:

• /api/langgraph/* → LangGraph Server (2024)
• /api/* (other) → Gateway API (8001)
• / (non-API) → Frontend (3000)

Running Backend Services Separately

From the backend directory:

# Terminal 1: LangGraph server
make dev

# Terminal 2: Gateway API
make gateway

Direct access (without nginx):

• LangGraph: http://localhost:2024
• Gateway: http://localhost:8001

Frontend Configuration

The frontend uses environment variables to connect to backend services:

• NEXT_PUBLIC_LANGGRAPH_BASE_URL - Defaults to /api/langgraph (through nginx)
• NEXT_PUBLIC_BACKEND_BASE_URL - Defaults to empty string (through nginx)

When using make dev from root, the frontend automatically connects through nginx.

Key Features

File Upload

Multi-file upload with automatic document conversion:

• Endpoint: POST /api/threads/{thread_id}/uploads
• Supports: PDF, PPT, Excel, Word documents (converted via markitdown)
• Files stored in thread-isolated directories
• Agent receives uploaded file list via UploadsMiddleware

See docs/FILE_UPLOAD.md for details.

Plan Mode

TodoList middleware for complex multi-step tasks:

• Controlled via runtime config: config.configurable.is_plan_mode = True
• Provides write_todos tool for task tracking
• One task in_progress at a time, real-time updates

See docs/plan_mode_usage.md for details.

Context Summarization

Automatic conversation summarization when approaching token limits:

• Configured in config.yaml under summarization key
• Trigger types: tokens, messages, or fraction of max input
• Keeps recent messages while summarizing older ones

See docs/summarization.md for details.

Vision Support

For models with supports_vision: true:

• ViewImageMiddleware processes images in conversation
• view_image_tool added to agent's toolset
• Images automatically converted to base64 and injected into state

Code Style

• Uses ruff for linting and formatting
• Line length: 240 characters
• Python 3.12+ with type hints
• Double quotes, space indentation

Documentation

See docs/ directory for detailed documentation:

• CONFIGURATION.md - Configuration options
• ARCHITECTURE.md - Architecture details
• API.md - API reference
• SETUP.md - Setup guide
• FILE_UPLOAD.md - File upload feature
• PATH_EXAMPLES.md - Path types and usage
• summarization.md - Context summarization
• plan_mode_usage.md - Plan mode with TodoList