refactor: split backend into harness (deerflow.*) and app (app.*) (#1131)

* refactor: extract shared utils to break harness→app cross-layer imports

Move _validate_skill_frontmatter to src/skills/validation.py and
CONVERTIBLE_EXTENSIONS + convert_file_to_markdown to src/utils/file_conversion.py.
This eliminates the two reverse dependencies from client.py (harness layer)
into gateway/routers/ (app layer), preparing for the harness/app package split.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: split backend/src into harness (deerflow.*) and app (app.*)

Physically split the monolithic backend/src/ package into two layers:

- **Harness** (`packages/harness/deerflow/`): publishable agent framework
  package with import prefix `deerflow.*`. Contains agents, sandbox, tools,
  models, MCP, skills, config, and all core infrastructure.

- **App** (`app/`): unpublished application code with import prefix `app.*`.
  Contains gateway (FastAPI REST API) and channels (IM integrations).

Key changes:
- Move 13 harness modules to packages/harness/deerflow/ via git mv
- Move gateway + channels to app/ via git mv
- Rename all imports: src.* → deerflow.* (harness) / app.* (app layer)
- Set up uv workspace with deerflow-harness as workspace member
- Update langgraph.json, config.example.yaml, all scripts, Docker files
- Add build-system (hatchling) to harness pyproject.toml
- Add PYTHONPATH=. to gateway startup commands for app.* resolution
- Update ruff.toml with known-first-party for import sorting
- Update all documentation to reflect new directory structure

Boundary rule enforced: harness code never imports from app.
All 429 tests pass. Lint clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add harness→app boundary check test and update docs

Add test_harness_boundary.py that scans all Python files in
packages/harness/deerflow/ and fails if any `from app.*` or
`import app.*` statement is found. This enforces the architectural
rule that the harness layer never depends on the app layer.

Update CLAUDE.md to document the harness/app split architecture,
import conventions, and the boundary enforcement test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add config versioning with auto-upgrade on startup

When config.example.yaml schema changes, developers' local config.yaml
files can silently become outdated. This adds a config_version field and
auto-upgrade mechanism so breaking changes (like src.* → deerflow.*
renames) are applied automatically before services start.

- Add config_version: 1 to config.example.yaml
- Add startup version check warning in AppConfig.from_file()
- Add scripts/config-upgrade.sh with migration registry for value replacements
- Add `make config-upgrade` target
- Auto-run config-upgrade in serve.sh and start-daemon.sh before starting services
- Add config error hints in service failure messages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix comments

* fix: update src.* import in test_sandbox_tools_security to deerflow.*

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: handle empty config and search parent dirs for config.example.yaml

Address Copilot review comments on PR #1131:
- Guard against yaml.safe_load() returning None for empty config files
- Search parent directories for config.example.yaml instead of only
  looking next to config.yaml, fixing detection in common setups

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: correct skills root path depth and config_version type coercion

- loader.py: fix get_skills_root_path() to use 5 parent levels (was 3)
  after harness split, file lives at packages/harness/deerflow/skills/
  so parent×3 resolved to backend/packages/harness/ instead of backend/
- app_config.py: coerce config_version to int() before comparison in
  _check_config_version() to prevent TypeError when YAML stores value
  as string (e.g. config_version: "1")
- tests: add regression tests for both fixes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: update test imports from src.* to deerflow.*/app.* after harness refactor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
DanielWalnut
2026-03-14 22:55:52 +08:00
committed by GitHub
parent 9b49a80dda
commit 76803b826f
198 changed files with 1786 additions and 941 deletions

View File

@@ -143,12 +143,12 @@ Root-level orchestration and config:
Backend core:
- `backend/src/agents/` - lead agent, middleware chain, memory
- `backend/src/gateway/` - FastAPI gateway API
- `backend/src/sandbox/` - sandbox provider + tool wrappers
- `backend/src/subagents/` - subagent registry/execution
- `backend/src/mcp/` - MCP integration
- `backend/langgraph.json` - graph entrypoint (`src.agents:make_lead_agent`)
- `backend/packages/harness/deerflow/agents/` - lead agent, middleware chain, memory
- `backend/app/gateway/` - FastAPI gateway API
- `backend/packages/harness/deerflow/sandbox/` - sandbox provider + tool wrappers
- `backend/packages/harness/deerflow/subagents/` - subagent registry/execution
- `backend/packages/harness/deerflow/mcp/` - MCP integration
- `backend/langgraph.json` - graph entrypoint (`deerflow.agents:make_lead_agent`)
- `backend/pyproject.toml` - Python deps and `requires-python`
- `backend/ruff.toml` - lint/format policy
- `backend/tests/` - backend unit and integration-like tests

1
.gitignore vendored
View File

@@ -51,3 +51,4 @@ web/
# Deployment artifacts
backend/Dockerfile.langgraph
config.yaml.bak

View File

@@ -1,12 +1,13 @@
# DeerFlow - Unified Development Environment
.PHONY: help config check install dev dev-daemon start stop up down clean docker-init docker-start docker-stop docker-logs docker-logs-frontend docker-logs-gateway
.PHONY: help config config-upgrade check install dev dev-daemon start stop up down clean docker-init docker-start docker-stop docker-logs docker-logs-frontend docker-logs-gateway
PYTHON ?= python
help:
@echo "DeerFlow Development Commands:"
@echo " make config - Generate local config files (aborts if config already exists)"
@echo " make config-upgrade - Merge new fields from config.example.yaml into config.yaml"
@echo " make check - Check if all required tools are installed"
@echo " make install - Install all dependencies (frontend + backend)"
@echo " make setup-sandbox - Pre-pull sandbox container image (recommended)"
@@ -31,6 +32,9 @@ help:
config:
@$(PYTHON) ./scripts/configure.py
config-upgrade:
@./scripts/config-upgrade.sh
# Check required tools
check:
@$(PYTHON) ./scripts/check.py
@@ -96,7 +100,7 @@ dev-daemon:
stop:
@echo "Stopping all services..."
@-pkill -f "langgraph dev" 2>/dev/null || true
@-pkill -f "uvicorn src.gateway.app:app" 2>/dev/null || true
@-pkill -f "uvicorn app.gateway.app:app" 2>/dev/null || true
@-pkill -f "next dev" 2>/dev/null || true
@-pkill -f "next start" 2>/dev/null || true
@-pkill -f "next-server" 2>/dev/null || true

View File

@@ -145,7 +145,7 @@ make docker-init # Pull sandbox image (only once or when image updates)
make docker-start # Start services (auto-detects sandbox mode from config.yaml)
```
`make docker-start` starts `provisioner` only when `config.yaml` uses provisioner mode (`sandbox.use: src.community.aio_sandbox:AioSandboxProvider` with `provisioner_url`).
`make docker-start` starts `provisioner` only when `config.yaml` uses provisioner mode (`sandbox.use: deerflow.community.aio_sandbox:AioSandboxProvider` with `provisioner_url`).
**Production** (builds images locally, mounts runtime config and data):
@@ -437,7 +437,7 @@ DeerFlow is model-agnostic — it works with any LLM that implements the OpenAI-
DeerFlow can be used as an embedded Python library without running the full HTTP services. The `DeerFlowClient` provides direct in-process access to all agent and Gateway capabilities, returning the same response schemas as the HTTP Gateway API:
```python
from src.client import DeerFlowClient
from deerflow.client import DeerFlowClient
client = DeerFlowClient()
@@ -456,7 +456,7 @@ client.update_skill("web-search", enabled=True)
client.upload_files("thread-1", ["./report.pdf"]) # {"success": True, "files": [...]}
```
All dict-returning methods are validated against Gateway Pydantic response models in CI (`TestGatewayConformance`), ensuring the embedded client stays in sync with the HTTP API schemas. See `backend/src/client.py` for full API documentation.
All dict-returning methods are validated against Gateway Pydantic response models in CI (`TestGatewayConformance`), ensuring the embedded client stays in sync with the HTTP API schemas. See `backend/packages/harness/deerflow/client.py` for full API documentation.
## Documentation

View File

@@ -22,33 +22,38 @@ deer-flow/
├── backend/ # Backend application (this directory)
│ ├── Makefile # Backend-only commands (dev, gateway, lint)
│ ├── langgraph.json # LangGraph server configuration
│ ├── src/
│ │ ── agents/ # LangGraph agent system
│ │ ├── lead_agent/ # Main agent (factory + system prompt)
│ │ ── middlewares/ # 10 middleware components
│ │ ├── memory/ # Memory extraction, queue, prompts
│ │ │ ── thread_state.py # ThreadState schema
│ ├── packages/
│ │ ── harness/ # deerflow-harness package (import: deerflow.*)
│ │ ├── pyproject.toml
│ │ ── deerflow/
│ │ ├── agents/ # LangGraph agent system
│ │ ── lead_agent/ # Main agent (factory + system prompt)
│ │ │ ├── middlewares/ # 10 middleware components
│ │ │ ├── memory/ # Memory extraction, queue, prompts
│ │ │ └── thread_state.py # ThreadState schema
│ │ ├── sandbox/ # Sandbox execution system
│ │ │ ├── local/ # Local filesystem provider
│ │ │ ├── sandbox.py # Abstract Sandbox interface
│ │ │ ├── tools.py # bash, ls, read/write/str_replace
│ │ │ └── middleware.py # Sandbox lifecycle management
│ │ ├── subagents/ # Subagent delegation system
│ │ │ ├── builtins/ # general-purpose, bash agents
│ │ │ ├── executor.py # Background execution engine
│ │ │ └── registry.py # Agent registry
│ │ ├── tools/builtins/ # Built-in tools (present_files, ask_clarification, view_image)
│ │ ├── mcp/ # MCP integration (tools, cache, client)
│ │ ├── models/ # Model factory with thinking/vision support
│ │ ├── skills/ # Skills discovery, loading, parsing
│ │ ├── config/ # Configuration system (app, model, sandbox, tool, etc.)
│ │ ├── community/ # Community tools (tavily, jina_ai, firecrawl, image_search, aio_sandbox)
│ │ ├── reflection/ # Dynamic module loading (resolve_variable, resolve_class)
│ │ ├── utils/ # Utilities (network, readability)
│ │ └── client.py # Embedded Python client (DeerFlowClient)
│ ├── app/ # Application layer (import: app.*)
│ │ ├── gateway/ # FastAPI Gateway API
│ │ │ ├── app.py # FastAPI application
│ │ │ └── routers/ # 6 route modules
│ │ ── sandbox/ # Sandbox execution system
│ │ │ ├── local/ # Local filesystem provider
│ │ │ ├── sandbox.py # Abstract Sandbox interface
│ │ │ ├── tools.py # bash, ls, read/write/str_replace
│ │ │ └── middleware.py # Sandbox lifecycle management
│ │ ├── subagents/ # Subagent delegation system
│ │ │ ├── builtins/ # general-purpose, bash agents
│ │ │ ├── executor.py # Background execution engine
│ │ │ └── registry.py # Agent registry
│ │ ├── tools/builtins/ # Built-in tools (present_files, ask_clarification, view_image)
│ │ ├── mcp/ # MCP integration (tools, cache, client)
│ │ ├── models/ # Model factory with thinking/vision support
│ │ ├── skills/ # Skills discovery, loading, parsing
│ │ ├── config/ # Configuration system (app, model, sandbox, tool, etc.)
│ │ ├── community/ # Community tools (tavily, jina_ai, firecrawl, image_search, aio_sandbox)
│ │ ├── reflection/ # Dynamic module loading (resolve_variable, resolve_class)
│ │ ├── utils/ # Utilities (network, readability)
│ │ └── client.py # Embedded Python client (DeerFlowClient)
│ │ ── channels/ # IM platform integrations
│ ├── tests/ # Test suite
│ └── docs/ # Documentation
├── frontend/ # Next.js frontend application
@@ -92,19 +97,48 @@ Regression tests related to Docker/provisioner behavior:
- `tests/test_docker_sandbox_mode_detection.py` (mode detection from `config.yaml`)
- `tests/test_provisioner_kubeconfig.py` (kubeconfig file/directory handling)
Boundary check (harness → app import firewall):
- `tests/test_harness_boundary.py` — ensures `packages/harness/deerflow/` never imports from `app.*`
CI runs these regression tests for every pull request via [.github/workflows/backend-unit-tests.yml](../.github/workflows/backend-unit-tests.yml).
## Architecture
### Harness / App Split
The backend is split into two layers with a strict dependency direction:
- **Harness** (`packages/harness/deerflow/`): Publishable agent framework package (`deerflow-harness`). Import prefix: `deerflow.*`. Contains agent orchestration, tools, sandbox, models, MCP, skills, config — everything needed to build and run agents.
- **App** (`app/`): Unpublished application code. Import prefix: `app.*`. Contains the FastAPI Gateway API and IM channel integrations (Feishu, Slack, Telegram).
**Dependency rule**: App imports deerflow, but deerflow never imports app. This boundary is enforced by `tests/test_harness_boundary.py` which runs in CI.
**Import conventions**:
```python
# Harness internal
from deerflow.agents import make_lead_agent
from deerflow.models import create_chat_model
# App internal
from app.gateway.app import app
from app.channels.service import start_channel_service
# App → Harness (allowed)
from deerflow.config import get_app_config
# Harness → App (FORBIDDEN — enforced by test_harness_boundary.py)
# from app.gateway.routers.uploads import ... # ← will fail CI
```
### Agent System
**Lead Agent** (`src/agents/lead_agent/agent.py`):
**Lead Agent** (`packages/harness/deerflow/agents/lead_agent/agent.py`):
- Entry point: `make_lead_agent(config: RunnableConfig)` registered in `langgraph.json`
- Dynamic model selection via `create_chat_model()` with thinking/vision support
- Tools loaded via `get_available_tools()` - combines sandbox, built-in, MCP, community, and subagent tools
- System prompt generated by `apply_prompt_template()` with skills, memory, and subagent instructions
**ThreadState** (`src/agents/thread_state.py`):
**ThreadState** (`packages/harness/deerflow/agents/thread_state.py`):
- Extends `AgentState` with: `sandbox`, `thread_data`, `title`, `artifacts`, `todos`, `uploaded_files`, `viewed_images`
- Uses custom reducers: `merge_artifacts` (deduplicate), `merge_viewed_images` (merge/clear)
@@ -116,7 +150,7 @@ CI runs these regression tests for every pull request via [.github/workflows/bac
### Middleware Chain
Middlewares execute in strict order in `src/agents/lead_agent/agent.py`:
Middlewares execute in strict order in `packages/harness/deerflow/agents/lead_agent/agent.py`:
1. **ThreadDataMiddleware** - Creates per-thread directories (`backend/.deer-flow/threads/{thread_id}/user-data/{workspace,uploads,outputs}`)
2. **UploadsMiddleware** - Tracks and injects newly uploaded files into conversation
@@ -136,6 +170,8 @@ Middlewares execute in strict order in `src/agents/lead_agent/agent.py`:
Setup: Copy `config.example.yaml` to `config.yaml` in the **project root** directory.
**Config Versioning**: `config.example.yaml` has a `config_version` field. On startup, `AppConfig.from_file()` compares user version vs example version and emits a warning if outdated. Missing `config_version` = version 0. Run `make config-upgrade` to auto-merge missing fields. When changing the config schema, bump `config_version` in `config.example.yaml`.
Configuration priority:
1. Explicit `config_path` argument
2. `DEER_FLOW_CONFIG_PATH` environment variable
@@ -154,7 +190,7 @@ Configuration priority:
3. `extensions_config.json` in current directory (backend/)
4. `extensions_config.json` in parent directory (project root - **recommended location**)
### Gateway API (`src/gateway/`)
### Gateway API (`app/gateway/`)
FastAPI application on port 8001 with health check at `GET /health`.
@@ -172,13 +208,13 @@ FastAPI application on port 8001 with health check at `GET /health`.
Proxied through nginx: `/api/langgraph/*` → LangGraph, all other `/api/*` → Gateway.
### Sandbox System (`src/sandbox/`)
### Sandbox System (`packages/harness/deerflow/sandbox/`)
**Interface**: Abstract `Sandbox` with `execute_command`, `read_file`, `write_file`, `list_dir`
**Provider Pattern**: `SandboxProvider` with `acquire`, `get`, `release` lifecycle
**Implementations**:
- `LocalSandboxProvider` - Singleton local filesystem execution with path mappings
- `AioSandboxProvider` (`src/community/`) - Docker-based isolation
- `AioSandboxProvider` (`packages/harness/deerflow/community/`) - Docker-based isolation
**Virtual Path System**:
- Agent sees: `/mnt/user-data/{workspace,uploads,outputs}`, `/mnt/skills`
@@ -186,14 +222,14 @@ Proxied through nginx: `/api/langgraph/*` → LangGraph, all other `/api/*` →
- Translation: `replace_virtual_path()` / `replace_virtual_paths_in_command()`
- Detection: `is_local_sandbox()` checks `sandbox_id == "local"`
**Sandbox Tools** (in `src/sandbox/tools.py`):
**Sandbox Tools** (in `packages/harness/deerflow/sandbox/tools.py`):
- `bash` - Execute commands with path translation and error handling
- `ls` - Directory listing (tree format, max 2 levels)
- `read_file` - Read file contents with optional line range
- `write_file` - Write/append to files, creates directories
- `str_replace` - Substring replacement (single or all occurrences)
### Subagent System (`src/subagents/`)
### Subagent System (`packages/harness/deerflow/subagents/`)
**Built-in Agents**: `general-purpose` (all tools except `task`) and `bash` (command specialist)
**Execution**: Dual thread pool - `_scheduler_pool` (3 workers) + `_execution_pool` (3 workers)
@@ -201,7 +237,7 @@ Proxied through nginx: `/api/langgraph/*` → LangGraph, all other `/api/*` →
**Flow**: `task()` tool → `SubagentExecutor` → background thread → poll 5s → SSE events → result
**Events**: `task_started`, `task_running`, `task_completed`/`task_failed`/`task_timed_out`
### Tool System (`src/tools/`)
### Tool System (`packages/harness/deerflow/tools/`)
`get_available_tools(groups, include_mcp, model_name, subagent_enabled)` assembles:
1. **Config-defined tools** - Resolved from `config.yaml` via `resolve_variable()`
@@ -213,13 +249,13 @@ Proxied through nginx: `/api/langgraph/*` → LangGraph, all other `/api/*` →
4. **Subagent tool** (if enabled):
- `task` - Delegate to subagent (description, prompt, subagent_type, max_turns)
**Community tools** (`src/community/`):
**Community tools** (`packages/harness/deerflow/community/`):
- `tavily/` - Web search (5 results default) and web fetch (4KB limit)
- `jina_ai/` - Web fetch via Jina reader API with readability extraction
- `firecrawl/` - Web scraping via Firecrawl API
- `image_search/` - Image search via DuckDuckGo
### MCP System (`src/mcp/`)
### MCP System (`packages/harness/deerflow/mcp/`)
- Uses `langchain-mcp-adapters` `MultiServerMCPClient` for multi-server management
- **Lazy initialization**: Tools loaded on first use via `get_cached_mcp_tools()`
@@ -228,7 +264,7 @@ Proxied through nginx: `/api/langgraph/*` → LangGraph, all other `/api/*` →
- **OAuth (HTTP/SSE)**: Supports token endpoint flows (`client_credentials`, `refresh_token`) with automatic token refresh + Authorization header injection
- **Runtime updates**: Gateway API saves to extensions_config.json; LangGraph detects via mtime
### Skills System (`src/skills/`)
### Skills System (`packages/harness/deerflow/skills/`)
- **Location**: `deer-flow/skills/{public,custom}/`
- **Format**: Directory with `SKILL.md` (YAML frontmatter: name, description, license, allowed-tools)
@@ -236,7 +272,7 @@ Proxied through nginx: `/api/langgraph/*` → LangGraph, all other `/api/*` →
- **Injection**: Enabled skills listed in agent system prompt with container paths
- **Installation**: `POST /api/skills/install` extracts .skill ZIP archive to custom/ directory
### Model Factory (`src/models/factory.py`)
### Model Factory (`packages/harness/deerflow/models/factory.py`)
- `create_chat_model(name, thinking_enabled)` instantiates LLM from config via reflection
- Supports `thinking_enabled` flag with per-model `when_thinking_enabled` overrides
@@ -244,7 +280,7 @@ Proxied through nginx: `/api/langgraph/*` → LangGraph, all other `/api/*` →
- Config values starting with `$` resolved as environment variables
- Missing provider modules surface actionable install hints from reflection resolvers (for example `uv add langchain-google-genai`)
### IM Channels System (`src/channels/`)
### IM Channels System (`app/channels/`)
Bridges external messaging platforms (Feishu, Slack, Telegram) to the DeerFlow agent via the LangGraph Server.
@@ -273,7 +309,7 @@ Bridges external messaging platforms (Feishu, Slack, Telegram) to the DeerFlow a
- `gateway_url` - Gateway API URL for auxiliary commands (default: `http://localhost:8001`)
- Per-channel configs: `feishu` (app_id, app_secret), `slack` (bot_token, app_token), `telegram` (bot_token)
### Memory System (`src/agents/memory/`)
### Memory System (`packages/harness/deerflow/agents/memory/`)
**Components**:
- `updater.py` - LLM-based memory updates with fact extraction and atomic file I/O
@@ -300,7 +336,7 @@ Bridges external messaging platforms (Feishu, Slack, Telegram) to the DeerFlow a
- `max_facts` / `fact_confidence_threshold` - Fact storage limits (100 / 0.7)
- `max_injection_tokens` - Token limit for prompt injection (2000)
### Reflection System (`src/reflection/`)
### Reflection System (`packages/harness/deerflow/reflection/`)
- `resolve_variable(path)` - Import module and return variable (e.g., `module.path:variable_name`)
- `resolve_class(path, base_class)` - Import and validate class against base class
@@ -324,11 +360,11 @@ Bridges external messaging platforms (Feishu, Slack, Telegram) to the DeerFlow a
Both can be modified at runtime via Gateway API endpoints or `DeerFlowClient` methods.
### Embedded Client (`src/client.py`)
### Embedded Client (`packages/harness/deerflow/client.py`)
`DeerFlowClient` provides direct in-process access to all DeerFlow capabilities without HTTP services. All return types align with the Gateway API response schemas, so consumer code works identically in HTTP and embedded modes.
**Architecture**: Imports the same `src/` modules that LangGraph Server and Gateway API use. Shares the same config files and data directories. No FastAPI dependency.
**Architecture**: Imports the same `deerflow` modules that LangGraph Server and Gateway API use. Shares the same config files and data directories. No FastAPI dependency.
**Agent Conversation** (replaces LangGraph Server):
- `chat(message, thread_id)` — synchronous, returns final text
@@ -367,7 +403,7 @@ Both can be modified at runtime via Gateway API endpoints or `DeerFlowClient` me
- Run the full suite before and after your change: `make test`
- Tests must pass before a feature is considered complete
- For lightweight config/utility modules, prefer pure unit tests with no external dependencies
- If a module causes circular import issues in tests, add a `sys.modules` mock in `tests/conftest.py` (see existing example for `src.subagents.executor`)
- If a module causes circular import issues in tests, add a `sys.modules` mock in `tests/conftest.py` (see existing example for `deerflow.subagents.executor`)
```bash
# Run all tests

View File

@@ -227,7 +227,7 @@ Example test:
```python
import pytest
from src.models.factory import create_chat_model
from deerflow.models.factory import create_chat_model
def test_create_chat_model_with_valid_name():
"""Test that a valid model name creates a model instance."""
@@ -269,10 +269,10 @@ Include in your PR description:
### Adding New Tools
1. Create tool in `src/tools/builtins/` or `src/community/`:
1. Create tool in `packages/harness/deerflow/tools/builtins/` or `packages/harness/deerflow/community/`:
```python
# src/tools/builtins/my_tool.py
# packages/harness/deerflow/tools/builtins/my_tool.py
from langchain_core.tools import tool
@tool
@@ -294,15 +294,15 @@ def my_tool(param: str) -> str:
tools:
- name: my_tool
group: my_group
use: src.tools.builtins.my_tool:my_tool
use: deerflow.tools.builtins.my_tool:my_tool
```
### Adding New Middleware
1. Create middleware in `src/agents/middlewares/`:
1. Create middleware in `packages/harness/deerflow/agents/middlewares/`:
```python
# src/agents/middlewares/my_middleware.py
# packages/harness/deerflow/agents/middlewares/my_middleware.py
from langchain.agents.middleware import BaseMiddleware
from langchain_core.runnables import RunnableConfig
@@ -315,7 +315,7 @@ class MyMiddleware(BaseMiddleware):
return state
```
2. Register in `src/agents/lead_agent/agent.py`:
2. Register in `packages/harness/deerflow/agents/lead_agent/agent.py`:
```python
middlewares = [
@@ -329,10 +329,10 @@ middlewares = [
### Adding New API Endpoints
1. Create router in `src/gateway/routers/`:
1. Create router in `app/gateway/routers/`:
```python
# src/gateway/routers/my_router.py
# app/gateway/routers/my_router.py
from fastapi import APIRouter
router = APIRouter(prefix="/my-endpoint", tags=["my-endpoint"])
@@ -348,10 +348,10 @@ async def create_item(data: dict):
return {"created": data}
```
2. Register in `src/gateway/app.py`:
2. Register in `app/gateway/app.py`:
```python
from src.gateway.routers import my_router
from app.gateway.routers import my_router
app.include_router(my_router.router)
```
@@ -360,7 +360,7 @@ app.include_router(my_router.router)
When adding new configuration options:
1. Update `src/config/app_config.py` with new fields
1. Update `packages/harness/deerflow/config/app_config.py` with new fields
2. Add default values in `config.example.yaml`
3. Document in `docs/CONFIGURATION.md`

View File

@@ -36,4 +36,4 @@ RUN --mount=type=cache,target=/root/.cache/uv \
EXPOSE 8001 2024
# Default command (can be overridden in docker-compose)
CMD ["sh", "-c", "uv run uvicorn src.gateway.app:app --host 0.0.0.0 --port 8001"]
CMD ["sh", "-c", "cd backend && PYTHONPATH=. uv run uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001"]

View File

@@ -5,7 +5,7 @@ dev:
uv run langgraph dev --no-browser --allow-blocking --no-reload
gateway:
uv run uvicorn src.gateway.app:app --host 0.0.0.0 --port 8001
PYTHONPATH=. uv run uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001
test:
PYTHONPATH=. uv run pytest tests/ -v

View File

@@ -5,8 +5,8 @@ Provides a pluggable channel system that connects external messaging platforms
which uses ``langgraph-sdk`` to communicate with the underlying LangGraph Server.
"""
from src.channels.base import Channel
from src.channels.message_bus import InboundMessage, MessageBus, OutboundMessage
from app.channels.base import Channel
from app.channels.message_bus import InboundMessage, MessageBus, OutboundMessage
__all__ = [
"Channel",

View File

@@ -6,7 +6,7 @@ import logging
from abc import ABC, abstractmethod
from typing import Any
from src.channels.message_bus import InboundMessage, InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
from app.channels.message_bus import InboundMessage, InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
logger = logging.getLogger(__name__)

View File

@@ -8,8 +8,8 @@ import logging
import threading
from typing import Any
from src.channels.base import Channel
from src.channels.message_bus import InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
from app.channels.base import Channel
from app.channels.message_bus import InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
logger = logging.getLogger(__name__)

View File

@@ -9,8 +9,8 @@ import time
from collections.abc import Mapping
from typing import Any
from src.channels.message_bus import InboundMessage, InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
from src.channels.store import ChannelStore
from app.channels.message_bus import InboundMessage, InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
from app.channels.store import ChannelStore
logger = logging.getLogger(__name__)
@@ -242,7 +242,7 @@ def _resolve_attachments(thread_id: str, artifacts: list[str]) -> list[ResolvedA
Skips artifacts that cannot be resolved (missing files, invalid paths)
and logs warnings for them.
"""
from src.config.paths import get_paths
from deerflow.config.paths import get_paths
attachments: list[ResolvedAttachment] = []
paths = get_paths()
@@ -266,14 +266,16 @@ def _resolve_attachments(thread_id: str, artifacts: list[str]) -> list[ResolvedA
continue
mime, _ = mimetypes.guess_type(str(actual))
mime = mime or "application/octet-stream"
attachments.append(ResolvedAttachment(
virtual_path=virtual_path,
actual_path=actual,
filename=actual.name,
mime_type=mime,
size=actual.stat().st_size,
is_image=mime.startswith("image/"),
))
attachments.append(
ResolvedAttachment(
virtual_path=virtual_path,
actual_path=actual,
filename=actual.name,
mime_type=mime,
size=actual.stat().st_size,
is_image=mime.startswith("image/"),
)
)
except (ValueError, OSError) as exc:
logger.warning("[Manager] failed to resolve artifact %s: %s", virtual_path, exc)
return attachments
@@ -348,12 +350,7 @@ class ChannelManager:
def _resolve_run_params(self, msg: InboundMessage, thread_id: str) -> tuple[str, dict[str, Any], dict[str, Any]]:
channel_layer, user_layer = self._resolve_session_layer(msg)
assistant_id = (
user_layer.get("assistant_id")
or channel_layer.get("assistant_id")
or self._default_session.get("assistant_id")
or self._assistant_id
)
assistant_id = user_layer.get("assistant_id") or channel_layer.get("assistant_id") or self._default_session.get("assistant_id") or self._assistant_id
if not isinstance(assistant_id, str) or not assistant_id.strip():
assistant_id = self._assistant_id

View File

@@ -5,17 +5,17 @@ from __future__ import annotations
import logging
from typing import Any
from src.channels.manager import ChannelManager
from src.channels.message_bus import MessageBus
from src.channels.store import ChannelStore
from app.channels.manager import ChannelManager
from app.channels.message_bus import MessageBus
from app.channels.store import ChannelStore
logger = logging.getLogger(__name__)
# Channel name → import path for lazy loading
_CHANNEL_REGISTRY: dict[str, str] = {
"feishu": "src.channels.feishu:FeishuChannel",
"slack": "src.channels.slack:SlackChannel",
"telegram": "src.channels.telegram:TelegramChannel",
"feishu": "app.channels.feishu:FeishuChannel",
"slack": "app.channels.slack:SlackChannel",
"telegram": "app.channels.telegram:TelegramChannel",
}
@@ -49,7 +49,7 @@ class ChannelService:
@classmethod
def from_app_config(cls) -> ChannelService:
"""Create a ChannelService from the application config."""
from src.config.app_config import get_app_config
from deerflow.config.app_config import get_app_config
config = get_app_config()
channels_config = {}
@@ -116,7 +116,7 @@ class ChannelService:
return False
try:
from src.reflection import resolve_class
from deerflow.reflection import resolve_class
channel_cls = resolve_class(import_path, base_class=None)
except Exception:

View File

@@ -8,8 +8,8 @@ from typing import Any
from markdown_to_mrkdwn import SlackMarkdownConverter
from src.channels.base import Channel
from src.channels.message_bus import InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
from app.channels.base import Channel
from app.channels.message_bus import InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
logger = logging.getLogger(__name__)

View File

@@ -35,7 +35,7 @@ class ChannelStore:
def __init__(self, path: str | Path | None = None) -> None:
if path is None:
from src.config.paths import get_paths
from deerflow.config.paths import get_paths
path = Path(get_paths().base_dir) / "channels" / "store.json"
self._path = Path(path)

View File

@@ -7,8 +7,8 @@ import logging
import threading
from typing import Any
from src.channels.base import Channel
from src.channels.message_bus import InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
from app.channels.base import Channel
from app.channels.message_bus import InboundMessageType, MessageBus, OutboundMessage, ResolvedAttachment
logger = logging.getLogger(__name__)

View File

@@ -4,9 +4,8 @@ from contextlib import asynccontextmanager
from fastapi import FastAPI
from src.config.app_config import get_app_config
from src.gateway.config import get_gateway_config
from src.gateway.routers import (
from app.gateway.config import get_gateway_config
from app.gateway.routers import (
agents,
artifacts,
channels,
@@ -17,6 +16,7 @@ from src.gateway.routers import (
suggestions,
uploads,
)
from deerflow.config.app_config import get_app_config
# Configure logging
logging.basicConfig(
@@ -50,7 +50,7 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
# Start IM channel service if any channels are configured
try:
from src.channels.service import start_channel_service
from app.channels.service import start_channel_service
channel_service = await start_channel_service()
logger.info("Channel service started: %s", channel_service.get_status())
@@ -61,7 +61,7 @@ async def lifespan(app: FastAPI) -> AsyncGenerator[None, None]:
# Stop channel service on shutdown
try:
from src.channels.service import stop_channel_service
from app.channels.service import stop_channel_service
await stop_channel_service()
except Exception:

View File

@@ -4,7 +4,7 @@ from pathlib import Path
from fastapi import HTTPException
from src.config.paths import get_paths
from deerflow.config.paths import get_paths
def resolve_thread_virtual_path(thread_id: str, virtual_path: str) -> Path:

View File

@@ -8,8 +8,8 @@ import yaml
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel, Field
from src.config.agents_config import AgentConfig, list_custom_agents, load_agent_config, load_agent_soul
from src.config.paths import get_paths
from deerflow.config.agents_config import AgentConfig, list_custom_agents, load_agent_config, load_agent_soul
from deerflow.config.paths import get_paths
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["agents"])

View File

@@ -7,7 +7,7 @@ from urllib.parse import quote
from fastapi import APIRouter, HTTPException, Request
from fastapi.responses import FileResponse, HTMLResponse, PlainTextResponse, Response
from src.gateway.path_utils import resolve_thread_virtual_path
from app.gateway.path_utils import resolve_thread_virtual_path
logger = logging.getLogger(__name__)

View File

@@ -25,7 +25,7 @@ class ChannelRestartResponse(BaseModel):
@router.get("/", response_model=ChannelStatusResponse)
async def get_channels_status() -> ChannelStatusResponse:
"""Get the status of all IM channels."""
from src.channels.service import get_channel_service
from app.channels.service import get_channel_service
service = get_channel_service()
if service is None:
@@ -37,7 +37,7 @@ async def get_channels_status() -> ChannelStatusResponse:
@router.post("/{name}/restart", response_model=ChannelRestartResponse)
async def restart_channel(name: str) -> ChannelRestartResponse:
"""Restart a specific IM channel."""
from src.channels.service import get_channel_service
from app.channels.service import get_channel_service
service = get_channel_service()
if service is None:

View File

@@ -6,7 +6,7 @@ from typing import Literal
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel, Field
from src.config.extensions_config import ExtensionsConfig, get_extensions_config, reload_extensions_config
from deerflow.config.extensions_config import ExtensionsConfig, get_extensions_config, reload_extensions_config
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api", tags=["mcp"])

View File

@@ -3,8 +3,8 @@
from fastapi import APIRouter
from pydantic import BaseModel, Field
from src.agents.memory.updater import get_memory_data, reload_memory_data
from src.config.memory_config import get_memory_config
from deerflow.agents.memory.updater import get_memory_data, reload_memory_data
from deerflow.config.memory_config import get_memory_config
router = APIRouter(prefix="/api", tags=["memory"])

View File

@@ -1,7 +1,7 @@
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel, Field
from src.config import get_app_config
from deerflow.config import get_app_config
router = APIRouter(prefix="/api", tags=["models"])

View File

@@ -1,22 +1,19 @@
import json
import logging
import re
import shutil
import stat
import tempfile
import zipfile
from collections.abc import Mapping
from pathlib import Path
from typing import cast
import yaml
from fastapi import APIRouter, HTTPException
from pydantic import BaseModel, Field
from src.config.extensions_config import ExtensionsConfig, SkillStateConfig, get_extensions_config, reload_extensions_config
from src.gateway.path_utils import resolve_thread_virtual_path
from src.skills import Skill, load_skills
from src.skills.loader import get_skills_root_path
from app.gateway.path_utils import resolve_thread_virtual_path
from deerflow.config.extensions_config import ExtensionsConfig, SkillStateConfig, get_extensions_config, reload_extensions_config
from deerflow.skills import Skill, load_skills
from deerflow.skills.loader import get_skills_root_path
from deerflow.skills.validation import _validate_skill_frontmatter
logger = logging.getLogger(__name__)
@@ -129,23 +126,6 @@ class SkillInstallResponse(BaseModel):
message: str = Field(..., description="Installation result message")
# Allowed properties in SKILL.md frontmatter
ALLOWED_FRONTMATTER_PROPERTIES = {
"name",
"description",
"license",
"allowed-tools",
"metadata",
"compatibility",
"version",
"author",
}
def _safe_load_frontmatter(frontmatter_text: str) -> object:
return cast(object, yaml.safe_load(frontmatter_text))
def _should_ignore_archive_entry(path: Path) -> bool:
return path.name.startswith(".") or path.name == "__MACOSX"
@@ -159,81 +139,6 @@ def _resolve_skill_dir_from_archive_root(temp_path: Path) -> Path:
return temp_path
def _validate_skill_frontmatter(skill_dir: Path) -> tuple[bool, str, str | None]:
"""Validate a skill directory's SKILL.md frontmatter.
Args:
skill_dir: Path to the skill directory containing SKILL.md.
Returns:
Tuple of (is_valid, message, skill_name).
"""
skill_md = skill_dir / "SKILL.md"
if not skill_md.exists():
return False, "SKILL.md not found", None
content = skill_md.read_text()
if not content.startswith("---"):
return False, "No YAML frontmatter found", None
# Extract frontmatter
match = re.match(r"^---\n(.*?)\n---", content, re.DOTALL)
if not match:
return False, "Invalid frontmatter format", None
frontmatter_text = match.group(1)
# Parse YAML frontmatter
try:
parsed_frontmatter = _safe_load_frontmatter(frontmatter_text)
if not isinstance(parsed_frontmatter, Mapping):
return False, "Frontmatter must be a YAML dictionary", None
parsed_frontmatter = cast(Mapping[object, object], parsed_frontmatter)
frontmatter: dict[str, object] = {str(key): value for key, value in parsed_frontmatter.items()}
except yaml.YAMLError as e:
return False, f"Invalid YAML in frontmatter: {e}", None
# Check for unexpected properties
unexpected_keys = set(frontmatter.keys()) - ALLOWED_FRONTMATTER_PROPERTIES
if unexpected_keys:
return False, f"Unexpected key(s) in SKILL.md frontmatter: {', '.join(sorted(unexpected_keys))}", None
# Check required fields
if "name" not in frontmatter:
return False, "Missing 'name' in frontmatter", None
if "description" not in frontmatter:
return False, "Missing 'description' in frontmatter", None
# Validate name
name = frontmatter.get("name", "")
if not isinstance(name, str):
return False, f"Name must be a string, got {type(name).__name__}", None
name = name.strip()
if not name:
return False, "Name cannot be empty", None
# Check naming convention (hyphen-case: lowercase with hyphens)
if not re.match(r"^[a-z0-9-]+$", name):
return False, f"Name '{name}' should be hyphen-case (lowercase letters, digits, and hyphens only)", None
if name.startswith("-") or name.endswith("-") or "--" in name:
return False, f"Name '{name}' cannot start/end with hyphen or contain consecutive hyphens", None
if len(name) > 64:
return False, f"Name is too long ({len(name)} characters). Maximum is 64 characters.", None
# Validate description
description = frontmatter.get("description", "")
if not isinstance(description, str):
return False, f"Description must be a string, got {type(description).__name__}", None
description = description.strip()
if description:
if "<" in description or ">" in description:
return False, "Description cannot contain angle brackets (< or >)", None
if len(description) > 1024:
return False, f"Description is too long ({len(description)} characters). Maximum is 1024 characters.", None
return True, "Skill is valid!", name
def _skill_to_response(skill: Skill) -> SkillResponse:
"""Convert a Skill object to a SkillResponse."""
return SkillResponse(

View File

@@ -4,7 +4,7 @@ import logging
from fastapi import APIRouter
from pydantic import BaseModel, Field
from src.models import create_chat_model
from deerflow.models import create_chat_model
logger = logging.getLogger(__name__)

View File

@@ -6,24 +6,14 @@ from pathlib import Path
from fastapi import APIRouter, File, HTTPException, UploadFile
from pydantic import BaseModel
from src.config.paths import VIRTUAL_PATH_PREFIX, get_paths
from src.sandbox.sandbox_provider import get_sandbox_provider
from deerflow.config.paths import VIRTUAL_PATH_PREFIX, get_paths
from deerflow.sandbox.sandbox_provider import get_sandbox_provider
from deerflow.utils.file_conversion import CONVERTIBLE_EXTENSIONS, convert_file_to_markdown
logger = logging.getLogger(__name__)
router = APIRouter(prefix="/api/threads/{thread_id}/uploads", tags=["uploads"])
# File extensions that should be converted to markdown
CONVERTIBLE_EXTENSIONS = {
".pdf",
".ppt",
".pptx",
".xls",
".xlsx",
".doc",
".docx",
}
class UploadResponse(BaseModel):
"""Response model for file upload."""
@@ -47,32 +37,6 @@ def get_uploads_dir(thread_id: str) -> Path:
return base_dir
async def convert_file_to_markdown(file_path: Path) -> Path | None:
"""Convert a file to markdown using markitdown.
Args:
file_path: Path to the file to convert.
Returns:
Path to the markdown file if conversion was successful, None otherwise.
"""
try:
from markitdown import MarkItDown
md = MarkItDown()
result = md.convert(str(file_path))
# Save as .md file with same name
md_path = file_path.with_suffix(".md")
md_path.write_text(result.text_content, encoding="utf-8")
logger.info(f"Converted {file_path.name} to markdown: {md_path.name}")
return md_path
except Exception as e:
logger.error(f"Failed to convert {file_path.name} to markdown: {e}")
return None
@router.post("", response_model=UploadResponse)
async def upload_files(
thread_id: str,

View File

@@ -3,6 +3,12 @@
Debug script for lead_agent.
Run this file directly in VS Code with breakpoints.
Requirements:
Run with `uv run` from the backend/ directory so that the uv workspace
resolves deerflow-harness and app packages correctly:
cd backend && PYTHONPATH=. uv run python debug.py
Usage:
1. Set breakpoints in agent.py or other files
2. Press F5 or use "Run and Debug" panel
@@ -11,21 +17,14 @@ Usage:
import asyncio
import logging
import os
import sys
# Ensure we can import from src
sys.path.insert(0, os.path.dirname(os.path.abspath(__file__)))
# Load environment variables
from dotenv import load_dotenv
from langchain_core.messages import HumanMessage
from src.agents import make_lead_agent
from deerflow.agents import make_lead_agent
load_dotenv()
# Configure logging
logging.basicConfig(
level=logging.INFO,
format="%(asctime)s - %(name)s - %(levelname)s - %(message)s",
@@ -36,7 +35,7 @@ logging.basicConfig(
async def main():
# Initialize MCP tools at startup
try:
from src.mcp import initialize_mcp_tools
from deerflow.mcp import initialize_mcp_tools
await initialize_mcp_tools()
except Exception as e:

View File

@@ -80,7 +80,7 @@ docker stop <id> # Auto-removes due to --rm
### Implementation Details
The implementation is in `backend/src/community/aio_sandbox/aio_sandbox_provider.py`:
The implementation is in `backend/packages/harness/deerflow/community/aio_sandbox/aio_sandbox_provider.py`:
- `_detect_container_runtime()`: Detects available runtime at startup
- `_start_container()`: Uses detected runtime, skips Docker-specific options for Apple Container
@@ -93,14 +93,14 @@ No configuration changes are needed! The system works automatically.
However, you can verify the runtime in use by checking the logs:
```
INFO:src.community.aio_sandbox.aio_sandbox_provider:Detected Apple Container: container version 0.1.0
INFO:src.community.aio_sandbox.aio_sandbox_provider:Starting sandbox container using container: ...
INFO:deerflow.community.aio_sandbox.aio_sandbox_provider:Detected Apple Container: container version 0.1.0
INFO:deerflow.community.aio_sandbox.aio_sandbox_provider:Starting sandbox container using container: ...
```
Or for Docker:
```
INFO:src.community.aio_sandbox.aio_sandbox_provider:Apple Container not available, falling back to Docker
INFO:src.community.aio_sandbox.aio_sandbox_provider:Starting sandbox container using docker: ...
INFO:deerflow.community.aio_sandbox.aio_sandbox_provider:Apple Container not available, falling back to Docker
INFO:deerflow.community.aio_sandbox.aio_sandbox_provider:Starting sandbox container using docker: ...
```
## Container Images
@@ -109,7 +109,7 @@ Both runtimes use OCI-compatible images. The default image works with both:
```yaml
sandbox:
use: src.community.aio_sandbox:AioSandboxProvider
use: deerflow.community.aio_sandbox:AioSandboxProvider
image: enterprise-public-cn-beijing.cr.volces.com/vefaas-public/all-in-one-sandbox:latest # Default image
```

View File

@@ -55,7 +55,7 @@ This document provides a comprehensive overview of the DeerFlow backend architec
The LangGraph server is the core agent runtime, built on LangGraph for robust multi-agent workflow orchestration.
**Entry Point**: `src/agents/lead_agent/agent.py:make_lead_agent`
**Entry Point**: `packages/harness/deerflow/agents/lead_agent/agent.py:make_lead_agent`
**Key Responsibilities**:
- Agent creation and configuration
@@ -70,7 +70,7 @@ The LangGraph server is the core agent runtime, built on LangGraph for robust mu
{
"agent": {
"type": "agent",
"path": "src.agents:make_lead_agent"
"path": "deerflow.agents:make_lead_agent"
}
}
```
@@ -79,7 +79,7 @@ The LangGraph server is the core agent runtime, built on LangGraph for robust mu
FastAPI application providing REST endpoints for non-agent operations.
**Entry Point**: `src/gateway/app.py`
**Entry Point**: `app/gateway/app.py`
**Routers**:
- `models.py` - `/api/models` - Model listing and details
@@ -158,7 +158,7 @@ class ThreadState(AgentState):
▼ ▼
┌─────────────────────────┐ ┌─────────────────────────┐
│ LocalSandboxProvider │ │ AioSandboxProvider │
│ (src/sandbox/local.py) │ │ (src/community/) │
│ (packages/harness/deerflow/sandbox/local.py) │ │ (packages/harness/deerflow/community/) │
│ │ │ │
│ - Singleton instance │ │ - Docker-based │
│ - Direct execution │ │ - Isolated containers │
@@ -192,7 +192,7 @@ class ThreadState(AgentState):
┌─────────────────────┐ ┌─────────────────────┐ ┌─────────────────────┐
│ Built-in Tools │ │ Configured Tools │ │ MCP Tools │
│ (src/tools/) │ │ (config.yaml) │ │ (extensions.json) │
│ (packages/harness/deerflow/tools/) │ │ (config.yaml) │ │ (extensions.json) │
├─────────────────────┤ ├─────────────────────┤ ├─────────────────────┤
│ - present_file │ │ - web_search │ │ - github │
│ - ask_clarification │ │ - web_fetch │ │ - filesystem │
@@ -208,7 +208,7 @@ class ThreadState(AgentState):
┌─────────────────────────┐
│ get_available_tools() │
│ (src/tools/__init__) │
│ (packages/harness/deerflow/tools/__init__) │
└─────────────────────────┘
```
@@ -217,7 +217,7 @@ class ThreadState(AgentState):
```
┌─────────────────────────────────────────────────────────────────────────┐
│ Model Factory │
│ (src/models/factory.py) │
│ (packages/harness/deerflow/models/factory.py) │
└─────────────────────────────────────────────────────────────────────────┘
config.yaml:
@@ -264,7 +264,7 @@ config.yaml:
```
┌─────────────────────────────────────────────────────────────────────────┐
│ MCP Integration │
│ (src/mcp/manager.py) │
│ (packages/harness/deerflow/mcp/manager.py) │
└─────────────────────────────────────────────────────────────────────────┘
extensions_config.json:
@@ -302,7 +302,7 @@ extensions_config.json:
```
┌─────────────────────────────────────────────────────────────────────────┐
│ Skills System │
│ (src/skills/loader.py) │
│ (packages/harness/deerflow/skills/loader.py) │
└─────────────────────────────────────────────────────────────────────────┘
Directory Structure:

View File

@@ -50,7 +50,7 @@ checkpointer = PostgresSaver.from_conn_string(
```json
{
"graphs": {
"lead_agent": "src.agents:lead_agent"
"lead_agent": "deerflow.agents:lead_agent"
},
"checkpointer": "checkpointer:checkpointer"
}
@@ -71,7 +71,7 @@ title:
或在代码中配置:
```python
from src.config.title_config import TitleConfig, set_title_config
from deerflow.config.title_config import TitleConfig, set_title_config
set_title_config(TitleConfig(
enabled=True,
@@ -185,7 +185,7 @@ sequenceDiagram
```python
# 测试 title 生成
import pytest
from src.agents.title_middleware import TitleMiddleware
from deerflow.agents.title_middleware import TitleMiddleware
def test_title_generation():
# TODO: 添加单元测试
@@ -243,11 +243,11 @@ def after_agent(self, state: TitleMiddlewareState, runtime: Runtime) -> dict | N
## 相关文件
- [`src/agents/thread_state.py`](../src/agents/thread_state.py) - ThreadState 定义
- [`src/agents/title_middleware.py`](../src/agents/title_middleware.py) - TitleMiddleware 实现
- [`src/config/title_config.py`](../src/config/title_config.py) - 配置管理
- [`packages/harness/deerflow/agents/thread_state.py`](../packages/harness/deerflow/agents/thread_state.py) - ThreadState 定义
- [`packages/harness/deerflow/agents/title_middleware.py`](../packages/harness/deerflow/agents/title_middleware.py) - TitleMiddleware 实现
- [`packages/harness/deerflow/config/title_config.py`](../packages/harness/deerflow/config/title_config.py) - 配置管理
- [`config.yaml`](../config.yaml) - 配置文件
- [`src/agents/lead_agent/agent.py`](../src/agents/lead_agent/agent.py) - Middleware 注册
- [`packages/harness/deerflow/agents/lead_agent/agent.py`](../packages/harness/deerflow/agents/lead_agent/agent.py) - Middleware 注册
## 参考资料

View File

@@ -2,6 +2,19 @@
This guide explains how to configure DeerFlow for your environment.
## Config Versioning
`config.example.yaml` contains a `config_version` field that tracks schema changes. When the example version is higher than your local `config.yaml`, the application emits a startup warning:
```
WARNING - Your config.yaml (version 0) is outdated — the latest version is 1.
Run `make config-upgrade` to merge new fields into your config.
```
- **Missing `config_version`** in your config is treated as version 0.
- Run `make config-upgrade` to auto-merge missing fields (your existing values are preserved, a `.bak` backup is created).
- When changing the config schema, bump `config_version` in `config.example.yaml`.
## Configuration Sections
### Models
@@ -103,7 +116,7 @@ Configure specific tools available to the agent:
tools:
- name: web_search
group: web
use: src.community.tavily.tools:web_search_tool
use: deerflow.community.tavily.tools:web_search_tool
max_results: 5
# api_key: $TAVILY_API_KEY # Optional
```
@@ -124,13 +137,13 @@ DeerFlow supports multiple sandbox execution modes. Configure your preferred mod
**Local Execution** (runs sandbox code directly on the host machine):
```yaml
sandbox:
use: src.sandbox.local:LocalSandboxProvider # Local execution
use: deerflow.sandbox.local:LocalSandboxProvider # Local execution
```
**Docker Execution** (runs sandbox code in isolated Docker containers):
```yaml
sandbox:
use: src.community.aio_sandbox:AioSandboxProvider # Docker-based sandbox
use: deerflow.community.aio_sandbox:AioSandboxProvider # Docker-based sandbox
```
**Docker Execution with Kubernetes** (runs sandbox code in Kubernetes pods via provisioner service):
@@ -139,7 +152,7 @@ This mode runs each sandbox in an isolated Kubernetes Pod on your **host machine
```yaml
sandbox:
use: src.community.aio_sandbox:AioSandboxProvider
use: deerflow.community.aio_sandbox:AioSandboxProvider
provisioner_url: http://provisioner:8002
```
@@ -152,13 +165,13 @@ Choose between local execution or Docker-based isolation:
**Option 1: Local Sandbox** (default, simpler setup):
```yaml
sandbox:
use: src.sandbox.local:LocalSandboxProvider
use: deerflow.sandbox.local:LocalSandboxProvider
```
**Option 2: Docker Sandbox** (isolated, more secure):
```yaml
sandbox:
use: src.community.aio_sandbox:AioSandboxProvider
use: deerflow.community.aio_sandbox:AioSandboxProvider
port: 8080
auto_start: true
container_prefix: deer-flow-sandbox

View File

@@ -212,11 +212,11 @@ backend/.deer-flow/threads/
### 组件
1. **Upload Router** (`src/gateway/routers/uploads.py`)
1. **Upload Router** (`app/gateway/routers/uploads.py`)
- 处理文件上传、列表、删除请求
- 使用 markitdown 转换文档
2. **Uploads Middleware** (`src/agents/middlewares/uploads_middleware.py`)
2. **Uploads Middleware** (`packages/harness/deerflow/agents/middlewares/uploads_middleware.py`)
- 在每次 Agent 请求前注入文件列表
- 自动生成格式化的文件列表消息

View File

@@ -0,0 +1,343 @@
# DeerFlow 后端拆分设计文档Harness + App
> 状态Draft
> 作者DeerFlow Team
> 日期2026-03-13
## 1. 背景与动机
DeerFlow 后端当前是一个单一 Python 包(`src.*`),包含了从底层 agent 编排到上层用户产品的所有代码。随着项目发展,这种结构带来了几个问题:
- **复用困难**其他产品CLI 工具、Slack bot、第三方集成想用 agent 能力,必须依赖整个后端,包括 FastAPI、IM SDK 等不需要的依赖
- **职责模糊**agent 编排逻辑和用户产品逻辑混在同一个 `src/` 下,边界不清晰
- **依赖膨胀**LangGraph Server 运行时不需要 FastAPI/uvicorn/Slack SDK但当前必须安装全部依赖
本文档提出将后端拆分为两部分:**deerflow-harness**(可发布的 agent 框架包)和 **app**(不打包的用户产品代码)。
## 2. 核心概念
### 2.1 Harness线束/框架层)
Harness 是 agent 的构建与编排框架,回答 **"如何构建和运行 agent"** 的问题:
- Agent 工厂与生命周期管理
- Middleware pipeline
- 工具系统(内置工具 + MCP + 社区工具)
- 沙箱执行环境
- 子 agent 委派
- 记忆系统
- 技能加载与注入
- 模型工厂
- 配置系统
**Harness 是一个可发布的 Python 包**`deerflow-harness`),可以独立安装和使用。
**Harness 的设计原则**:对上层应用完全无感知。它不知道也不关心谁在调用它——可以是 Web App、CLI、Slack Bot、或者一个单元测试。
### 2.2 App应用层
App 是面向用户的产品代码,回答 **"如何将 agent 呈现给用户"** 的问题:
- Gateway APIFastAPI REST 接口)
- IM Channels飞书、Slack、Telegram 集成)
- Custom Agent 的 CRUD 管理
- 文件上传/下载的 HTTP 接口
**App 不打包、不发布**,它是 DeerFlow 项目内部的应用代码,直接运行。
**App 依赖 Harness但 Harness 不依赖 App。**
### 2.3 边界划分
| 模块 | 归属 | 说明 |
|------|------|------|
| `config/` | Harness | 配置系统是基础设施 |
| `reflection/` | Harness | 动态模块加载工具 |
| `utils/` | Harness | 通用工具函数 |
| `agents/` | Harness | Agent 工厂、middleware、state、memory |
| `subagents/` | Harness | 子 agent 委派系统 |
| `sandbox/` | Harness | 沙箱执行环境 |
| `tools/` | Harness | 工具注册与发现 |
| `mcp/` | Harness | MCP 协议集成 |
| `skills/` | Harness | 技能加载、解析、定义 schema |
| `models/` | Harness | LLM 模型工厂 |
| `community/` | Harness | 社区工具tavily、jina 等) |
| `client.py` | Harness | 嵌入式 Python 客户端 |
| `gateway/` | App | FastAPI REST API |
| `channels/` | App | IM 平台集成 |
**关于 Custom Agents**agent 定义格式(`config.yaml` + `SOUL.md` schema由 Harness 层的 `config/agents_config.py` 定义但文件的存储、CRUD、发现机制由 App 层的 `gateway/routers/agents.py` 负责。
## 3. 目标架构
### 3.1 目录结构
```
backend/
├── packages/
│ └── harness/
│ ├── pyproject.toml # deerflow-harness 包定义
│ └── deerflow/ # Python 包根import 前缀: deerflow.*
│ ├── __init__.py
│ ├── config/
│ ├── reflection/
│ ├── utils/
│ ├── agents/
│ │ ├── lead_agent/
│ │ ├── middlewares/
│ │ ├── memory/
│ │ ├── checkpointer/
│ │ └── thread_state.py
│ ├── subagents/
│ ├── sandbox/
│ ├── tools/
│ ├── mcp/
│ ├── skills/
│ ├── models/
│ ├── community/
│ └── client.py
├── app/ # 不打包import 前缀: app.*
│ ├── __init__.py
│ ├── gateway/
│ │ ├── __init__.py
│ │ ├── app.py
│ │ ├── config.py
│ │ ├── path_utils.py
│ │ └── routers/
│ └── channels/
│ ├── __init__.py
│ ├── base.py
│ ├── manager.py
│ ├── service.py
│ ├── store.py
│ ├── message_bus.py
│ ├── feishu.py
│ ├── slack.py
│ └── telegram.py
├── pyproject.toml # uv workspace root
├── langgraph.json
├── tests/
├── docs/
└── Makefile
```
### 3.2 Import 规则
两个层使用不同的 import 前缀,职责边界一目了然:
```python
# ---------------------------------------------------------------
# Harness 内部互相引用deerflow.* 前缀)
# ---------------------------------------------------------------
from deerflow.agents import make_lead_agent
from deerflow.models import create_chat_model
from deerflow.config import get_app_config
from deerflow.tools import get_available_tools
# ---------------------------------------------------------------
# App 内部互相引用app.* 前缀)
# ---------------------------------------------------------------
from app.gateway.app import app
from app.gateway.routers.uploads import upload_files
from app.channels.service import start_channel_service
# ---------------------------------------------------------------
# App 调用 Harness单向依赖Harness 永远不 import app
# ---------------------------------------------------------------
from deerflow.agents import make_lead_agent
from deerflow.models import create_chat_model
from deerflow.skills import load_skills
from deerflow.config.extensions_config import get_extensions_config
```
**App 调用 Harness 示例 — Gateway 中启动 agent**
```python
# app/gateway/routers/chat.py
from deerflow.agents.lead_agent.agent import make_lead_agent
from deerflow.models import create_chat_model
from deerflow.config import get_app_config
async def create_chat_session(thread_id: str, model_name: str):
config = get_app_config()
model = create_chat_model(name=model_name)
agent = make_lead_agent(config=...)
# ... 使用 agent 处理用户消息
```
**App 调用 Harness 示例 — Channel 中查询 skills**
```python
# app/channels/manager.py
from deerflow.skills import load_skills
from deerflow.agents.memory.updater import get_memory_data
def handle_status_command():
skills = load_skills(enabled_only=True)
memory = get_memory_data()
return f"Skills: {len(skills)}, Memory facts: {len(memory.get('facts', []))}"
```
**禁止方向**Harness 代码中绝不能出现 `from app.``import app.`
### 3.3 为什么 App 不打包
| 方面 | 打包(放 packages/ 下) | 不打包(放 backend/app/ |
|------|------------------------|--------------------------|
| 命名空间 | 需要 pkgutil `extend_path` 合并,或独立前缀 | 天然独立,`app.*` vs `deerflow.*` |
| 发布需求 | 没有——App 是项目内部代码 | 不需要 pyproject.toml |
| 复杂度 | 需要管理两个包的构建、版本、依赖声明 | 直接运行,零额外配置 |
| 运行方式 | `pip install deerflow-app` | `PYTHONPATH=. uvicorn app.gateway.app:app` |
App 的唯一消费者是 DeerFlow 项目自身,没有独立发布的需求。放在 `backend/app/` 下作为普通 Python 包,通过 `PYTHONPATH` 或 editable install 让 Python 找到即可。
### 3.4 依赖关系
```
┌─────────────────────────────────────┐
│ app/ (不打包,直接运行) │
│ ├── fastapi, uvicorn │
│ ├── slack-sdk, lark-oapi, ... │
│ └── import deerflow.* │
└──────────────┬──────────────────────┘
┌─────────────────────────────────────┐
│ deerflow-harness (可发布的包) │
│ ├── langgraph, langchain │
│ ├── markitdown, pydantic, ... │
│ └── 零 app 依赖 │
└─────────────────────────────────────┘
```
**依赖分类**
| 分类 | 依赖包 |
|------|--------|
| Harness only | agent-sandbox, langchain*, langgraph*, markdownify, markitdown, pydantic, pyyaml, readabilipy, tavily-python, firecrawl-py, tiktoken, ddgs, duckdb, httpx, kubernetes, dotenv |
| App only | fastapi, uvicorn, sse-starlette, python-multipart, lark-oapi, slack-sdk, python-telegram-bot, markdown-to-mrkdwn |
| Shared | langgraph-sdkchannels 用 HTTP client, pydantic, httpx |
### 3.5 Workspace 配置
`backend/pyproject.toml`workspace root
```toml
[project]
name = "deer-flow"
version = "0.1.0"
requires-python = ">=3.12"
dependencies = ["deerflow-harness"]
[dependency-groups]
dev = ["pytest>=8.0.0", "ruff>=0.14.11"]
# App 的额外依赖fastapi 等)也声明在 workspace root因为 app 不打包
app = ["fastapi", "uvicorn", "sse-starlette", "python-multipart"]
channels = ["lark-oapi", "slack-sdk", "python-telegram-bot"]
[tool.uv.workspace]
members = ["packages/harness"]
[tool.uv.sources]
deerflow-harness = { workspace = true }
```
## 4. 当前的跨层依赖问题
在拆分之前,需要先解决 `client.py` 中两处从 harness 到 app 的反向依赖:
### 4.1 `_validate_skill_frontmatter`
```python
# client.py — harness 导入了 app 层代码
from src.gateway.routers.skills import _validate_skill_frontmatter
```
**解决方案**:将该函数提取到 `deerflow/skills/validation.py`。这是一个纯逻辑函数(解析 YAML frontmatter、校验字段与 FastAPI 无关。
### 4.2 `CONVERTIBLE_EXTENSIONS` + `convert_file_to_markdown`
```python
# client.py — harness 导入了 app 层代码
from src.gateway.routers.uploads import CONVERTIBLE_EXTENSIONS, convert_file_to_markdown
```
**解决方案**:将它们提取到 `deerflow/utils/file_conversion.py`。仅依赖 `markitdown` + `pathlib`,是通用工具函数。
## 5. 基础设施变更
### 5.1 LangGraph Server
LangGraph Server 只需要 harness 包。`langgraph.json` 更新:
```json
{
"dependencies": ["./packages/harness"],
"graphs": {
"lead_agent": "deerflow.agents:make_lead_agent"
},
"checkpointer": {
"path": "./packages/harness/deerflow/agents/checkpointer/async_provider.py:make_checkpointer"
}
}
```
### 5.2 Gateway API
```bash
# serve.sh / Makefile
# PYTHONPATH 包含 backend/ 根目录,使 app.* 和 deerflow.* 都能被找到
PYTHONPATH=. uvicorn app.gateway.app:app --host 0.0.0.0 --port 8001
```
### 5.3 Nginx
无需变更(只做 URL 路由,不涉及 Python 模块路径)。
### 5.4 Docker
Dockerfile 中的 module 引用从 `src.` 改为 `deerflow.` / `app.``COPY` 命令需覆盖 `packages/``app/` 目录。
## 6. 实施计划
分 3 个 PR 递进执行:
### PR 1提取共享工具函数Low Risk
1. 创建 `src/skills/validation.py`,从 `gateway/routers/skills.py` 提取 `_validate_skill_frontmatter`
2. 创建 `src/utils/file_conversion.py`,从 `gateway/routers/uploads.py` 提取文件转换逻辑
3. 更新 `client.py``gateway/routers/skills.py``gateway/routers/uploads.py` 的 import
4. 运行全部测试确认无回归
### PR 2Rename + 物理拆分High Risk原子操作
1. 创建 `packages/harness/` 目录,创建 `pyproject.toml`
2. `git mv` 将 harness 相关模块从 `src/` 移入 `packages/harness/deerflow/`
3. `git mv` 将 app 相关模块从 `src/` 移入 `app/`
4. 全局替换 import
- harness 模块:`src.*``deerflow.*`(所有 `.py` 文件、`langgraph.json`、测试、文档)
- app 模块:`src.gateway.*``app.gateway.*``src.channels.*``app.channels.*`
5. 更新 workspace root `pyproject.toml`
6. 更新 `langgraph.json``Makefile``Dockerfile`
7. `uv sync` + 全部测试 + 手动验证服务启动
### PR 3边界检查 + 文档Low Risk
1. 添加 lint 规则:检查 harness 不 import app 模块
2. 更新 `CLAUDE.md``README.md`
## 7. 风险与缓解
| 风险 | 影响 | 缓解措施 |
|------|------|----------|
| 全局 rename 误伤 | 字符串中的 `src` 被错误替换 | 正则精确匹配 `\bsrc\.`review diff |
| LangGraph Server 找不到模块 | 服务启动失败 | `langgraph.json``dependencies` 指向正确的 harness 包路径 |
| App 的 `PYTHONPATH` 缺失 | Gateway/Channel 启动 import 报错 | Makefile/Docker 统一设置 `PYTHONPATH=.` |
| `config.yaml` 中的 `use` 字段引用旧路径 | 运行时模块解析失败 | `config.yaml` 中的 `use` 字段同步更新为 `deerflow.*` |
| 测试中 `sys.path` 混乱 | 测试失败 | 用 editable install`uv sync`)确保 deerflow 可导入,`conftest.py` 中添加 `app/``sys.path` |
## 8. 未来演进
- **独立发布**harness 可以发布到内部 PyPI让其他项目直接 `pip install deerflow-harness`
- **插件化 App**:不同的 appweb、CLI、bot可以各自独立都依赖同一个 harness
- **更细粒度拆分**:如果 harness 内部模块继续增长,可以进一步拆分(如 `deerflow-sandbox``deerflow-mcp`

View File

@@ -33,6 +33,6 @@ No `current_context` argument is currently available in `main`.
## Verification Pointers
- Implementation: `backend/src/agents/memory/prompt.py`
- Prompt assembly: `backend/src/agents/lead_agent/prompt.py`
- Implementation: `packages/harness/deerflow/agents/memory/prompt.py`
- Prompt assembly: `packages/harness/deerflow/agents/lead_agent/prompt.py`
- Regression tests: `backend/tests/test_memory_prompt_injection.py`

View File

@@ -144,7 +144,7 @@ async function uploadAndProcess(threadId: string, file: File) {
```python
from pathlib import Path
from src.agents.middlewares.thread_data_middleware import THREAD_DATA_BASE_DIR
from deerflow.agents.middlewares.thread_data_middleware import THREAD_DATA_BASE_DIR
def process_uploaded_file(thread_id: str, filename: str):
# 使用实际路径

View File

@@ -30,7 +30,7 @@ DeerFlow uses a YAML configuration file that should be placed in the **project r
4. **Verify configuration**:
```bash
cd backend
python -c "from src.config import get_app_config; print('✓ Config loaded:', get_app_config().models[0].name)"
python -c "from deerflow.config import get_app_config; print('✓ Config loaded:', get_app_config().models[0].name)"
```
## Important Notes
@@ -51,7 +51,7 @@ The backend searches for `config.yaml` in this order:
## Sandbox Setup (Optional but Recommended)
If you plan to use Docker/Container-based sandbox (configured in `config.yaml` under `sandbox.use: src.community.aio_sandbox:AioSandboxProvider`), it's highly recommended to pre-pull the container image:
If you plan to use Docker/Container-based sandbox (configured in `config.yaml` under `sandbox.use: deerflow.community.aio_sandbox:AioSandboxProvider`), it's highly recommended to pre-pull the container image:
```bash
# From project root
@@ -72,7 +72,7 @@ If you skip this step, the image will be automatically pulled on first agent exe
```bash
# Check where the backend is looking
cd deer-flow/backend
python -c "from src.config.app_config import AppConfig; print(AppConfig.resolve_config_path())"
python -c "from deerflow.config.app_config import AppConfig; print(AppConfig.resolve_config_path())"
```
If it can't find the config:

View File

@@ -4,27 +4,27 @@
### 1. 核心实现文件
#### [`src/agents/thread_state.py`](../src/agents/thread_state.py)
#### [`packages/harness/deerflow/agents/thread_state.py`](../packages/harness/deerflow/agents/thread_state.py)
- ✅ 添加 `title: str | None = None` 字段到 `ThreadState`
#### [`src/config/title_config.py`](../src/config/title_config.py) (新建)
#### [`packages/harness/deerflow/config/title_config.py`](../packages/harness/deerflow/config/title_config.py) (新建)
- ✅ 创建 `TitleConfig` 配置类
- ✅ 支持配置enabled, max_words, max_chars, model_name, prompt_template
- ✅ 提供 `get_title_config()``set_title_config()` 函数
- ✅ 提供 `load_title_config_from_dict()` 从配置文件加载
#### [`src/agents/title_middleware.py`](../src/agents/title_middleware.py) (新建)
#### [`packages/harness/deerflow/agents/title_middleware.py`](../packages/harness/deerflow/agents/title_middleware.py) (新建)
- ✅ 创建 `TitleMiddleware`
- ✅ 实现 `_should_generate_title()` 检查是否需要生成
- ✅ 实现 `_generate_title()` 调用 LLM 生成标题
- ✅ 实现 `after_agent()` 钩子,在首次对话后自动触发
- ✅ 包含 fallback 策略LLM 失败时使用用户消息前几个词)
#### [`src/config/app_config.py`](../src/config/app_config.py)
#### [`packages/harness/deerflow/config/app_config.py`](../packages/harness/deerflow/config/app_config.py)
- ✅ 导入 `load_title_config_from_dict`
- ✅ 在 `from_file()` 中加载 title 配置
#### [`src/agents/lead_agent/agent.py`](../src/agents/lead_agent/agent.py)
#### [`packages/harness/deerflow/agents/lead_agent/agent.py`](../packages/harness/deerflow/agents/lead_agent/agent.py)
- ✅ 导入 `TitleMiddleware`
- ✅ 注册到 `middleware` 列表:`[SandboxMiddleware(), TitleMiddleware()]`
@@ -131,7 +131,7 @@ checkpointer = SqliteSaver.from_conn_string("checkpoints.db")
// langgraph.json
{
"graphs": {
"lead_agent": "src.agents:lead_agent"
"lead_agent": "deerflow.agents:lead_agent"
},
"checkpointer": "checkpointer:checkpointer"
}

View File

@@ -21,8 +21,8 @@
- [ ] Support for more document formats in upload
- [ ] Skill marketplace / remote skill installation
- [ ] Optimize async concurrency in agent hot path (IM channels multi-task scenario)
- Replace `time.sleep(5)` with `asyncio.sleep()` in `src/tools/builtins/task_tool.py` (subagent polling)
- Replace `subprocess.run()` with `asyncio.create_subprocess_shell()` in `src/sandbox/local/local_sandbox.py`
- Replace `time.sleep(5)` with `asyncio.sleep()` in `packages/harness/deerflow/tools/builtins/task_tool.py` (subagent polling)
- Replace `subprocess.run()` with `asyncio.create_subprocess_shell()` in `packages/harness/deerflow/sandbox/local/local_sandbox.py`
- Replace sync `requests` with `httpx.AsyncClient` in community tools (tavily, jina_ai, firecrawl, infoquest, image_search)
- Replace sync `model.invoke()` with async `model.ainvoke()` in title_middleware and memory updater
- Consider `asyncio.to_thread()` wrapper for remaining blocking file I/O

View File

@@ -19,7 +19,7 @@ Plan mode is controlled via **runtime configuration** through the `is_plan_mode`
```python
from langchain_core.runnables import RunnableConfig
from src.agents.lead_agent.agent import make_lead_agent
from deerflow.agents.lead_agent.agent import make_lead_agent
# Enable plan mode via runtime configuration
config = RunnableConfig(
@@ -72,7 +72,7 @@ The agent will skip using the todo list for:
```python
from langchain_core.runnables import RunnableConfig
from src.agents.lead_agent.agent import make_lead_agent
from deerflow.agents.lead_agent.agent import make_lead_agent
# Create agent with plan mode ENABLED
config_with_plan_mode = RunnableConfig(
@@ -101,7 +101,7 @@ You can enable/disable plan mode dynamically for different conversations or task
```python
from langchain_core.runnables import RunnableConfig
from src.agents.lead_agent.agent import make_lead_agent
from deerflow.agents.lead_agent.agent import make_lead_agent
def create_agent_for_task(task_complexity: str):
"""Create agent with plan mode based on task complexity."""
@@ -154,7 +154,7 @@ make_lead_agent(config)
## Implementation Details
### Agent Module
- **Location**: `src/agents/lead_agent/agent.py`
- **Location**: `packages/harness/deerflow/agents/lead_agent/agent.py`
- **Function**: `_create_todo_list_middleware(is_plan_mode: bool)` - Creates TodoListMiddleware if plan mode is enabled
- **Function**: `_build_middlewares(config: RunnableConfig)` - Builds middleware chain based on runtime config
- **Function**: `make_lead_agent(config: RunnableConfig)` - Creates agent with appropriate middlewares
@@ -194,7 +194,7 @@ DeerFlow uses custom `system_prompt` and `tool_description` for the TodoListMidd
- Comprehensive best practices section
- Task completion requirements to prevent premature marking
The custom prompts are defined in `_create_todo_list_middleware()` in `/Users/hetao/workspace/deer-flow/backend/src/agents/lead_agent/agent.py:57`.
The custom prompts are defined in `_create_todo_list_middleware()` in `/Users/hetao/workspace/deer-flow/backend/packages/harness/deerflow/agents/lead_agent/agent.py:57`.
## Notes

View File

@@ -269,8 +269,8 @@ The middleware intelligently preserves message context:
### Code Structure
- **Configuration**: `src/config/summarization_config.py`
- **Integration**: `src/agents/lead_agent/agent.py`
- **Configuration**: `packages/harness/deerflow/config/summarization_config.py`
- **Integration**: `packages/harness/deerflow/agents/lead_agent/agent.py`
- **Middleware**: Uses `langchain.agents.middleware.SummarizationMiddleware`
### Middleware Order

View File

@@ -65,7 +65,7 @@ The `task_status_tool` is no longer exposed to the LLM. It's kept in the codebas
### Polling Logic
Located in `src/tools/builtins/task_tool.py`:
Located in `packages/harness/deerflow/tools/builtins/task_tool.py`:
```python
# Start background execution
@@ -93,7 +93,7 @@ while True:
In addition to polling timeout, subagent execution now has a built-in timeout mechanism:
**Configuration** (`src/subagents/config.py`):
**Configuration** (`packages/harness/deerflow/subagents/config.py`):
```python
@dataclass
class SubagentConfig:

View File

@@ -6,9 +6,9 @@
],
"env": ".env",
"graphs": {
"lead_agent": "src.agents:make_lead_agent"
"lead_agent": "deerflow.agents:make_lead_agent"
},
"checkpointer": {
"path": "./src/agents/checkpointer/async_provider.py:make_checkpointer"
"path": "./packages/harness/deerflow/agents/checkpointer/async_provider.py:make_checkpointer"
}
}
}

View File

@@ -7,12 +7,12 @@ Supported backends: memory, sqlite, postgres.
Usage (e.g. FastAPI lifespan)::
from src.agents.checkpointer.async_provider import make_checkpointer
from deerflow.agents.checkpointer.async_provider import make_checkpointer
async with make_checkpointer() as checkpointer:
app.state.checkpointer = checkpointer # InMemorySaver if not configured
For sync usage see :mod:`src.agents.checkpointer.provider`.
For sync usage see :mod:`deerflow.agents.checkpointer.provider`.
"""
from __future__ import annotations
@@ -23,13 +23,13 @@ from collections.abc import AsyncIterator
from langgraph.types import Checkpointer
from src.agents.checkpointer.provider import (
from deerflow.agents.checkpointer.provider import (
POSTGRES_CONN_REQUIRED,
POSTGRES_INSTALL,
SQLITE_INSTALL,
_resolve_sqlite_conn_str,
)
from src.config.app_config import get_app_config
from deerflow.config.app_config import get_app_config
logger = logging.getLogger(__name__)

View File

@@ -7,7 +7,7 @@ Supported backends: memory, sqlite, postgres.
Usage::
from src.agents.checkpointer.provider import get_checkpointer, checkpointer_context
from deerflow.agents.checkpointer.provider import get_checkpointer, checkpointer_context
# Singleton — reused across calls, closed on process exit
cp = get_checkpointer()
@@ -25,9 +25,9 @@ from collections.abc import Iterator
from langgraph.types import Checkpointer
from src.config.app_config import get_app_config
from src.config.checkpointer_config import CheckpointerConfig
from src.config.paths import resolve_path
from deerflow.config.app_config import get_app_config
from deerflow.config.checkpointer_config import CheckpointerConfig
from deerflow.config.paths import resolve_path
logger = logging.getLogger(__name__)
@@ -128,8 +128,8 @@ def get_checkpointer() -> Checkpointer:
# Ensure app config is loaded before checking checkpointer config
# This prevents returning InMemorySaver when config.yaml actually has a checkpointer section
# but hasn't been loaded yet
from src.config.app_config import _app_config
from src.config.checkpointer_config import get_checkpointer_config
from deerflow.config.app_config import _app_config
from deerflow.config.checkpointer_config import get_checkpointer_config
if _app_config is None:
# Only load config if it hasn't been initialized yet

View File

@@ -4,20 +4,20 @@ from langchain.agents import create_agent
from langchain.agents.middleware import SummarizationMiddleware
from langchain_core.runnables import RunnableConfig
from src.agents.lead_agent.prompt import apply_prompt_template
from src.agents.middlewares.clarification_middleware import ClarificationMiddleware
from src.agents.middlewares.loop_detection_middleware import LoopDetectionMiddleware
from src.agents.middlewares.memory_middleware import MemoryMiddleware
from src.agents.middlewares.subagent_limit_middleware import SubagentLimitMiddleware
from src.agents.middlewares.title_middleware import TitleMiddleware
from src.agents.middlewares.todo_middleware import TodoMiddleware
from src.agents.middlewares.tool_error_handling_middleware import build_lead_runtime_middlewares
from src.agents.middlewares.view_image_middleware import ViewImageMiddleware
from src.agents.thread_state import ThreadState
from src.config.agents_config import load_agent_config
from src.config.app_config import get_app_config
from src.config.summarization_config import get_summarization_config
from src.models import create_chat_model
from deerflow.agents.lead_agent.prompt import apply_prompt_template
from deerflow.agents.middlewares.clarification_middleware import ClarificationMiddleware
from deerflow.agents.middlewares.loop_detection_middleware import LoopDetectionMiddleware
from deerflow.agents.middlewares.memory_middleware import MemoryMiddleware
from deerflow.agents.middlewares.subagent_limit_middleware import SubagentLimitMiddleware
from deerflow.agents.middlewares.title_middleware import TitleMiddleware
from deerflow.agents.middlewares.todo_middleware import TodoMiddleware
from deerflow.agents.middlewares.tool_error_handling_middleware import build_lead_runtime_middlewares
from deerflow.agents.middlewares.view_image_middleware import ViewImageMiddleware
from deerflow.agents.thread_state import ThreadState
from deerflow.config.agents_config import load_agent_config
from deerflow.config.app_config import get_app_config
from deerflow.config.summarization_config import get_summarization_config
from deerflow.models import create_chat_model
logger = logging.getLogger(__name__)
@@ -256,8 +256,8 @@ def _build_middlewares(config: RunnableConfig, model_name: str | None, agent_nam
def make_lead_agent(config: RunnableConfig):
# Lazy import to avoid circular dependency
from src.tools import get_available_tools
from src.tools.builtins import setup_agent
from deerflow.tools import get_available_tools
from deerflow.tools.builtins import setup_agent
cfg = config.get("configurable", {})

View File

@@ -1,7 +1,7 @@
from datetime import datetime
from src.config.agents_config import load_agent_soul
from src.skills import load_skills
from deerflow.config.agents_config import load_agent_soul
from deerflow.skills import load_skills
def _build_subagent_section(max_concurrent: int) -> str:
@@ -292,8 +292,8 @@ def _get_memory_context(agent_name: str | None = None) -> str:
Formatted memory context string wrapped in XML tags, or empty string if disabled.
"""
try:
from src.agents.memory import format_memory_for_injection, get_memory_data
from src.config.memory_config import get_memory_config
from deerflow.agents.memory import format_memory_for_injection, get_memory_data
from deerflow.config.memory_config import get_memory_config
config = get_memory_config()
if not config.enabled or not config.injection_enabled:
@@ -323,7 +323,7 @@ def get_skills_prompt_section(available_skills: set[str] | None = None) -> str:
skills = load_skills(enabled_only=True)
try:
from src.config import get_app_config
from deerflow.config import get_app_config
config = get_app_config()
container_base_path = config.skills.container_path

View File

@@ -6,19 +6,19 @@ This module provides a global memory mechanism that:
- Injects relevant memory into system prompts for personalized responses
"""
from src.agents.memory.prompt import (
from deerflow.agents.memory.prompt import (
FACT_EXTRACTION_PROMPT,
MEMORY_UPDATE_PROMPT,
format_conversation_for_update,
format_memory_for_injection,
)
from src.agents.memory.queue import (
from deerflow.agents.memory.queue import (
ConversationContext,
MemoryUpdateQueue,
get_memory_queue,
reset_memory_queue,
)
from src.agents.memory.updater import (
from deerflow.agents.memory.updater import (
MemoryUpdater,
get_memory_data,
reload_memory_data,

View File

@@ -6,7 +6,7 @@ from dataclasses import dataclass, field
from datetime import datetime
from typing import Any
from src.config.memory_config import get_memory_config
from deerflow.config.memory_config import get_memory_config
@dataclass
@@ -84,7 +84,7 @@ class MemoryUpdateQueue:
def _process_queue(self) -> None:
"""Process all queued conversation contexts."""
# Import here to avoid circular dependency
from src.agents.memory.updater import MemoryUpdater
from deerflow.agents.memory.updater import MemoryUpdater
with self._lock:
if self._processing:

View File

@@ -7,13 +7,13 @@ from datetime import datetime
from pathlib import Path
from typing import Any
from src.agents.memory.prompt import (
from deerflow.agents.memory.prompt import (
MEMORY_UPDATE_PROMPT,
format_conversation_for_update,
)
from src.config.memory_config import get_memory_config
from src.config.paths import get_paths
from src.models import create_chat_model
from deerflow.config.memory_config import get_memory_config
from deerflow.config.paths import get_paths
from deerflow.models import create_chat_model
def _get_memory_file_path(agent_name: str | None = None) -> Path:

View File

@@ -7,8 +7,8 @@ from langchain.agents import AgentState
from langchain.agents.middleware import AgentMiddleware
from langgraph.runtime import Runtime
from src.agents.memory.queue import get_memory_queue
from src.config.memory_config import get_memory_config
from deerflow.agents.memory.queue import get_memory_queue
from deerflow.config.memory_config import get_memory_config
class MemoryMiddlewareState(AgentState):

View File

@@ -7,7 +7,7 @@ from langchain.agents import AgentState
from langchain.agents.middleware import AgentMiddleware
from langgraph.runtime import Runtime
from src.subagents.executor import MAX_CONCURRENT_SUBAGENTS
from deerflow.subagents.executor import MAX_CONCURRENT_SUBAGENTS
logger = logging.getLogger(__name__)

View File

@@ -4,8 +4,8 @@ from langchain.agents import AgentState
from langchain.agents.middleware import AgentMiddleware
from langgraph.runtime import Runtime
from src.agents.thread_state import ThreadDataState
from src.config.paths import Paths, get_paths
from deerflow.agents.thread_state import ThreadDataState
from deerflow.config.paths import Paths, get_paths
class ThreadDataMiddlewareState(AgentState):

View File

@@ -6,8 +6,8 @@ from langchain.agents import AgentState
from langchain.agents.middleware import AgentMiddleware
from langgraph.runtime import Runtime
from src.config.title_config import get_title_config
from src.models import create_chat_model
from deerflow.config.title_config import get_title_config
from deerflow.models import create_chat_model
class TitleMiddlewareState(AgentState):

View File

@@ -26,10 +26,7 @@ class ToolErrorHandlingMiddleware(AgentMiddleware[AgentState]):
if len(detail) > 500:
detail = detail[:497] + "..."
content = (
f"Error: Tool '{tool_name}' failed with {exc.__class__.__name__}: {detail}. "
"Continue with available context, or choose an alternative tool."
)
content = f"Error: Tool '{tool_name}' failed with {exc.__class__.__name__}: {detail}. Continue with available context, or choose an alternative tool."
return ToolMessage(
content=content,
tool_call_id=tool_call_id,
@@ -75,8 +72,8 @@ def _build_runtime_middlewares(
lazy_init: bool = True,
) -> list[AgentMiddleware]:
"""Build shared base middlewares for agent execution."""
from src.agents.middlewares.thread_data_middleware import ThreadDataMiddleware
from src.sandbox.middleware import SandboxMiddleware
from deerflow.agents.middlewares.thread_data_middleware import ThreadDataMiddleware
from deerflow.sandbox.middleware import SandboxMiddleware
middlewares: list[AgentMiddleware] = [
ThreadDataMiddleware(lazy_init=lazy_init),
@@ -84,12 +81,12 @@ def _build_runtime_middlewares(
]
if include_uploads:
from src.agents.middlewares.uploads_middleware import UploadsMiddleware
from deerflow.agents.middlewares.uploads_middleware import UploadsMiddleware
middlewares.insert(1, UploadsMiddleware())
if include_dangling_tool_call_patch:
from src.agents.middlewares.dangling_tool_call_middleware import DanglingToolCallMiddleware
from deerflow.agents.middlewares.dangling_tool_call_middleware import DanglingToolCallMiddleware
middlewares.append(DanglingToolCallMiddleware())

View File

@@ -9,7 +9,7 @@ from langchain.agents.middleware import AgentMiddleware
from langchain_core.messages import HumanMessage
from langgraph.runtime import Runtime
from src.config.paths import Paths, get_paths
from deerflow.config.paths import Paths, get_paths
logger = logging.getLogger(__name__)

View File

@@ -7,7 +7,7 @@ from langchain.agents.middleware import AgentMiddleware
from langchain_core.messages import AIMessage, HumanMessage, ToolMessage
from langgraph.runtime import Runtime
from src.agents.thread_state import ViewedImageData
from deerflow.agents.thread_state import ViewedImageData
class ViewImageMiddlewareState(AgentState):

View File

@@ -4,7 +4,7 @@ Provides direct programmatic access to DeerFlow's agent capabilities
without requiring LangGraph Server or Gateway API processes.
Usage:
from src.client import DeerFlowClient
from deerflow.client import DeerFlowClient
client = DeerFlowClient()
response = client.chat("Analyze this paper for me", thread_id="my-thread")
@@ -34,13 +34,13 @@ from langchain.agents import create_agent
from langchain_core.messages import AIMessage, HumanMessage, SystemMessage, ToolMessage
from langchain_core.runnables import RunnableConfig
from src.agents.lead_agent.agent import _build_middlewares
from src.agents.lead_agent.prompt import apply_prompt_template
from src.agents.thread_state import ThreadState
from src.config.app_config import get_app_config, reload_app_config
from src.config.extensions_config import ExtensionsConfig, SkillStateConfig, get_extensions_config, reload_extensions_config
from src.config.paths import get_paths
from src.models import create_chat_model
from deerflow.agents.lead_agent.agent import _build_middlewares
from deerflow.agents.lead_agent.prompt import apply_prompt_template
from deerflow.agents.thread_state import ThreadState
from deerflow.config.app_config import get_app_config, reload_app_config
from deerflow.config.extensions_config import ExtensionsConfig, SkillStateConfig, get_extensions_config, reload_extensions_config
from deerflow.config.paths import get_paths
from deerflow.models import create_chat_model
logger = logging.getLogger(__name__)
@@ -81,7 +81,7 @@ class DeerFlowClient:
Example::
from src.client import DeerFlowClient
from deerflow.client import DeerFlowClient
client = DeerFlowClient()
@@ -211,7 +211,7 @@ class DeerFlowClient:
}
checkpointer = self._checkpointer
if checkpointer is None:
from src.agents.checkpointer import get_checkpointer
from deerflow.agents.checkpointer import get_checkpointer
checkpointer = get_checkpointer()
if checkpointer is not None:
@@ -224,7 +224,7 @@ class DeerFlowClient:
@staticmethod
def _get_tools(*, model_name: str | None, subagent_enabled: bool):
"""Lazy import to avoid circular dependency at module level."""
from src.tools import get_available_tools
from deerflow.tools import get_available_tools
return get_available_tools(model_name=model_name, subagent_enabled=subagent_enabled)
@@ -422,7 +422,7 @@ class DeerFlowClient:
Dict with "skills" key containing list of skill info dicts,
matching the Gateway API ``SkillsListResponse`` schema.
"""
from src.skills.loader import load_skills
from deerflow.skills.loader import load_skills
return {
"skills": [
@@ -443,7 +443,7 @@ class DeerFlowClient:
Returns:
Memory data dict (see src/agents/memory/updater.py for structure).
"""
from src.agents.memory.updater import get_memory_data
from deerflow.agents.memory.updater import get_memory_data
return get_memory_data()
@@ -528,7 +528,7 @@ class DeerFlowClient:
Returns:
Skill info dict, or None if not found.
"""
from src.skills.loader import load_skills
from deerflow.skills.loader import load_skills
skill = next((s for s in load_skills(enabled_only=False) if s.name == name), None)
if skill is None:
@@ -555,7 +555,7 @@ class DeerFlowClient:
ValueError: If the skill is not found.
OSError: If the config file cannot be written.
"""
from src.skills.loader import load_skills
from deerflow.skills.loader import load_skills
skills = load_skills(enabled_only=False)
skill = next((s for s in skills if s.name == name), None)
@@ -603,8 +603,8 @@ class DeerFlowClient:
FileNotFoundError: If the file does not exist.
ValueError: If the file is invalid.
"""
from src.gateway.routers.skills import _validate_skill_frontmatter
from src.skills.loader import get_skills_root_path
from deerflow.skills.loader import get_skills_root_path
from deerflow.skills.validation import _validate_skill_frontmatter
path = Path(skill_path)
if not path.exists():
@@ -664,7 +664,7 @@ class DeerFlowClient:
Returns:
The reloaded memory data dict.
"""
from src.agents.memory.updater import reload_memory_data
from deerflow.agents.memory.updater import reload_memory_data
return reload_memory_data()
@@ -674,7 +674,7 @@ class DeerFlowClient:
Returns:
Memory config dict.
"""
from src.config.memory_config import get_memory_config
from deerflow.config.memory_config import get_memory_config
config = get_memory_config()
return {
@@ -726,7 +726,7 @@ class DeerFlowClient:
FileNotFoundError: If any file does not exist.
ValueError: If any supplied path exists but is not a regular file.
"""
from src.gateway.routers.uploads import CONVERTIBLE_EXTENSIONS, convert_file_to_markdown
from deerflow.utils.file_conversion import CONVERTIBLE_EXTENSIONS, convert_file_to_markdown
# Validate all files upfront to avoid partial uploads.
resolved_files = []

View File

@@ -3,7 +3,7 @@ import logging
from agent_sandbox import Sandbox as AioSandboxClient
from src.sandbox.sandbox import Sandbox
from deerflow.sandbox.sandbox import Sandbox
logger = logging.getLogger(__name__)

View File

@@ -20,10 +20,10 @@ import threading
import time
import uuid
from src.config import get_app_config
from src.config.paths import VIRTUAL_PATH_PREFIX, Paths, get_paths
from src.sandbox.sandbox import Sandbox
from src.sandbox.sandbox_provider import SandboxProvider
from deerflow.config import get_app_config
from deerflow.config.paths import VIRTUAL_PATH_PREFIX, Paths, get_paths
from deerflow.sandbox.sandbox import Sandbox
from deerflow.sandbox.sandbox_provider import SandboxProvider
from .aio_sandbox import AioSandbox
from .backend import SandboxBackend, wait_for_sandbox_ready
@@ -51,7 +51,7 @@ class AioSandboxProvider(SandboxProvider):
- Remote/K8s mode (connect to pre-existing sandbox URL)
Configuration options in config.yaml under sandbox:
use: src.community.aio_sandbox:AioSandboxProvider
use: deerflow.community.aio_sandbox:AioSandboxProvider
image: <container image>
port: 8080 # Base port for local containers
container_prefix: deer-flow-sandbox

View File

@@ -10,7 +10,7 @@ import logging
import os
import subprocess
from src.utils.network import get_free_port, release_port
from deerflow.utils.network import get_free_port, release_port
from .backend import SandboxBackend, wait_for_sandbox_ready
from .sandbox_info import SandboxInfo

View File

@@ -36,7 +36,7 @@ class RemoteSandboxBackend(SandboxBackend):
Typical config.yaml::
sandbox:
use: src.community.aio_sandbox:AioSandboxProvider
use: deerflow.community.aio_sandbox:AioSandboxProvider
provisioner_url: http://provisioner:8002
"""

View File

@@ -3,7 +3,7 @@ import json
from firecrawl import FirecrawlApp
from langchain.tools import tool
from src.config import get_app_config
from deerflow.config import get_app_config
def _get_firecrawl_client() -> FirecrawlApp:

View File

@@ -7,7 +7,7 @@ import logging
from langchain.tools import tool
from src.config import get_app_config
from deerflow.config import get_app_config
logger = logging.getLogger(__name__)

View File

@@ -1,7 +1,7 @@
from langchain.tools import tool
from src.config import get_app_config
from src.utils.readability import ReadabilityExtractor
from deerflow.config import get_app_config
from deerflow.utils.readability import ReadabilityExtractor
from .infoquest_client import InfoQuestClient

View File

@@ -1,8 +1,8 @@
from langchain.tools import tool
from src.community.jina_ai.jina_client import JinaClient
from src.config import get_app_config
from src.utils.readability import ReadabilityExtractor
from deerflow.community.jina_ai.jina_client import JinaClient
from deerflow.config import get_app_config
from deerflow.utils.readability import ReadabilityExtractor
readability_extractor = ReadabilityExtractor()

View File

@@ -3,7 +3,7 @@ import json
from langchain.tools import tool
from tavily import TavilyClient
from src.config import get_app_config
from deerflow.config import get_app_config
def _get_tavily_client() -> TavilyClient:

View File

@@ -7,7 +7,7 @@ from typing import Any
import yaml
from pydantic import BaseModel
from src.config.paths import get_paths
from deerflow.config.paths import get_paths
logger = logging.getLogger(__name__)

View File

@@ -1,3 +1,4 @@
import logging
import os
from pathlib import Path
from typing import Any, Self
@@ -6,19 +7,21 @@ import yaml
from dotenv import load_dotenv
from pydantic import BaseModel, ConfigDict, Field
from src.config.checkpointer_config import CheckpointerConfig, load_checkpointer_config_from_dict
from src.config.extensions_config import ExtensionsConfig
from src.config.memory_config import load_memory_config_from_dict
from src.config.model_config import ModelConfig
from src.config.sandbox_config import SandboxConfig
from src.config.skills_config import SkillsConfig
from src.config.subagents_config import load_subagents_config_from_dict
from src.config.summarization_config import load_summarization_config_from_dict
from src.config.title_config import load_title_config_from_dict
from src.config.tool_config import ToolConfig, ToolGroupConfig
from deerflow.config.checkpointer_config import CheckpointerConfig, load_checkpointer_config_from_dict
from deerflow.config.extensions_config import ExtensionsConfig
from deerflow.config.memory_config import load_memory_config_from_dict
from deerflow.config.model_config import ModelConfig
from deerflow.config.sandbox_config import SandboxConfig
from deerflow.config.skills_config import SkillsConfig
from deerflow.config.subagents_config import load_subagents_config_from_dict
from deerflow.config.summarization_config import load_summarization_config_from_dict
from deerflow.config.title_config import load_title_config_from_dict
from deerflow.config.tool_config import ToolConfig, ToolGroupConfig
load_dotenv()
logger = logging.getLogger(__name__)
class AppConfig(BaseModel):
"""Config for the DeerFlow application"""
@@ -75,7 +78,11 @@ class AppConfig(BaseModel):
"""
resolved_path = cls.resolve_config_path(config_path)
with open(resolved_path, encoding="utf-8") as f:
config_data = yaml.safe_load(f)
config_data = yaml.safe_load(f) or {}
# Check config version before processing
cls._check_config_version(config_data, resolved_path)
config_data = cls.resolve_env_variables(config_data)
# Load title config if present
@@ -105,6 +112,52 @@ class AppConfig(BaseModel):
result = cls.model_validate(config_data)
return result
@classmethod
def _check_config_version(cls, config_data: dict, config_path: Path) -> None:
"""Check if the user's config.yaml is outdated compared to config.example.yaml.
Emits a warning if the user's config_version is lower than the example's.
Missing config_version is treated as version 0 (pre-versioning).
"""
try:
user_version = int(config_data.get("config_version", 0))
except (TypeError, ValueError):
user_version = 0
# Find config.example.yaml by searching config.yaml's directory and its parents
example_path = None
search_dir = config_path.parent
for _ in range(5): # search up to 5 levels
candidate = search_dir / "config.example.yaml"
if candidate.exists():
example_path = candidate
break
parent = search_dir.parent
if parent == search_dir:
break
search_dir = parent
if example_path is None:
return
try:
with open(example_path, encoding="utf-8") as f:
example_data = yaml.safe_load(f)
raw = example_data.get("config_version", 0) if example_data else 0
try:
example_version = int(raw)
except (TypeError, ValueError):
example_version = 0
except Exception:
return
if user_version < example_version:
logger.warning(
"Your config.yaml (version %d) is outdated — the latest version is %d. "
"Run `make config-upgrade` to merge new fields into your config.",
user_version,
example_version,
)
@classmethod
def resolve_env_variables(cls, config: Any) -> Any:
"""Recursively resolve environment variables in the config.

View File

@@ -27,7 +27,7 @@ class SandboxConfig(BaseModel):
use: str = Field(
...,
description="Class path of the sandbox provider (e.g. src.sandbox.local:LocalSandboxProvider)",
description="Class path of the sandbox provider (e.g. deerflow.sandbox.local:LocalSandboxProvider)",
)
image: str | None = Field(
default=None,

View File

@@ -31,7 +31,7 @@ class SkillsConfig(BaseModel):
return path.resolve()
else:
# Default: ../skills relative to backend directory
from src.skills.loader import get_skills_root_path
from deerflow.skills.loader import get_skills_root_path
return get_skills_root_path()

Some files were not shown because too many files have changed in this diff Show More