refactor: split backend into harness (deerflow.*) and app (app.*) (#1131)

* refactor: extract shared utils to break harness→app cross-layer imports

Move _validate_skill_frontmatter to src/skills/validation.py and
CONVERTIBLE_EXTENSIONS + convert_file_to_markdown to src/utils/file_conversion.py.
This eliminates the two reverse dependencies from client.py (harness layer)
into gateway/routers/ (app layer), preparing for the harness/app package split.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: split backend/src into harness (deerflow.*) and app (app.*)

Physically split the monolithic backend/src/ package into two layers:

- **Harness** (`packages/harness/deerflow/`): publishable agent framework
  package with import prefix `deerflow.*`. Contains agents, sandbox, tools,
  models, MCP, skills, config, and all core infrastructure.

- **App** (`app/`): unpublished application code with import prefix `app.*`.
  Contains gateway (FastAPI REST API) and channels (IM integrations).

Key changes:
- Move 13 harness modules to packages/harness/deerflow/ via git mv
- Move gateway + channels to app/ via git mv
- Rename all imports: src.* → deerflow.* (harness) / app.* (app layer)
- Set up uv workspace with deerflow-harness as workspace member
- Update langgraph.json, config.example.yaml, all scripts, Docker files
- Add build-system (hatchling) to harness pyproject.toml
- Add PYTHONPATH=. to gateway startup commands for app.* resolution
- Update ruff.toml with known-first-party for import sorting
- Update all documentation to reflect new directory structure

Boundary rule enforced: harness code never imports from app.
All 429 tests pass. Lint clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add harness→app boundary check test and update docs

Add test_harness_boundary.py that scans all Python files in
packages/harness/deerflow/ and fails if any `from app.*` or
`import app.*` statement is found. This enforces the architectural
rule that the harness layer never depends on the app layer.

Update CLAUDE.md to document the harness/app split architecture,
import conventions, and the boundary enforcement test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add config versioning with auto-upgrade on startup

When config.example.yaml schema changes, developers' local config.yaml
files can silently become outdated. This adds a config_version field and
auto-upgrade mechanism so breaking changes (like src.* → deerflow.*
renames) are applied automatically before services start.

- Add config_version: 1 to config.example.yaml
- Add startup version check warning in AppConfig.from_file()
- Add scripts/config-upgrade.sh with migration registry for value replacements
- Add `make config-upgrade` target
- Auto-run config-upgrade in serve.sh and start-daemon.sh before starting services
- Add config error hints in service failure messages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix comments

* fix: update src.* import in test_sandbox_tools_security to deerflow.*

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: handle empty config and search parent dirs for config.example.yaml

Address Copilot review comments on PR #1131:
- Guard against yaml.safe_load() returning None for empty config files
- Search parent directories for config.example.yaml instead of only
  looking next to config.yaml, fixing detection in common setups

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: correct skills root path depth and config_version type coercion

- loader.py: fix get_skills_root_path() to use 5 parent levels (was 3)
  after harness split, file lives at packages/harness/deerflow/skills/
  so parent×3 resolved to backend/packages/harness/ instead of backend/
- app_config.py: coerce config_version to int() before comparison in
  _check_config_version() to prevent TypeError when YAML stores value
  as string (e.g. config_version: "1")
- tests: add regression tests for both fixes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: update test imports from src.* to deerflow.*/app.* after harness refactor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

backend/packages/harness/deerflow/tools/__init__.py

@@ -0,0 +1,3 @@
+from .tools import get_available_tools
+
+__all__ = ["get_available_tools"]

backend/packages/harness/deerflow/tools/builtins/__init__.py

@@ -0,0 +1,13 @@
+from .clarification_tool import ask_clarification_tool
+from .present_file_tool import present_file_tool
+from .setup_agent_tool import setup_agent
+from .task_tool import task_tool
+from .view_image_tool import view_image_tool
+
+__all__ = [
+    "setup_agent",
+    "present_file_tool",
+    "ask_clarification_tool",
+    "view_image_tool",
+    "task_tool",
+]

backend/packages/harness/deerflow/tools/builtins/clarification_tool.py

@@ -0,0 +1,55 @@
+from typing import Literal
+
+from langchain.tools import tool
+
+
+@tool("ask_clarification", parse_docstring=True, return_direct=True)
+def ask_clarification_tool(
+    question: str,
+    clarification_type: Literal[
+        "missing_info",
+        "ambiguous_requirement",
+        "approach_choice",
+        "risk_confirmation",
+        "suggestion",
+    ],
+    context: str | None = None,
+    options: list[str] | None = None,
+) -> str:
+    """Ask the user for clarification when you need more information to proceed.
+
+    Use this tool when you encounter situations where you cannot proceed without user input:
+
+    - **Missing information**: Required details not provided (e.g., file paths, URLs, specific requirements)
+    - **Ambiguous requirements**: Multiple valid interpretations exist
+    - **Approach choices**: Several valid approaches exist and you need user preference
+    - **Risky operations**: Destructive actions that need explicit confirmation (e.g., deleting files, modifying production)
+    - **Suggestions**: You have a recommendation but want user approval before proceeding
+
+    The execution will be interrupted and the question will be presented to the user.
+    Wait for the user's response before continuing.
+
+    When to use ask_clarification:
+    - You need information that wasn't provided in the user's request
+    - The requirement can be interpreted in multiple ways
+    - Multiple valid implementation approaches exist
+    - You're about to perform a potentially dangerous operation
+    - You have a recommendation but need user approval
+
+    Best practices:
+    - Ask ONE clarification at a time for clarity
+    - Be specific and clear in your question
+    - Don't make assumptions when clarification is needed
+    - For risky operations, ALWAYS ask for confirmation
+    - After calling this tool, execution will be interrupted automatically
+
+    Args:
+        question: The clarification question to ask the user. Be specific and clear.
+        clarification_type: The type of clarification needed (missing_info, ambiguous_requirement, approach_choice, risk_confirmation, suggestion).
+        context: Optional context explaining why clarification is needed. Helps the user understand the situation.
+        options: Optional list of choices (for approach_choice or suggestion types). Present clear options for the user to choose from.
+    """
+    # This is a placeholder implementation
+    # The actual logic is handled by ClarificationMiddleware which intercepts this tool call
+    # and interrupts execution to present the question to the user
+    return "Clarification request processed by middleware"

backend/packages/harness/deerflow/tools/builtins/present_file_tool.py

@@ -0,0 +1,100 @@
+from pathlib import Path
+from typing import Annotated
+
+from langchain.tools import InjectedToolCallId, ToolRuntime, tool
+from langchain_core.messages import ToolMessage
+from langgraph.types import Command
+from langgraph.typing import ContextT
+
+from deerflow.agents.thread_state import ThreadState
+from deerflow.config.paths import VIRTUAL_PATH_PREFIX, get_paths
+
+OUTPUTS_VIRTUAL_PREFIX = f"{VIRTUAL_PATH_PREFIX}/outputs"
+
+
+def _normalize_presented_filepath(
+    runtime: ToolRuntime[ContextT, ThreadState],
+    filepath: str,
+) -> str:
+    """Normalize a presented file path to the `/mnt/user-data/outputs/*` contract.
+
+    Accepts either:
+    - A virtual sandbox path such as `/mnt/user-data/outputs/report.md`
+    - A host-side thread outputs path such as
+      `/app/backend/.deer-flow/threads/<thread>/user-data/outputs/report.md`
+
+    Returns:
+        The normalized virtual path.
+
+    Raises:
+        ValueError: If runtime metadata is missing or the path is outside the
+            current thread's outputs directory.
+    """
+    if runtime.state is None:
+        raise ValueError("Thread runtime state is not available")
+
+    thread_id = runtime.context.get("thread_id")
+    if not thread_id:
+        raise ValueError("Thread ID is not available in runtime context")
+
+    thread_data = runtime.state.get("thread_data") or {}
+    outputs_path = thread_data.get("outputs_path")
+    if not outputs_path:
+        raise ValueError("Thread outputs path is not available in runtime state")
+
+    outputs_dir = Path(outputs_path).resolve()
+    stripped = filepath.lstrip("/")
+    virtual_prefix = VIRTUAL_PATH_PREFIX.lstrip("/")
+
+    if stripped == virtual_prefix or stripped.startswith(virtual_prefix + "/"):
+        actual_path = get_paths().resolve_virtual_path(thread_id, filepath)
+    else:
+        actual_path = Path(filepath).expanduser().resolve()
+
+    try:
+        relative_path = actual_path.relative_to(outputs_dir)
+    except ValueError as exc:
+        raise ValueError(f"Only files in {OUTPUTS_VIRTUAL_PREFIX} can be presented: {filepath}") from exc
+
+    return f"{OUTPUTS_VIRTUAL_PREFIX}/{relative_path.as_posix()}"
+
+
+@tool("present_files", parse_docstring=True)
+def present_file_tool(
+    runtime: ToolRuntime[ContextT, ThreadState],
+    filepaths: list[str],
+    tool_call_id: Annotated[str, InjectedToolCallId],
+) -> Command:
+    """Make files visible to the user for viewing and rendering in the client interface.
+
+    When to use the present_files tool:
+
+    - Making any file available for the user to view, download, or interact with
+    - Presenting multiple related files at once
+    - After creating files that should be presented to the user
+
+    When NOT to use the present_files tool:
+    - When you only need to read file contents for your own processing
+    - For temporary or intermediate files not meant for user viewing
+
+    Notes:
+    - You should call this tool after creating files and moving them to the `/mnt/user-data/outputs` directory.
+    - This tool can be safely called in parallel with other tools. State updates are handled by a reducer to prevent conflicts.
+
+    Args:
+        filepaths: List of absolute file paths to present to the user. **Only** files in `/mnt/user-data/outputs` can be presented.
+    """
+    try:
+        normalized_paths = [_normalize_presented_filepath(runtime, filepath) for filepath in filepaths]
+    except ValueError as exc:
+        return Command(
+            update={"messages": [ToolMessage(f"Error: {exc}", tool_call_id=tool_call_id)]},
+        )
+
+    # The merge_artifacts reducer will handle merging and deduplication
+    return Command(
+        update={
+            "artifacts": normalized_paths,
+            "messages": [ToolMessage("Successfully presented files", tool_call_id=tool_call_id)],
+        },
+    )

backend/packages/harness/deerflow/tools/builtins/setup_agent_tool.py

@@ -0,0 +1,62 @@
+import logging
+
+import yaml
+from langchain_core.messages import ToolMessage
+from langchain_core.tools import tool
+from langgraph.prebuilt import ToolRuntime
+from langgraph.types import Command
+
+from deerflow.config.paths import get_paths
+
+logger = logging.getLogger(__name__)
+
+
+@tool
+def setup_agent(
+    soul: str,
+    description: str,
+    runtime: ToolRuntime,
+) -> Command:
+    """Setup the custom DeerFlow agent.
+
+    Args:
+        soul: Full SOUL.md content defining the agent's personality and behavior.
+        description: One-line description of what the agent does.
+    """
+
+    agent_name: str | None = runtime.context.get("agent_name")
+
+    try:
+        paths = get_paths()
+        agent_dir = paths.agent_dir(agent_name) if agent_name else paths.base_dir
+        agent_dir.mkdir(parents=True, exist_ok=True)
+
+        if agent_name:
+            # If agent_name is provided, we are creating a custom agent in the agents/ directory
+            config_data: dict = {"name": agent_name}
+            if description:
+                config_data["description"] = description
+
+            config_file = agent_dir / "config.yaml"
+            with open(config_file, "w", encoding="utf-8") as f:
+                yaml.dump(config_data, f, default_flow_style=False, allow_unicode=True)
+
+        soul_file = agent_dir / "SOUL.md"
+        soul_file.write_text(soul, encoding="utf-8")
+
+        logger.info(f"[agent_creator] Created agent '{agent_name}' at {agent_dir}")
+        return Command(
+            update={
+                "created_agent_name": agent_name,
+                "messages": [ToolMessage(content=f"Agent '{agent_name}' created successfully!", tool_call_id=runtime.tool_call_id)],
+            }
+        )
+
+    except Exception as e:
+        import shutil
+
+        if agent_name and agent_dir.exists():
+            # Cleanup the custom agent directory only if it was created but an error occurred during setup
+            shutil.rmtree(agent_dir)
+        logger.error(f"[agent_creator] Failed to create agent '{agent_name}': {e}", exc_info=True)
+        return Command(update={"messages": [ToolMessage(content=f"Error: {e}", tool_call_id=runtime.tool_call_id)]})

backend/packages/harness/deerflow/tools/builtins/task_tool.py

@@ -0,0 +1,195 @@
+"""Task tool for delegating work to subagents."""
+
+import logging
+import time
+import uuid
+from dataclasses import replace
+from typing import Annotated, Literal
+
+from langchain.tools import InjectedToolCallId, ToolRuntime, tool
+from langgraph.config import get_stream_writer
+from langgraph.typing import ContextT
+
+from deerflow.agents.lead_agent.prompt import get_skills_prompt_section
+from deerflow.agents.thread_state import ThreadState
+from deerflow.subagents import SubagentExecutor, get_subagent_config
+from deerflow.subagents.executor import SubagentStatus, cleanup_background_task, get_background_task_result
+
+logger = logging.getLogger(__name__)
+
+
+@tool("task", parse_docstring=True)
+def task_tool(
+    runtime: ToolRuntime[ContextT, ThreadState],
+    description: str,
+    prompt: str,
+    subagent_type: Literal["general-purpose", "bash"],
+    tool_call_id: Annotated[str, InjectedToolCallId],
+    max_turns: int | None = None,
+) -> str:
+    """Delegate a task to a specialized subagent that runs in its own context.
+
+    Subagents help you:
+    - Preserve context by keeping exploration and implementation separate
+    - Handle complex multi-step tasks autonomously
+    - Execute commands or operations in isolated contexts
+
+    Available subagent types:
+    - **general-purpose**: A capable agent for complex, multi-step tasks that require
+      both exploration and action. Use when the task requires complex reasoning,
+      multiple dependent steps, or would benefit from isolated context.
+    - **bash**: Command execution specialist for running bash commands. Use for
+      git operations, build processes, or when command output would be verbose.
+
+    When to use this tool:
+    - Complex tasks requiring multiple steps or tools
+    - Tasks that produce verbose output
+    - When you want to isolate context from the main conversation
+    - Parallel research or exploration tasks
+
+    When NOT to use this tool:
+    - Simple, single-step operations (use tools directly)
+    - Tasks requiring user interaction or clarification
+
+    Args:
+        description: A short (3-5 word) description of the task for logging/display. ALWAYS PROVIDE THIS PARAMETER FIRST.
+        prompt: The task description for the subagent. Be specific and clear about what needs to be done. ALWAYS PROVIDE THIS PARAMETER SECOND.
+        subagent_type: The type of subagent to use. ALWAYS PROVIDE THIS PARAMETER THIRD.
+        max_turns: Optional maximum number of agent turns. Defaults to subagent's configured max.
+    """
+    # Get subagent configuration
+    config = get_subagent_config(subagent_type)
+    if config is None:
+        return f"Error: Unknown subagent type '{subagent_type}'. Available: general-purpose, bash"
+
+    # Build config overrides
+    overrides: dict = {}
+
+    skills_section = get_skills_prompt_section()
+    if skills_section:
+        overrides["system_prompt"] = config.system_prompt + "\n\n" + skills_section
+
+    if max_turns is not None:
+        overrides["max_turns"] = max_turns
+
+    if overrides:
+        config = replace(config, **overrides)
+
+    # Extract parent context from runtime
+    sandbox_state = None
+    thread_data = None
+    thread_id = None
+    parent_model = None
+    trace_id = None
+
+    if runtime is not None:
+        sandbox_state = runtime.state.get("sandbox")
+        thread_data = runtime.state.get("thread_data")
+        thread_id = runtime.context.get("thread_id")
+
+        # Try to get parent model from configurable
+        metadata = runtime.config.get("metadata", {})
+        parent_model = metadata.get("model_name")
+
+        # Get or generate trace_id for distributed tracing
+        trace_id = metadata.get("trace_id") or str(uuid.uuid4())[:8]
+
+    # Get available tools (excluding task tool to prevent nesting)
+    # Lazy import to avoid circular dependency
+    from deerflow.tools import get_available_tools
+
+    # Subagents should not have subagent tools enabled (prevent recursive nesting)
+    tools = get_available_tools(model_name=parent_model, subagent_enabled=False)
+
+    # Create executor
+    executor = SubagentExecutor(
+        config=config,
+        tools=tools,
+        parent_model=parent_model,
+        sandbox_state=sandbox_state,
+        thread_data=thread_data,
+        thread_id=thread_id,
+        trace_id=trace_id,
+    )
+
+    # Start background execution (always async to prevent blocking)
+    # Use tool_call_id as task_id for better traceability
+    task_id = executor.execute_async(prompt, task_id=tool_call_id)
+
+    # Poll for task completion in backend (removes need for LLM to poll)
+    poll_count = 0
+    last_status = None
+    last_message_count = 0  # Track how many AI messages we've already sent
+    # Polling timeout: execution timeout + 60s buffer, checked every 5s
+    max_poll_count = (config.timeout_seconds + 60) // 5
+
+    logger.info(f"[trace={trace_id}] Started background task {task_id} (subagent={subagent_type}, timeout={config.timeout_seconds}s, polling_limit={max_poll_count} polls)")
+
+    writer = get_stream_writer()
+    # Send Task Started message'
+    writer({"type": "task_started", "task_id": task_id, "description": description})
+
+    while True:
+        result = get_background_task_result(task_id)
+
+        if result is None:
+            logger.error(f"[trace={trace_id}] Task {task_id} not found in background tasks")
+            writer({"type": "task_failed", "task_id": task_id, "error": "Task disappeared from background tasks"})
+            cleanup_background_task(task_id)
+            return f"Error: Task {task_id} disappeared from background tasks"
+
+        # Log status changes for debugging
+        if result.status != last_status:
+            logger.info(f"[trace={trace_id}] Task {task_id} status: {result.status.value}")
+            last_status = result.status
+
+        # Check for new AI messages and send task_running events
+        current_message_count = len(result.ai_messages)
+        if current_message_count > last_message_count:
+            # Send task_running event for each new message
+            for i in range(last_message_count, current_message_count):
+                message = result.ai_messages[i]
+                writer(
+                    {
+                        "type": "task_running",
+                        "task_id": task_id,
+                        "message": message,
+                        "message_index": i + 1,  # 1-based index for display
+                        "total_messages": current_message_count,
+                    }
+                )
+                logger.info(f"[trace={trace_id}] Task {task_id} sent message #{i + 1}/{current_message_count}")
+            last_message_count = current_message_count
+
+        # Check if task completed, failed, or timed out
+        if result.status == SubagentStatus.COMPLETED:
+            writer({"type": "task_completed", "task_id": task_id, "result": result.result})
+            logger.info(f"[trace={trace_id}] Task {task_id} completed after {poll_count} polls")
+            cleanup_background_task(task_id)
+            return f"Task Succeeded. Result: {result.result}"
+        elif result.status == SubagentStatus.FAILED:
+            writer({"type": "task_failed", "task_id": task_id, "error": result.error})
+            logger.error(f"[trace={trace_id}] Task {task_id} failed: {result.error}")
+            cleanup_background_task(task_id)
+            return f"Task failed. Error: {result.error}"
+        elif result.status == SubagentStatus.TIMED_OUT:
+            writer({"type": "task_timed_out", "task_id": task_id, "error": result.error})
+            logger.warning(f"[trace={trace_id}] Task {task_id} timed out: {result.error}")
+            cleanup_background_task(task_id)
+            return f"Task timed out. Error: {result.error}"
+
+        # Still running, wait before next poll
+        time.sleep(5)  # Poll every 5 seconds
+        poll_count += 1
+
+        # Polling timeout as a safety net (in case thread pool timeout doesn't work)
+        # Set to execution timeout + 60s buffer, in 5s poll intervals
+        # This catches edge cases where the background task gets stuck
+        # Note: We don't call cleanup_background_task here because the task may
+        # still be running in the background. The cleanup will happen when the
+        # executor completes and sets a terminal status.
+        if poll_count > max_poll_count:
+            timeout_minutes = config.timeout_seconds // 60
+            logger.error(f"[trace={trace_id}] Task {task_id} polling timed out after {poll_count} polls (should have been caught by thread pool timeout)")
+            writer({"type": "task_timed_out", "task_id": task_id})
+            return f"Task polling timed out after {timeout_minutes} minutes. This may indicate the background task is stuck. Status: {result.status.value}"

backend/packages/harness/deerflow/tools/builtins/view_image_tool.py

@@ -0,0 +1,94 @@
+import base64
+import mimetypes
+from pathlib import Path
+from typing import Annotated
+
+from langchain.tools import InjectedToolCallId, ToolRuntime, tool
+from langchain_core.messages import ToolMessage
+from langgraph.types import Command
+from langgraph.typing import ContextT
+
+from deerflow.agents.thread_state import ThreadState
+from deerflow.sandbox.tools import get_thread_data, replace_virtual_path
+
+
+@tool("view_image", parse_docstring=True)
+def view_image_tool(
+    runtime: ToolRuntime[ContextT, ThreadState],
+    image_path: str,
+    tool_call_id: Annotated[str, InjectedToolCallId],
+) -> Command:
+    """Read an image file.
+
+    Use this tool to read an image file and make it available for display.
+
+    When to use the view_image tool:
+    - When you need to view an image file.
+
+    When NOT to use the view_image tool:
+    - For non-image files (use present_files instead)
+    - For multiple files at once (use present_files instead)
+
+    Args:
+        image_path: Absolute path to the image file. Common formats supported: jpg, jpeg, png, webp.
+    """
+    # Replace virtual path with actual path
+    # /mnt/user-data/* paths are mapped to thread-specific directories
+    thread_data = get_thread_data(runtime)
+    actual_path = replace_virtual_path(image_path, thread_data)
+
+    # Validate that the path is absolute
+    path = Path(actual_path)
+    if not path.is_absolute():
+        return Command(
+            update={"messages": [ToolMessage(f"Error: Path must be absolute, got: {image_path}", tool_call_id=tool_call_id)]},
+        )
+
+    # Validate that the file exists
+    if not path.exists():
+        return Command(
+            update={"messages": [ToolMessage(f"Error: Image file not found: {image_path}", tool_call_id=tool_call_id)]},
+        )
+
+    # Validate that it's a file (not a directory)
+    if not path.is_file():
+        return Command(
+            update={"messages": [ToolMessage(f"Error: Path is not a file: {image_path}", tool_call_id=tool_call_id)]},
+        )
+
+    # Validate image extension
+    valid_extensions = {".jpg", ".jpeg", ".png", ".webp"}
+    if path.suffix.lower() not in valid_extensions:
+        return Command(
+            update={"messages": [ToolMessage(f"Error: Unsupported image format: {path.suffix}. Supported formats: {', '.join(valid_extensions)}", tool_call_id=tool_call_id)]},
+        )
+
+    # Detect MIME type from file extension
+    mime_type, _ = mimetypes.guess_type(actual_path)
+    if mime_type is None:
+        # Fallback to default MIME types for common image formats
+        extension_to_mime = {
+            ".jpg": "image/jpeg",
+            ".jpeg": "image/jpeg",
+            ".png": "image/png",
+            ".webp": "image/webp",
+        }
+        mime_type = extension_to_mime.get(path.suffix.lower(), "application/octet-stream")
+
+    # Read image file and convert to base64
+    try:
+        with open(actual_path, "rb") as f:
+            image_data = f.read()
+            image_base64 = base64.b64encode(image_data).decode("utf-8")
+    except Exception as e:
+        return Command(
+            update={"messages": [ToolMessage(f"Error reading image file: {str(e)}", tool_call_id=tool_call_id)]},
+        )
+
+    # Update viewed_images in state
+    # The merge_viewed_images reducer will handle merging with existing images
+    new_viewed_images = {image_path: {"base64": image_base64, "mime_type": mime_type}}
+
+    return Command(
+        update={"viewed_images": new_viewed_images, "messages": [ToolMessage("Successfully read image", tool_call_id=tool_call_id)]},
+    )

backend/packages/harness/deerflow/tools/tools.py

@@ -0,0 +1,84 @@
+import logging
+
+from langchain.tools import BaseTool
+
+from deerflow.config import get_app_config
+from deerflow.reflection import resolve_variable
+from deerflow.tools.builtins import ask_clarification_tool, present_file_tool, task_tool, view_image_tool
+
+logger = logging.getLogger(__name__)
+
+BUILTIN_TOOLS = [
+    present_file_tool,
+    ask_clarification_tool,
+]
+
+SUBAGENT_TOOLS = [
+    task_tool,
+    # task_status_tool is no longer exposed to LLM (backend handles polling internally)
+]
+
+
+def get_available_tools(
+    groups: list[str] | None = None,
+    include_mcp: bool = True,
+    model_name: str | None = None,
+    subagent_enabled: bool = False,
+) -> list[BaseTool]:
+    """Get all available tools from config.
+
+    Note: MCP tools should be initialized at application startup using
+    `initialize_mcp_tools()` from deerflow.mcp module.
+
+    Args:
+        groups: Optional list of tool groups to filter by.
+        include_mcp: Whether to include tools from MCP servers (default: True).
+        model_name: Optional model name to determine if vision tools should be included.
+        subagent_enabled: Whether to include subagent tools (task, task_status).
+
+    Returns:
+        List of available tools.
+    """
+    config = get_app_config()
+    loaded_tools = [resolve_variable(tool.use, BaseTool) for tool in config.tools if groups is None or tool.group in groups]
+
+    # Get cached MCP tools if enabled
+    # NOTE: We use ExtensionsConfig.from_file() instead of config.extensions
+    # to always read the latest configuration from disk. This ensures that changes
+    # made through the Gateway API (which runs in a separate process) are immediately
+    # reflected when loading MCP tools.
+    mcp_tools = []
+    if include_mcp:
+        try:
+            from deerflow.config.extensions_config import ExtensionsConfig
+            from deerflow.mcp.cache import get_cached_mcp_tools
+
+            extensions_config = ExtensionsConfig.from_file()
+            if extensions_config.get_enabled_mcp_servers():
+                mcp_tools = get_cached_mcp_tools()
+                if mcp_tools:
+                    logger.info(f"Using {len(mcp_tools)} cached MCP tool(s)")
+        except ImportError:
+            logger.warning("MCP module not available. Install 'langchain-mcp-adapters' package to enable MCP tools.")
+        except Exception as e:
+            logger.error(f"Failed to get cached MCP tools: {e}")
+
+    # Conditionally add tools based on config
+    builtin_tools = BUILTIN_TOOLS.copy()
+
+    # Add subagent tools only if enabled via runtime parameter
+    if subagent_enabled:
+        builtin_tools.extend(SUBAGENT_TOOLS)
+        logger.info("Including subagent tools (task)")
+
+    # If no model_name specified, use the first model (default)
+    if model_name is None and config.models:
+        model_name = config.models[0].name
+
+    # Add view_image_tool only if the model supports vision
+    model_config = config.get_model_config(model_name) if model_name else None
+    if model_config is not None and model_config.supports_vision:
+        builtin_tools.append(view_image_tool)
+        logger.info(f"Including view_image_tool for model '{model_name}' (supports_vision=True)")
+
+    return loaded_tools + builtin_tools + mcp_tools