Files
deer-flow/backend/tests/test_infoquest_client.py
DanielWalnut d119214fee feat(harness): integration ACP agent tool (#1344)
* refactor: extract shared utils to break harness→app cross-layer imports

Move _validate_skill_frontmatter to src/skills/validation.py and
CONVERTIBLE_EXTENSIONS + convert_file_to_markdown to src/utils/file_conversion.py.
This eliminates the two reverse dependencies from client.py (harness layer)
into gateway/routers/ (app layer), preparing for the harness/app package split.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: split backend/src into harness (deerflow.*) and app (app.*)

Physically split the monolithic backend/src/ package into two layers:

- **Harness** (`packages/harness/deerflow/`): publishable agent framework
  package with import prefix `deerflow.*`. Contains agents, sandbox, tools,
  models, MCP, skills, config, and all core infrastructure.

- **App** (`app/`): unpublished application code with import prefix `app.*`.
  Contains gateway (FastAPI REST API) and channels (IM integrations).

Key changes:
- Move 13 harness modules to packages/harness/deerflow/ via git mv
- Move gateway + channels to app/ via git mv
- Rename all imports: src.* → deerflow.* (harness) / app.* (app layer)
- Set up uv workspace with deerflow-harness as workspace member
- Update langgraph.json, config.example.yaml, all scripts, Docker files
- Add build-system (hatchling) to harness pyproject.toml
- Add PYTHONPATH=. to gateway startup commands for app.* resolution
- Update ruff.toml with known-first-party for import sorting
- Update all documentation to reflect new directory structure

Boundary rule enforced: harness code never imports from app.
All 429 tests pass. Lint clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add harness→app boundary check test and update docs

Add test_harness_boundary.py that scans all Python files in
packages/harness/deerflow/ and fails if any `from app.*` or
`import app.*` statement is found. This enforces the architectural
rule that the harness layer never depends on the app layer.

Update CLAUDE.md to document the harness/app split architecture,
import conventions, and the boundary enforcement test.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add config versioning with auto-upgrade on startup

When config.example.yaml schema changes, developers' local config.yaml
files can silently become outdated. This adds a config_version field and
auto-upgrade mechanism so breaking changes (like src.* → deerflow.*
renames) are applied automatically before services start.

- Add config_version: 1 to config.example.yaml
- Add startup version check warning in AppConfig.from_file()
- Add scripts/config-upgrade.sh with migration registry for value replacements
- Add `make config-upgrade` target
- Auto-run config-upgrade in serve.sh and start-daemon.sh before starting services
- Add config error hints in service failure messages

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix comments

* fix: update src.* import in test_sandbox_tools_security to deerflow.*

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: handle empty config and search parent dirs for config.example.yaml

Address Copilot review comments on PR #1131:
- Guard against yaml.safe_load() returning None for empty config files
- Search parent directories for config.example.yaml instead of only
  looking next to config.yaml, fixing detection in common setups

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: correct skills root path depth and config_version type coercion

- loader.py: fix get_skills_root_path() to use 5 parent levels (was 3)
  after harness split, file lives at packages/harness/deerflow/skills/
  so parent×3 resolved to backend/packages/harness/ instead of backend/
- app_config.py: coerce config_version to int() before comparison in
  _check_config_version() to prevent TypeError when YAML stores value
  as string (e.g. config_version: "1")
- tests: add regression tests for both fixes

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix: update test imports from src.* to deerflow.*/app.* after harness refactor

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat(harness): add tool-first ACP agent invocation (#37)

* feat(harness): add tool-first ACP agent invocation

* build(harness): make ACP dependency required

* fix(harness): address ACP review feedback

* feat(harness): decouple ACP agent workspace from thread data

ACP agents (codex, claude-code) previously used per-thread workspace
directories, causing path resolution complexity and coupling task
execution to DeerFlow's internal thread data layout. This change:

- Replace _resolve_cwd() with a fixed _get_work_dir() that always uses
  {base_dir}/acp-workspace/, eliminating virtual path translation and
  thread_id lookups
- Introduce /mnt/acp-workspace virtual path for lead agent read-only
  access to ACP agent output files (same pattern as /mnt/skills)
- Add security guards: read-only validation, path traversal prevention,
  command path allowlisting, and output masking for acp-workspace
- Update system prompt and tool description to guide LLM: send
  self-contained tasks to ACP agents, copy results via /mnt/acp-workspace
- Add 11 new security tests for ACP workspace path handling

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(prompt): inject ACP section only when ACP agents are configured

The ACP agent guidance in the system prompt is now conditionally built
by _build_acp_section(), which checks get_acp_agents() and returns an
empty string when no ACP agents are configured. This avoids polluting
the prompt with irrelevant instructions for users who don't use ACP.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix lint

* fix(harness): address Copilot review comments on sandbox path handling and ACP tool

- local_sandbox: fix path-segment boundary bug in _resolve_path (== or startswith +"/")
  and add lookahead in _resolve_paths_in_command regex to prevent /mnt/skills matching
  inside /mnt/skills-extra
- local_sandbox_provider: replace print() with logger.warning(..., exc_info=True)
- invoke_acp_agent_tool: guard getattr(option, "optionId") with None default + continue;
  move full prompt from INFO to DEBUG level (truncated to 200 chars)
- sandbox/tools: fix _get_acp_workspace_host_path docstring to match implementation;
  remove misleading "read-only" language from validate_local_bash_command_paths

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(acp): thread-isolated workspaces, permission guardrail, and ContextVar registry

P1.1 – ACP workspace thread isolation
- Add `Paths.acp_workspace_dir(thread_id)` for per-thread paths
- `_get_work_dir(thread_id)` in invoke_acp_agent_tool now uses
  `{base_dir}/threads/{thread_id}/acp-workspace/`; falls back to
  global workspace when thread_id is absent or invalid
- `_invoke` extracts thread_id from `RunnableConfig` via
  `Annotated[RunnableConfig, InjectedToolArg]`
- `sandbox/tools.py`: `_get_acp_workspace_host_path(thread_id)`,
  `_resolve_acp_workspace_path(path, thread_id)`, and all callers
  (`replace_virtual_paths_in_command`, `mask_local_paths_in_output`,
  `ls_tool`, `read_file_tool`) now resolve ACP paths per-thread

P1.2 – ACP permission guardrail
- New `auto_approve_permissions: bool = False` field in `ACPAgentConfig`
- `_build_permission_response(options, *, auto_approve: bool)` now
  defaults to deny; only approves when `auto_approve=True`
- Document field in `config.example.yaml`

P2 – Deferred tool registry race condition
- Replace module-level `_registry` global with `contextvars.ContextVar`
- Each asyncio request context gets its own registry; worker threads
  inherit the context automatically via `loop.run_in_executor`
- Expose `get_deferred_registry` / `set_deferred_registry` /
  `reset_deferred_registry` helpers

Tests: 831 pass (57 for affected modules, 3 new tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(sandbox): mount /mnt/acp-workspace in docker sandbox container

The AioSandboxProvider was not mounting the ACP workspace into the
sandbox container, so /mnt/acp-workspace was inaccessible when the lead
agent tried to read ACP results in docker mode.

Changes:
- `ensure_thread_dirs`: also create `acp-workspace/` (chmod 0o777) so
  the directory exists before the sandbox container starts — required
  for Docker volume mounts
- `_get_thread_mounts`: add read-only `/mnt/acp-workspace` mount using
  the per-thread host path (`host_paths.acp_workspace_dir(thread_id)`)
- Update stale CLAUDE.md description (was "fixed global workspace")

Tests: `test_aio_sandbox_provider.py` (4 new tests)

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(lint): remove unused imports in test_aio_sandbox_provider

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix config

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-26 14:20:18 +08:00

349 lines
16 KiB
Python

"""Tests for InfoQuest client and tools."""
import json
from unittest.mock import MagicMock, patch
from deerflow.community.infoquest import tools
from deerflow.community.infoquest.infoquest_client import InfoQuestClient
class TestInfoQuestClient:
def test_infoquest_client_initialization(self):
"""Test InfoQuestClient initialization with different parameters."""
# Test with default parameters
client = InfoQuestClient()
assert client.fetch_time == -1
assert client.fetch_timeout == -1
assert client.fetch_navigation_timeout == -1
assert client.search_time_range == -1
# Test with custom parameters
client = InfoQuestClient(fetch_time=10, fetch_timeout=30, fetch_navigation_timeout=60, search_time_range=24)
assert client.fetch_time == 10
assert client.fetch_timeout == 30
assert client.fetch_navigation_timeout == 60
assert client.search_time_range == 24
@patch("deerflow.community.infoquest.infoquest_client.requests.post")
def test_fetch_success(self, mock_post):
"""Test successful fetch operation."""
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.text = json.dumps({"reader_result": "<html><body>Test content</body></html>"})
mock_post.return_value = mock_response
client = InfoQuestClient()
result = client.fetch("https://example.com")
assert result == "<html><body>Test content</body></html>"
mock_post.assert_called_once()
args, kwargs = mock_post.call_args
assert args[0] == "https://reader.infoquest.bytepluses.com"
assert kwargs["json"]["url"] == "https://example.com"
assert kwargs["json"]["format"] == "HTML"
@patch("deerflow.community.infoquest.infoquest_client.requests.post")
def test_fetch_non_200_status(self, mock_post):
"""Test fetch operation with non-200 status code."""
mock_response = MagicMock()
mock_response.status_code = 404
mock_response.text = "Not Found"
mock_post.return_value = mock_response
client = InfoQuestClient()
result = client.fetch("https://example.com")
assert result == "Error: fetch API returned status 404: Not Found"
@patch("deerflow.community.infoquest.infoquest_client.requests.post")
def test_fetch_empty_response(self, mock_post):
"""Test fetch operation with empty response."""
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.text = ""
mock_post.return_value = mock_response
client = InfoQuestClient()
result = client.fetch("https://example.com")
assert result == "Error: no result found"
@patch("deerflow.community.infoquest.infoquest_client.requests.post")
def test_web_search_raw_results_success(self, mock_post):
"""Test successful web_search_raw_results operation."""
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {"search_result": {"results": [{"content": {"results": {"organic": [{"title": "Test Result", "desc": "Test description", "url": "https://example.com"}]}}}], "images_results": []}}
mock_post.return_value = mock_response
client = InfoQuestClient()
result = client.web_search_raw_results("test query", "")
assert "search_result" in result
mock_post.assert_called_once()
args, kwargs = mock_post.call_args
assert args[0] == "https://search.infoquest.bytepluses.com"
assert kwargs["json"]["query"] == "test query"
@patch("deerflow.community.infoquest.infoquest_client.requests.post")
def test_web_search_success(self, mock_post):
"""Test successful web_search operation."""
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {"search_result": {"results": [{"content": {"results": {"organic": [{"title": "Test Result", "desc": "Test description", "url": "https://example.com"}]}}}], "images_results": []}}
mock_post.return_value = mock_response
client = InfoQuestClient()
result = client.web_search("test query")
# Check if result is a valid JSON string with expected content
result_data = json.loads(result)
assert len(result_data) == 1
assert result_data[0]["title"] == "Test Result"
assert result_data[0]["url"] == "https://example.com"
def test_clean_results(self):
"""Test clean_results method with sample raw results."""
raw_results = [
{
"content": {
"results": {
"organic": [{"title": "Test Page", "desc": "Page description", "url": "https://example.com/page1"}],
"top_stories": {"items": [{"title": "Test News", "source": "Test Source", "time_frame": "2 hours ago", "url": "https://example.com/news1"}]},
}
}
}
]
cleaned = InfoQuestClient.clean_results(raw_results)
assert len(cleaned) == 2
assert cleaned[0]["type"] == "page"
assert cleaned[0]["title"] == "Test Page"
assert cleaned[1]["type"] == "news"
assert cleaned[1]["title"] == "Test News"
@patch("deerflow.community.infoquest.tools._get_infoquest_client")
def test_web_search_tool(self, mock_get_client):
"""Test web_search_tool function."""
mock_client = MagicMock()
mock_client.web_search.return_value = json.dumps([])
mock_get_client.return_value = mock_client
result = tools.web_search_tool.run("test query")
assert result == json.dumps([])
mock_get_client.assert_called_once()
mock_client.web_search.assert_called_once_with("test query")
@patch("deerflow.community.infoquest.tools._get_infoquest_client")
def test_web_fetch_tool(self, mock_get_client):
"""Test web_fetch_tool function."""
mock_client = MagicMock()
mock_client.fetch.return_value = "<html><body>Test content</body></html>"
mock_get_client.return_value = mock_client
result = tools.web_fetch_tool.run("https://example.com")
assert result == "# Untitled\n\nTest content"
mock_get_client.assert_called_once()
mock_client.fetch.assert_called_once_with("https://example.com")
@patch("deerflow.community.infoquest.tools.get_app_config")
def test_get_infoquest_client(self, mock_get_app_config):
"""Test _get_infoquest_client function with config."""
mock_config = MagicMock()
# Add image_search config to the side_effect
mock_config.get_tool_config.side_effect = [
MagicMock(model_extra={"search_time_range": 24}), # web_search config
MagicMock(model_extra={"fetch_time": 10, "timeout": 30, "navigation_timeout": 60}), # web_fetch config
MagicMock(model_extra={"image_search_time_range": 7, "image_size": "l"}), # image_search config
]
mock_get_app_config.return_value = mock_config
client = tools._get_infoquest_client()
assert client.search_time_range == 24
assert client.fetch_time == 10
assert client.fetch_timeout == 30
assert client.fetch_navigation_timeout == 60
assert client.image_search_time_range == 7
assert client.image_size == "l"
@patch("deerflow.community.infoquest.infoquest_client.requests.post")
def test_web_search_api_error(self, mock_post):
"""Test web_search operation with API error."""
mock_post.side_effect = Exception("Connection error")
client = InfoQuestClient()
result = client.web_search("test query")
assert "Error" in result
def test_clean_results_with_image_search(self):
"""Test clean_results_with_image_search method with sample raw results."""
raw_results = [{"content": {"results": {"images_results": [{"original": "https://example.com/image1.jpg", "title": "Test Image 1", "url": "https://example.com/page1"}]}}}]
cleaned = InfoQuestClient.clean_results_with_image_search(raw_results)
assert len(cleaned) == 1
assert cleaned[0]["image_url"] == "https://example.com/image1.jpg"
assert cleaned[0]["title"] == "Test Image 1"
def test_clean_results_with_image_search_empty(self):
"""Test clean_results_with_image_search method with empty results."""
raw_results = [{"content": {"results": {"images_results": []}}}]
cleaned = InfoQuestClient.clean_results_with_image_search(raw_results)
assert len(cleaned) == 0
def test_clean_results_with_image_search_no_images(self):
"""Test clean_results_with_image_search method with no images_results field."""
raw_results = [{"content": {"results": {"organic": [{"title": "Test Page"}]}}}]
cleaned = InfoQuestClient.clean_results_with_image_search(raw_results)
assert len(cleaned) == 0
class TestImageSearch:
@patch("deerflow.community.infoquest.infoquest_client.requests.post")
def test_image_search_raw_results_success(self, mock_post):
"""Test successful image_search_raw_results operation."""
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {"search_result": {"results": [{"content": {"results": {"images_results": [{"original": "https://example.com/image1.jpg", "title": "Test Image", "url": "https://example.com/page1"}]}}}]}}
mock_post.return_value = mock_response
client = InfoQuestClient()
result = client.image_search_raw_results("test query")
assert "search_result" in result
mock_post.assert_called_once()
args, kwargs = mock_post.call_args
assert args[0] == "https://search.infoquest.bytepluses.com"
assert kwargs["json"]["query"] == "test query"
@patch("deerflow.community.infoquest.infoquest_client.requests.post")
def test_image_search_raw_results_with_parameters(self, mock_post):
"""Test image_search_raw_results with all parameters."""
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {"search_result": {"results": [{"content": {"results": {"images_results": [{"original": "https://example.com/image1.jpg"}]}}}]}}
mock_post.return_value = mock_response
client = InfoQuestClient(image_search_time_range=30, image_size="l")
client.image_search_raw_results(query="cat", site="unsplash.com", output_format="JSON")
mock_post.assert_called_once()
args, kwargs = mock_post.call_args
assert kwargs["json"]["query"] == "cat"
assert kwargs["json"]["time_range"] == 30
assert kwargs["json"]["site"] == "unsplash.com"
assert kwargs["json"]["image_size"] == "l"
assert kwargs["json"]["format"] == "JSON"
@patch("deerflow.community.infoquest.infoquest_client.requests.post")
def test_image_search_raw_results_invalid_time_range(self, mock_post):
"""Test image_search_raw_results with invalid time_range parameter."""
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {"search_result": {"results": [{"content": {"results": {"images_results": []}}}]}}
mock_post.return_value = mock_response
# Create client with invalid time_range (should be ignored)
client = InfoQuestClient(image_search_time_range=400, image_size="x")
client.image_search_raw_results(
query="test",
site="",
)
mock_post.assert_called_once()
args, kwargs = mock_post.call_args
assert kwargs["json"]["query"] == "test"
assert "time_range" not in kwargs["json"]
assert "image_size" not in kwargs["json"]
@patch("deerflow.community.infoquest.infoquest_client.requests.post")
def test_image_search_success(self, mock_post):
"""Test successful image_search operation."""
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {"search_result": {"results": [{"content": {"results": {"images_results": [{"original": "https://example.com/image1.jpg", "title": "Test Image", "url": "https://example.com/page1"}]}}}]}}
mock_post.return_value = mock_response
client = InfoQuestClient()
result = client.image_search("cat")
# Check if result is a valid JSON string with expected content
result_data = json.loads(result)
assert len(result_data) == 1
assert result_data[0]["image_url"] == "https://example.com/image1.jpg"
assert result_data[0]["title"] == "Test Image"
@patch("deerflow.community.infoquest.infoquest_client.requests.post")
def test_image_search_with_all_parameters(self, mock_post):
"""Test image_search with all optional parameters."""
mock_response = MagicMock()
mock_response.status_code = 200
mock_response.json.return_value = {"search_result": {"results": [{"content": {"results": {"images_results": [{"original": "https://example.com/image1.jpg"}]}}}]}}
mock_post.return_value = mock_response
# Create client with image search parameters
client = InfoQuestClient(image_search_time_range=7, image_size="m")
client.image_search(query="dog", site="flickr.com", output_format="JSON")
mock_post.assert_called_once()
args, kwargs = mock_post.call_args
assert kwargs["json"]["query"] == "dog"
assert kwargs["json"]["time_range"] == 7
assert kwargs["json"]["site"] == "flickr.com"
assert kwargs["json"]["image_size"] == "m"
@patch("deerflow.community.infoquest.infoquest_client.requests.post")
def test_image_search_api_error(self, mock_post):
"""Test image_search operation with API error."""
mock_post.side_effect = Exception("Connection error")
client = InfoQuestClient()
result = client.image_search("test query")
assert "Error" in result
@patch("deerflow.community.infoquest.tools._get_infoquest_client")
def test_image_search_tool(self, mock_get_client):
"""Test image_search_tool function."""
mock_client = MagicMock()
mock_client.image_search.return_value = json.dumps([{"image_url": "https://example.com/image1.jpg"}])
mock_get_client.return_value = mock_client
result = tools.image_search_tool.run({"query": "test query"})
# Check if result is a valid JSON string
result_data = json.loads(result)
assert len(result_data) == 1
assert result_data[0]["image_url"] == "https://example.com/image1.jpg"
mock_get_client.assert_called_once()
mock_client.image_search.assert_called_once_with("test query")
# In /Users/bytedance/python/deer-flowv2/deer-flow/backend/tests/test_infoquest_client.py
@patch("deerflow.community.infoquest.tools._get_infoquest_client")
def test_image_search_tool_with_parameters(self, mock_get_client):
"""Test image_search_tool function with all parameters (extra parameters will be ignored)."""
mock_client = MagicMock()
mock_client.image_search.return_value = json.dumps([{"image_url": "https://example.com/image1.jpg"}])
mock_get_client.return_value = mock_client
# Pass all parameters as a dictionary (extra parameters will be ignored)
tools.image_search_tool.run({"query": "sunset", "time_range": 30, "site": "unsplash.com", "image_size": "l"})
mock_get_client.assert_called_once()
# image_search_tool only passes query to client.image_search
# site parameter is empty string by default
mock_client.image_search.assert_called_once_with("sunset")